2024-04-26T21:52:58.770 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:52:58.771 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/orig.config.yaml 2024-04-26T21:52:58.824 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:52:58.915 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239 branch: wip-rishabh-testing-20240426.111959 description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} email: null first_in_suite: false job_id: '7675239' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: fs: - name: a - name: b session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - Replacing daemon mds - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: cephfs_name: a client.1: cephfs_name: b install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: cephfs_name: a client.1: cephfs_name: b syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSRV7yFXnXoPmx6i1rOJ/GZGdRyjHl9zExFiX7zxM6J8PBhpIElPz9pKRIDq2ILxIOSPO4RLKvmqhFS56vmXM4= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSO+8fZrkKoUsg+LCSjg08A1kMkizaMwDod6+Fl9CDNXbRdmn3s8sz/UWUt+rRx0Dc4Q0EtupklDqA1nGTlqgo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mds_thrash: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:52:58.915 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:52:58.916 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:52:58.916 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:52:58.917 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:52:58.939 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:52:58.939 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:52:58.939 INFO:teuthology.packaging:ref: None 2024-04-26T21:52:58.939 INFO:teuthology.packaging:tag: None 2024-04-26T21:52:58.939 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:52:58.939 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:52:58.939 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:52:59.188 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:52:59.189 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:52:59.196 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:52:59.196 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:52:59.209 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:52:59.229 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:52:59.234 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:52:59.259 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:47:51.093482', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSRV7yFXnXoPmx6i1rOJ/GZGdRyjHl9zExFiX7zxM6J8PBhpIElPz9pKRIDq2ILxIOSPO4RLKvmqhFS56vmXM4='} 2024-04-26T21:52:59.279 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:47:51.095196', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSO+8fZrkKoUsg+LCSjg08A1kMkizaMwDod6+Fl9CDNXbRdmn3s8sz/UWUt+rRx0Dc4Q0EtupklDqA1nGTlqgo='} 2024-04-26T21:52:59.279 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:52:59.285 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-26T21:52:59.285 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-04-26T21:52:59.285 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:52:59.356 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff73d857160>, signals=[15]) 2024-04-26T21:52:59.356 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:52:59.362 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:52:59.362 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T21:52:59.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:52:59.438 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T21:52:59.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:52:59.513 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:52:59.519 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-04-26T21:52:59.539 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-04-26T21:52:59.539 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-04-26T21:52:59.595 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-04-26T21:52:59.595 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-04-26T21:52:59.595 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-04-26T21:52:59.595 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:52:59.595 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-04-26T21:52:59.595 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:52:59.596 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:52:59.597 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-04-26T21:52:59.620 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-04-26T21:52:59.638 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-04-26T21:52:59.638 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:52:59.696 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-04-26T21:52:59.697 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:52:59.697 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:52:59.697 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:52:59.697 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:52:59.697 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:52:59.697 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:52:59.697 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-04-26T21:52:59.719 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:52:59.729 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:52:59.734 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:52:59.734 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:52:59.737 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:52:59.753 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:52:59.758 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:52:59.758 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:52:59.794 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:52:59.908 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:52:59.964 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:52:59.978 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:52:59.978 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-04-26T21:52:59.978 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T21:52:59.978 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:52:59.979 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:52:59.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:52:59.995 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-04-26T21:52:59.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:52:59.996 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-04-26T21:53:00.052 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:53:00.052 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-437.el9.x86_64 2024-04-26T21:53:00.053 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2024-04-26T21:53:00.054 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:53:00.054 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-437.el9.x86_64 2024-04-26T21:53:00.054 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-04-26T21:53:02.276 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T21:53:03.651 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:02 2024-04-26T21:53:04.471 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T21:53:05.501 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T21:53:08.569 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-26T21:53:10.069 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.2 MB 00:01 2024-04-26T21:53:10.161 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-26T21:53:11.470 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-26T21:53:12.530 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T21:53:13.762 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 6.0 MB/s | 21 MB 00:03 2024-04-26T21:53:17.666 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T21:53:18.870 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T21:53:19.520 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:53:19.520 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:53:19.610 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-26T21:53:19.611 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-26T21:53:19.611 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-26T21:53:19.723 DEBUG:teuthology.orchestra.run.smithi135:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:53:20.220 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:53:17 PM UTC. 2024-04-26T21:53:20.339 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout:Reinstalling: 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:53:20.375 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 255 k 2024-04-26T21:53:20.376 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 0 2024-04-26T21:53:20.376 INFO:teuthology.orchestra.run.smithi135.stderr:Operation aborted. 2024-04-26T21:53:20.438 INFO:teuthology.orchestra.run.smithi135.stdout:Is this ok [y/N]: 2024-04-26T21:53:20.438 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum reinstall -y kernel || true 2024-04-26T21:53:20.776 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:53:20.776 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:53:20.885 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-26T21:53:20.886 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-04-26T21:53:20.886 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-26T21:53:20.930 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:53:17 PM UTC. 2024-04-26T21:53:21.025 DEBUG:teuthology.orchestra.run.smithi139:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:53:21.046 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout:Reinstalling: 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-26T21:53:21.047 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:53:21.048 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 255 k 2024-04-26T21:53:21.048 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 0 2024-04-26T21:53:21.048 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-26T21:53:21.400 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-26T21:53:21.400 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:53:21.400 INFO:teuthology.orchestra.run.smithi135.stdout:Total 722 kB/s | 255 kB 00:00 2024-04-26T21:53:21.415 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-26T21:53:21.525 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:53:18 PM UTC. 2024-04-26T21:53:21.599 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-26T21:53:21.599 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-26T21:53:21.604 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-26T21:53:21.604 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-26T21:53:21.651 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-26T21:53:21.666 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalling: 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 255 k 2024-04-26T21:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 0 2024-04-26T21:53:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:Operation aborted. 2024-04-26T21:53:21.740 INFO:teuthology.orchestra.run.smithi139.stdout:Is this ok [y/N]: 2024-04-26T21:53:21.740 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum reinstall -y kernel || true 2024-04-26T21:53:21.741 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-26T21:53:21.801 INFO:teuthology.orchestra.run.smithi135.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:53:21.801 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:53:22.238 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 09:53:18 PM UTC. 2024-04-26T21:53:22.354 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-26T21:53:22.354 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalling: 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 255 k 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 0 2024-04-26T21:53:22.355 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-04-26T21:53:22.556 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:53:22.556 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:53:23.088 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-437.el9.x86_64.rpm 563 kB/s | 255 kB 00:00 2024-04-26T21:53:23.088 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:53:23.089 INFO:teuthology.orchestra.run.smithi139.stdout:Total 347 kB/s | 255 kB 00:00 2024-04-26T21:53:23.109 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-04-26T21:53:23.286 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:53:23.286 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:53:23.287 INFO:teuthology.orchestra.run.smithi135.stdout:Reinstalled: 2024-04-26T21:53:23.287 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:53:23.287 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:53:23.287 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-26T21:53:23.302 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-04-26T21:53:23.302 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-04-26T21:53:23.307 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-04-26T21:53:23.308 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-04-26T21:53:23.438 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-04-26T21:53:23.455 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:53:23.501 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:53:23.501 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:53:23.502 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:53:23.502 DEBUG:teuthology.parallel:result is None 2024-04-26T21:53:23.514 INFO:teuthology.orchestra.run.smithi139.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:53:23.514 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:53:24.267 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:53:24.267 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:53:24.963 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:53:24.963 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:53:24.963 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalled: 2024-04-26T21:53:24.964 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:53:24.964 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:53:24.964 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-26T21:53:25.134 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:53:25.180 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:53:25.181 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:53:25.181 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:53:25.182 DEBUG:teuthology.parallel:result is None 2024-04-26T21:53:25.182 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:53:25.189 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:53:25.189 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:53:25.192 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:53:25.207 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:53:25.212 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:53:25.219 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:53:25.219 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:53:25.249 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:53:25.275 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:53:25.281 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:53:25.281 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:53:25.311 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:53:25.342 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:53:25.348 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:53:25.361 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:53:25.376 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:53:25.377 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:53:25.384 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:53:25.385 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:53:25.406 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:53:25.441 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:53:25.449 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:53:25.449 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:53:25.473 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:53:25.496 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:53:25.567 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:53:25.614 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:53:25.695 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:53:25.735 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:53:25.735 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:53:25.799 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:53:25.855 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:53:25.886 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:53:25.960 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:53:26.007 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:53:26.007 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:53:26.070 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-04-26T21:53:26.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-04-26T21:53:26.105 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:53:26.146 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:53:26.650 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:53:26.658 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:53:26.658 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:53:26.669 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:53:26.677 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:53:26.678 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-04-26T21:53:26.740 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-04-26T21:53:26.742 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-04-26T21:53:26.792 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-04-26T21:53:26.794 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:53:26.794 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-04-26T21:53:26.818 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-04-26T21:53:26.819 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-04-26T21:53:26.863 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-04-26T21:53:26.864 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi135.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:53:26.864 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:53:26.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:53:26.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:53:26.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:53:26.928 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:53:26.928 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:53:26.963 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:53:27.003 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:53:27.012 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:53:27.028 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:53:27.029 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi135.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:56:55.315 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi135.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-04-26T21:56:55.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-26T21:56:55.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:56:55.392 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-04-26T21:56:55.424 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-26T21:56:55.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-04-26T21:56:55.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:56:55.498 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-04-26T21:56:55.577 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-04-26T21:56:55.577 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:56:55.588 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:56:55.588 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:56:55.588 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:56:55.590 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:56:55.591 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:56:55.615 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:56:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:56:55.645 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:56:55.651 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-04-26T21:56:55.658 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:56:55.661 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-04-26T21:56:55.673 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:56:55.680 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-04-26T21:56:55.686 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:56:55.690 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-04-26T21:56:55.703 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:56:55.715 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:56:55.744 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:56:55.748 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:56:55.748 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-04-26T21:56:55.748 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.748 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.748 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.748 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.771 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:56:55.774 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:56:55.774 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-04-26T21:56:55.774 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.775 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.775 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.775 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:56:55.775 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:56:55.783 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:56:55.783 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:56:55.783 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:56:55.784 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:56:55.789 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:56:55.789 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:56:55.790 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:56:55.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:56:55.792 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:56:55.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:56:55.944 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:56:55.944 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:56:56.015 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:56:56.015 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:56:56.062 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:56:56.062 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:56:56.062 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:56:56.087 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:56:56.087 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:56:56.127 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:56:56.128 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:56:56.128 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:56:56.152 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:56:56.152 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:56:56.153 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:56:56.216 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:56:56.227 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:56:56.288 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:56:56.289 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-04-26T21:56:56.291 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-04-26T21:56:56.351 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-04-26T21:56:56.353 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-04-26T21:56:56.513 INFO:teuthology.orchestra.run.smithi139.stdout:48 files removed 2024-04-26T21:56:56.540 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:56:56.576 INFO:teuthology.orchestra.run.smithi135.stdout:48 files removed 2024-04-26T21:56:56.603 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:56:57.123 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 258 kB/s | 84 kB 00:00 2024-04-26T21:56:57.270 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-04-26T21:56:57.540 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T21:56:57.621 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-04-26T21:56:57.841 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 7.2 kB/s | 1.9 kB 00:00 2024-04-26T21:56:57.972 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-26T21:56:58.457 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:56:58.497 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T21:57:00.258 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-26T21:57:04.276 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-26T21:57:06.050 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T21:57:12.124 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 19 MB 00:12 2024-04-26T21:57:22.540 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 928 kB/s | 6.2 MB 00:06 2024-04-26T21:57:23.966 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-26T21:57:25.020 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-26T21:57:25.579 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T21:57:30.033 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-26T21:57:30.551 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-26T21:57:32.250 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:57:32.251 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:57:32.262 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:57:32.262 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:57:32.263 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:57:32.264 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:57:32.266 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:57:32.268 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:57:32.269 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:57:32.328 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-26T21:57:32.337 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-26T21:57:32.337 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-04-26T21:57:32.337 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-26T21:57:32.337 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:57:32.338 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:57:32.339 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:57:32.340 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:57:32.341 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:57:32.342 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:57:32.343 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:57:32.344 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:57:32.345 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:57:32.346 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:57:32.347 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:57:32.348 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:57:32.349 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:57:32.350 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 4 Packages 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:57:32.351 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 222 M 2024-04-26T21:57:32.352 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-04-26T21:57:32.662 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:57:32.662 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:57:32.672 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:57:32.673 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:57:32.674 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:57:32.675 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:57:32.676 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:57:32.678 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:57:32.679 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:57:32.739 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:57:32.748 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:57:32.749 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:57:32.750 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:57:32.751 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:57:32.752 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:57:32.753 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:57:32.754 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:57:32.755 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:57:32.756 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:57:32.757 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:57:32.758 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:57:32.759 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:57:32.760 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-04-26T21:57:32.761 INFO:teuthology.orchestra.run.smithi135.stdout:Install 148 Packages 2024-04-26T21:57:32.762 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 4 Packages 2024-04-26T21:57:32.762 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:57:32.763 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 222 M 2024-04-26T21:57:32.763 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-26T21:57:33.326 INFO:teuthology.orchestra.run.smithi139.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-26T21:57:33.701 INFO:teuthology.orchestra.run.smithi139.stdout:(2/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 8.8 MB/s | 5.4 MB 00:00 2024-04-26T21:57:33.735 INFO:teuthology.orchestra.run.smithi139.stdout:(3/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.1 MB/s | 889 kB 00:00 2024-04-26T21:57:33.760 INFO:teuthology.orchestra.run.smithi139.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 2.4 MB/s | 145 kB 00:00 2024-04-26T21:57:33.919 INFO:teuthology.orchestra.run.smithi139.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-04-26T21:57:34.036 INFO:teuthology.orchestra.run.smithi135.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-26T21:57:34.319 INFO:teuthology.orchestra.run.smithi139.stdout:(6/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 4.0 MB/s | 2.4 MB 00:00 2024-04-26T21:57:34.386 INFO:teuthology.orchestra.run.smithi139.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 10 MB/s | 4.7 MB 00:00 2024-04-26T21:57:35.162 INFO:teuthology.orchestra.run.smithi135.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 791 kB/s | 889 kB 00:01 2024-04-26T21:57:35.428 INFO:teuthology.orchestra.run.smithi135.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 544 kB/s | 145 kB 00:00 2024-04-26T21:57:35.728 INFO:teuthology.orchestra.run.smithi139.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 7.9 MB/s | 11 MB 00:01 2024-04-26T21:57:35.928 INFO:teuthology.orchestra.run.smithi139.stdout:(9/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 7.4 MB/s | 21 MB 00:02 2024-04-26T21:57:35.953 INFO:teuthology.orchestra.run.smithi139.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 112 kB/s | 25 kB 00:00 2024-04-26T21:57:36.012 INFO:teuthology.orchestra.run.smithi139.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.7 MB/s | 225 kB 00:00 2024-04-26T21:57:36.095 INFO:teuthology.orchestra.run.smithi139.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 325 kB/s | 27 kB 00:00 2024-04-26T21:57:36.254 INFO:teuthology.orchestra.run.smithi139.stdout:(13/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 8.7 MB/s | 17 MB 00:01 2024-04-26T21:57:36.271 INFO:teuthology.orchestra.run.smithi139.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 181 kB/s | 32 kB 00:00 2024-04-26T21:57:36.346 INFO:teuthology.orchestra.run.smithi139.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.1 MB/s | 163 kB 00:00 2024-04-26T21:57:36.371 INFO:teuthology.orchestra.run.smithi139.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 6.2 MB/s | 747 kB 00:00 2024-04-26T21:57:36.397 INFO:teuthology.orchestra.run.smithi139.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T21:57:36.429 INFO:teuthology.orchestra.run.smithi135.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-26T21:57:36.456 INFO:teuthology.orchestra.run.smithi139.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 5.8 MB/s | 501 kB 00:00 2024-04-26T21:57:36.548 INFO:teuthology.orchestra.run.smithi139.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 492 kB/s | 45 kB 00:00 2024-04-26T21:57:36.656 INFO:teuthology.orchestra.run.smithi139.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:57:36.798 INFO:teuthology.orchestra.run.smithi139.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.1 MB/s | 161 kB 00:00 2024-04-26T21:57:36.862 INFO:teuthology.orchestra.run.smithi135.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-26T21:57:36.890 INFO:teuthology.orchestra.run.smithi139.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.4 MB/s | 321 kB 00:00 2024-04-26T21:57:37.124 INFO:teuthology.orchestra.run.smithi139.stdout:(23/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.4 MB/s | 5.4 MB 00:00 2024-04-26T21:57:37.157 INFO:teuthology.orchestra.run.smithi139.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 298 kB 00:00 2024-04-26T21:57:37.174 INFO:teuthology.orchestra.run.smithi139.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 99 kB 00:00 2024-04-26T21:57:37.233 INFO:teuthology.orchestra.run.smithi139.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T21:57:37.350 INFO:teuthology.orchestra.run.smithi139.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.4 MB/s | 171 kB 00:00 2024-04-26T21:57:37.450 INFO:teuthology.orchestra.run.smithi139.stdout:(28/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 11 MB/s | 3.1 MB 00:00 2024-04-26T21:57:37.475 INFO:teuthology.orchestra.run.smithi139.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 273 kB/s | 34 kB 00:00 2024-04-26T21:57:37.501 INFO:teuthology.orchestra.run.smithi139.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.8 MB/s | 142 kB 00:00 2024-04-26T21:57:37.784 INFO:teuthology.orchestra.run.smithi139.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 11 MB/s | 3.5 MB 00:00 2024-04-26T21:57:37.804 INFO:teuthology.orchestra.run.smithi135.stdout:(6/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.0 MB/s | 2.4 MB 00:02 2024-04-26T21:57:37.859 INFO:teuthology.orchestra.run.smithi139.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.3 MB/s | 249 kB 00:00 2024-04-26T21:57:37.926 INFO:teuthology.orchestra.run.smithi139.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 740 kB/s | 49 kB 00:00 2024-04-26T21:57:38.002 INFO:teuthology.orchestra.run.smithi139.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 220 kB/s | 16 kB 00:00 2024-04-26T21:57:38.063 INFO:teuthology.orchestra.run.smithi135.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-04-26T21:57:38.085 INFO:teuthology.orchestra.run.smithi139.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 273 kB 00:00 2024-04-26T21:57:38.202 INFO:teuthology.orchestra.run.smithi139.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-26T21:57:38.236 INFO:teuthology.orchestra.run.smithi139.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T21:57:38.261 INFO:teuthology.orchestra.run.smithi139.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 314 kB/s | 18 kB 00:00 2024-04-26T21:57:38.886 INFO:teuthology.orchestra.run.smithi139.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 123 kB/s | 80 kB 00:00 2024-04-26T21:57:39.653 INFO:teuthology.orchestra.run.smithi139.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 255 kB/s | 195 kB 00:00 2024-04-26T21:57:39.687 INFO:teuthology.orchestra.run.smithi139.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 563 kB/s | 803 kB 00:01 2024-04-26T21:57:39.963 INFO:teuthology.orchestra.run.smithi135.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 5.6 MB/s | 11 MB 00:01 2024-04-26T21:57:40.013 INFO:teuthology.orchestra.run.smithi135.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 506 kB/s | 25 kB 00:00 2024-04-26T21:57:40.470 INFO:teuthology.orchestra.run.smithi139.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 310 kB/s | 253 kB 00:00 2024-04-26T21:57:41.104 INFO:teuthology.orchestra.run.smithi139.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 168 kB/s | 106 kB 00:00 2024-04-26T21:57:41.596 INFO:teuthology.orchestra.run.smithi139.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 658 kB/s | 1.2 MB 00:01 2024-04-26T21:57:41.813 INFO:teuthology.orchestra.run.smithi139.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2024-04-26T21:57:42.005 INFO:teuthology.orchestra.run.smithi135.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 21 MB 00:08 2024-04-26T21:57:42.114 INFO:teuthology.orchestra.run.smithi135.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.0 MB/s | 225 kB 00:00 2024-04-26T21:57:42.172 INFO:teuthology.orchestra.run.smithi135.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 465 kB/s | 27 kB 00:00 2024-04-26T21:57:42.239 INFO:teuthology.orchestra.run.smithi135.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 476 kB/s | 32 kB 00:00 2024-04-26T21:57:42.304 INFO:teuthology.orchestra.run.smithi139.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-04-26T21:57:42.465 INFO:teuthology.orchestra.run.smithi135.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.2 MB/s | 747 kB 00:00 2024-04-26T21:57:42.513 INFO:teuthology.orchestra.run.smithi139.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-26T21:57:42.531 INFO:teuthology.orchestra.run.smithi135.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 163 kB 00:00 2024-04-26T21:57:42.582 INFO:teuthology.orchestra.run.smithi135.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T21:57:42.588 INFO:teuthology.orchestra.run.smithi139.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-26T21:57:42.613 INFO:teuthology.orchestra.run.smithi139.stdout:(49/152): python3-urllib3-1.26.5-5.el9.noarch.r 277 kB/s | 215 kB 00:00 2024-04-26T21:57:42.639 INFO:teuthology.orchestra.run.smithi139.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T21:57:42.697 INFO:teuthology.orchestra.run.smithi139.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-26T21:57:42.723 INFO:teuthology.orchestra.run.smithi139.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-26T21:57:42.724 INFO:teuthology.orchestra.run.smithi135.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 3.5 MB/s | 501 kB 00:00 2024-04-26T21:57:42.748 INFO:teuthology.orchestra.run.smithi139.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-04-26T21:57:42.790 INFO:teuthology.orchestra.run.smithi139.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-04-26T21:57:42.832 INFO:teuthology.orchestra.run.smithi139.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-26T21:57:42.874 INFO:teuthology.orchestra.run.smithi139.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-26T21:57:42.949 INFO:teuthology.orchestra.run.smithi139.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-26T21:57:42.974 INFO:teuthology.orchestra.run.smithi139.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-26T21:57:43.016 INFO:teuthology.orchestra.run.smithi139.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-26T21:57:43.050 INFO:teuthology.orchestra.run.smithi139.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-26T21:57:43.075 INFO:teuthology.orchestra.run.smithi139.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-26T21:57:43.101 INFO:teuthology.orchestra.run.smithi139.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-04-26T21:57:43.126 INFO:teuthology.orchestra.run.smithi139.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-26T21:57:43.201 INFO:teuthology.orchestra.run.smithi139.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-26T21:57:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-04-26T21:57:43.360 INFO:teuthology.orchestra.run.smithi139.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-26T21:57:43.427 INFO:teuthology.orchestra.run.smithi139.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-26T21:57:43.469 INFO:teuthology.orchestra.run.smithi139.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-26T21:57:43.510 INFO:teuthology.orchestra.run.smithi139.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi139.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T21:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 2.8 MB/s | 17 MB 00:06 2024-04-26T21:57:43.908 INFO:teuthology.orchestra.run.smithi135.stdout:(19/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-04-26T21:57:43.934 INFO:teuthology.orchestra.run.smithi135.stdout:(20/152): python3-ceph-argparse-19.0.0-3306.gbc 447 kB/s | 45 kB 00:00 2024-04-26T21:57:43.963 INFO:teuthology.orchestra.run.smithi139.stdout:(71/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 6.1 MB/s | 49 MB 00:08 2024-04-26T21:57:43.967 INFO:teuthology.orchestra.run.smithi135.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 2.2 MB/s | 135 kB 00:00 2024-04-26T21:57:43.996 INFO:teuthology.orchestra.run.smithi139.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-04-26T21:57:44.001 INFO:teuthology.orchestra.run.smithi135.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T21:57:44.068 INFO:teuthology.orchestra.run.smithi135.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.1 MB/s | 321 kB 00:00 2024-04-26T21:57:44.102 INFO:teuthology.orchestra.run.smithi135.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 298 kB 00:00 2024-04-26T21:57:44.114 INFO:teuthology.orchestra.run.smithi139.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 9.6 MB/s | 5.8 MB 00:00 2024-04-26T21:57:44.136 INFO:teuthology.orchestra.run.smithi135.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.4 MB/s | 99 kB 00:00 2024-04-26T21:57:44.147 INFO:teuthology.orchestra.run.smithi139.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi135.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T21:57:44.172 INFO:teuthology.orchestra.run.smithi139.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T21:57:44.231 INFO:teuthology.orchestra.run.smithi139.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-26T21:57:44.262 INFO:teuthology.orchestra.run.smithi135.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-26T21:57:44.281 INFO:teuthology.orchestra.run.smithi139.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-26T21:57:44.320 INFO:teuthology.orchestra.run.smithi135.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 588 kB/s | 34 kB 00:00 2024-04-26T21:57:44.331 INFO:teuthology.orchestra.run.smithi139.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-26T21:57:44.373 INFO:teuthology.orchestra.run.smithi139.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 835 kB/s | 35 kB 00:00 2024-04-26T21:57:44.387 INFO:teuthology.orchestra.run.smithi135.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.1 MB/s | 142 kB 00:00 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi139.stdout:(80/152): python3-jsonpatch-1.21-16.el9.noarch. 113 kB/s | 26 kB 00:00 2024-04-26T21:57:44.491 INFO:teuthology.orchestra.run.smithi139.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-26T21:57:44.516 INFO:teuthology.orchestra.run.smithi139.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-26T21:57:44.541 INFO:teuthology.orchestra.run.smithi139.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-26T21:57:44.566 INFO:teuthology.orchestra.run.smithi139.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T21:57:44.591 INFO:teuthology.orchestra.run.smithi139.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-26T21:57:44.617 INFO:teuthology.orchestra.run.smithi139.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-26T21:57:44.725 INFO:teuthology.orchestra.run.smithi139.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-26T21:57:44.742 INFO:teuthology.orchestra.run.smithi139.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-26T21:57:44.768 INFO:teuthology.orchestra.run.smithi139.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-04-26T21:57:44.771 INFO:teuthology.orchestra.run.smithi135.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.9 MB/s | 3.1 MB 00:00 2024-04-26T21:57:44.793 INFO:teuthology.orchestra.run.smithi139.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 762 kB/s | 51 kB 00:00 2024-04-26T21:57:44.818 INFO:teuthology.orchestra.run.smithi139.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-04-26T21:57:44.868 INFO:teuthology.orchestra.run.smithi139.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 890 kB/s | 44 kB 00:00 2024-04-26T21:57:44.893 INFO:teuthology.orchestra.run.smithi139.stdout:(93/152): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-04-26T21:57:44.944 INFO:teuthology.orchestra.run.smithi139.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-04-26T21:57:44.994 INFO:teuthology.orchestra.run.smithi139.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 953 kB/s | 47 kB 00:00 2024-04-26T21:57:45.036 INFO:teuthology.orchestra.run.smithi139.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-04-26T21:57:45.186 INFO:teuthology.orchestra.run.smithi139.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-26T21:57:45.211 INFO:teuthology.orchestra.run.smithi139.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-04-26T21:57:45.237 INFO:teuthology.orchestra.run.smithi139.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 289 kB/s | 58 kB 00:00 2024-04-26T21:57:45.238 INFO:teuthology.orchestra.run.smithi135.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 4.2 MB/s | 3.5 MB 00:00 2024-04-26T21:57:45.262 INFO:teuthology.orchestra.run.smithi139.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T21:57:45.287 INFO:teuthology.orchestra.run.smithi139.stdout:(101/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 935 kB/s | 94 kB 00:00 2024-04-26T21:57:45.313 INFO:teuthology.orchestra.run.smithi139.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 592 kB/s | 45 kB 00:00 2024-04-26T21:57:45.355 INFO:teuthology.orchestra.run.smithi135.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.1 MB/s | 249 kB 00:00 2024-04-26T21:57:45.363 INFO:teuthology.orchestra.run.smithi139.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 221 kB/s | 22 kB 00:00 2024-04-26T21:57:45.438 INFO:teuthology.orchestra.run.smithi135.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 593 kB/s | 49 kB 00:00 2024-04-26T21:57:45.463 INFO:teuthology.orchestra.run.smithi139.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-26T21:57:45.506 INFO:teuthology.orchestra.run.smithi135.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 246 kB/s | 16 kB 00:00 2024-04-26T21:57:45.631 INFO:teuthology.orchestra.run.smithi135.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T21:57:45.788 INFO:teuthology.orchestra.run.smithi139.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-04-26T21:57:45.847 INFO:teuthology.orchestra.run.smithi139.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-26T21:57:45.872 INFO:teuthology.orchestra.run.smithi139.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 640 kB/s | 309 kB 00:00 2024-04-26T21:57:45.898 INFO:teuthology.orchestra.run.smithi135.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 2.8 MB/s | 762 kB 00:00 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi135.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 274 kB/s | 18 kB 00:00 2024-04-26T21:57:46.131 INFO:teuthology.orchestra.run.smithi139.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 92 kB/s | 26 kB 00:00 2024-04-26T21:57:46.239 INFO:teuthology.orchestra.run.smithi139.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 132 kB/s | 49 kB 00:00 2024-04-26T21:57:46.249 INFO:teuthology.orchestra.run.smithi135.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 5.0 MB/s | 7.4 MB 00:01 2024-04-26T21:57:47.057 INFO:teuthology.orchestra.run.smithi135.stdout:(39/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 7.0 MB/s | 49 MB 00:07 2024-04-26T21:57:47.083 INFO:teuthology.orchestra.run.smithi135.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 72 kB/s | 80 kB 00:01 2024-04-26T21:57:47.406 INFO:teuthology.orchestra.run.smithi139.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 53 kB/s | 67 kB 00:01 2024-04-26T21:57:47.531 INFO:teuthology.orchestra.run.smithi139.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 117 kB/s | 151 kB 00:01 2024-04-26T21:57:47.810 INFO:teuthology.orchestra.run.smithi135.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 260 kB/s | 195 kB 00:00 2024-04-26T21:57:47.910 INFO:teuthology.orchestra.run.smithi135.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 306 kB/s | 253 kB 00:00 2024-04-26T21:57:47.931 INFO:teuthology.orchestra.run.smithi139.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.6 MB/s | 839 kB 00:00 2024-04-26T21:57:48.057 INFO:teuthology.orchestra.run.smithi139.stdout:(113/152): libarrow-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-04-26T21:57:48.077 INFO:teuthology.orchestra.run.smithi135.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 446 kB/s | 803 kB 00:01 2024-04-26T21:57:48.082 INFO:teuthology.orchestra.run.smithi139.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 996 kB/s | 548 kB 00:00 2024-04-26T21:57:48.107 INFO:teuthology.orchestra.run.smithi139.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 247 kB/s | 43 kB 00:00 2024-04-26T21:57:48.183 INFO:teuthology.orchestra.run.smithi139.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 256 kB/s | 32 kB 00:00 2024-04-26T21:57:48.208 INFO:teuthology.orchestra.run.smithi139.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-04-26T21:57:48.233 INFO:teuthology.orchestra.run.smithi139.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-26T21:57:48.283 INFO:teuthology.orchestra.run.smithi139.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-26T21:57:48.300 INFO:teuthology.orchestra.run.smithi139.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-04-26T21:57:48.325 INFO:teuthology.orchestra.run.smithi139.stdout:(121/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-26T21:57:48.384 INFO:teuthology.orchestra.run.smithi139.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-26T21:57:48.401 INFO:teuthology.orchestra.run.smithi139.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-26T21:57:48.451 INFO:teuthology.orchestra.run.smithi139.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-26T21:57:48.476 INFO:teuthology.orchestra.run.smithi139.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-26T21:57:48.527 INFO:teuthology.orchestra.run.smithi139.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-04-26T21:57:48.552 INFO:teuthology.orchestra.run.smithi139.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-26T21:57:48.569 INFO:teuthology.orchestra.run.smithi135.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 162 kB/s | 106 kB 00:00 2024-04-26T21:57:48.577 INFO:teuthology.orchestra.run.smithi139.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-26T21:57:48.627 INFO:teuthology.orchestra.run.smithi139.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-26T21:57:48.669 INFO:teuthology.orchestra.run.smithi139.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-04-26T21:57:48.711 INFO:teuthology.orchestra.run.smithi139.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-04-26T21:57:48.736 INFO:teuthology.orchestra.run.smithi139.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-04-26T21:57:48.770 INFO:teuthology.orchestra.run.smithi139.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-04-26T21:57:48.778 INFO:teuthology.orchestra.run.smithi135.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 193 kB/s | 135 kB 00:00 2024-04-26T21:57:48.804 INFO:teuthology.orchestra.run.smithi139.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T21:57:48.837 INFO:teuthology.orchestra.run.smithi139.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-26T21:57:48.863 INFO:teuthology.orchestra.run.smithi139.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 974 kB/s | 90 kB 00:00 2024-04-26T21:57:48.888 INFO:teuthology.orchestra.run.smithi139.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 336 kB/s | 28 kB 00:00 2024-04-26T21:57:48.913 INFO:teuthology.orchestra.run.smithi139.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-26T21:57:48.938 INFO:teuthology.orchestra.run.smithi139.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T21:57:48.972 INFO:teuthology.orchestra.run.smithi139.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-26T21:57:48.997 INFO:teuthology.orchestra.run.smithi139.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-26T21:57:49.023 INFO:teuthology.orchestra.run.smithi139.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-26T21:57:49.056 INFO:teuthology.orchestra.run.smithi139.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T21:57:49.082 INFO:teuthology.orchestra.run.smithi139.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-26T21:57:49.115 INFO:teuthology.orchestra.run.smithi139.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-26T21:57:49.140 INFO:teuthology.orchestra.run.smithi139.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-26T21:57:49.165 INFO:teuthology.orchestra.run.smithi139.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-26T21:57:49.257 INFO:teuthology.orchestra.run.smithi139.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T21:57:49.278 INFO:teuthology.orchestra.run.smithi135.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-04-26T21:57:49.491 INFO:teuthology.orchestra.run.smithi139.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-04-26T21:57:49.503 INFO:teuthology.orchestra.run.smithi135.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-26T21:57:49.554 INFO:teuthology.orchestra.run.smithi135.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 287 kB/s | 215 kB 00:00 2024-04-26T21:57:49.583 INFO:teuthology.orchestra.run.smithi139.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-26T21:57:49.587 INFO:teuthology.orchestra.run.smithi135.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-26T21:57:49.654 INFO:teuthology.orchestra.run.smithi135.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-04-26T21:57:49.705 INFO:teuthology.orchestra.run.smithi135.stdout:(51/152): python3-cryptography-36.0.1-4.el9.x86 663 kB/s | 1.2 MB 00:01 2024-04-26T21:57:49.730 INFO:teuthology.orchestra.run.smithi135.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 607 kB/s | 107 kB 00:00 2024-04-26T21:57:49.756 INFO:teuthology.orchestra.run.smithi135.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-04-26T21:57:49.798 INFO:teuthology.orchestra.run.smithi135.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-04-26T21:57:49.823 INFO:teuthology.orchestra.run.smithi135.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-04-26T21:57:49.848 INFO:teuthology.orchestra.run.smithi135.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-26T21:57:49.874 INFO:teuthology.orchestra.run.smithi135.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T21:57:49.899 INFO:teuthology.orchestra.run.smithi135.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-26T21:57:49.933 INFO:teuthology.orchestra.run.smithi135.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T21:57:49.983 INFO:teuthology.orchestra.run.smithi135.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 510 kB/s | 25 kB 00:00 2024-04-26T21:57:50.042 INFO:teuthology.orchestra.run.smithi135.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.6 MB/s | 3.0 MB 00:00 2024-04-26T21:57:50.067 INFO:teuthology.orchestra.run.smithi135.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-26T21:57:50.092 INFO:teuthology.orchestra.run.smithi135.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-04-26T21:57:50.118 INFO:teuthology.orchestra.run.smithi135.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-26T21:57:50.143 INFO:teuthology.orchestra.run.smithi135.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-04-26T21:57:50.168 INFO:teuthology.orchestra.run.smithi135.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-26T21:57:50.244 INFO:teuthology.orchestra.run.smithi135.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-26T21:57:50.328 INFO:teuthology.orchestra.run.smithi135.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-04-26T21:57:50.370 INFO:teuthology.orchestra.run.smithi135.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-26T21:57:50.398 INFO:teuthology.orchestra.run.smithi135.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T21:57:50.479 INFO:teuthology.orchestra.run.smithi135.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-26T21:57:50.571 INFO:teuthology.orchestra.run.smithi135.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-26T21:57:50.596 INFO:teuthology.orchestra.run.smithi135.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-04-26T21:57:50.663 INFO:teuthology.orchestra.run.smithi135.stdout:(74/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-04-26T21:57:50.688 INFO:teuthology.orchestra.run.smithi135.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-26T21:57:50.713 INFO:teuthology.orchestra.run.smithi135.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-26T21:57:50.730 INFO:teuthology.orchestra.run.smithi135.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 609 kB/s | 26 kB 00:00 2024-04-26T21:57:50.764 INFO:teuthology.orchestra.run.smithi135.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 379 kB/s | 19 kB 00:00 2024-04-26T21:57:50.873 INFO:teuthology.orchestra.run.smithi135.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-26T21:57:50.899 INFO:teuthology.orchestra.run.smithi135.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 259 kB/s | 35 kB 00:00 2024-04-26T21:57:50.924 INFO:teuthology.orchestra.run.smithi135.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 892 kB/s | 172 kB 00:00 2024-04-26T21:57:51.274 INFO:teuthology.orchestra.run.smithi135.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-26T21:57:51.332 INFO:teuthology.orchestra.run.smithi135.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T21:57:51.392 INFO:teuthology.orchestra.run.smithi135.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-04-26T21:57:51.450 INFO:teuthology.orchestra.run.smithi135.stdout:(85/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 454 kB/s | 250 kB 00:00 2024-04-26T21:57:51.484 INFO:teuthology.orchestra.run.smithi135.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-26T21:57:51.509 INFO:teuthology.orchestra.run.smithi135.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-26T21:57:51.542 INFO:teuthology.orchestra.run.smithi135.stdout:(88/152): python3-oauthlib-3.1.1-5.el9.noarch.r 359 kB/s | 222 kB 00:00 2024-04-26T21:57:51.568 INFO:teuthology.orchestra.run.smithi135.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 866 kB/s | 51 kB 00:00 2024-04-26T21:57:51.594 INFO:teuthology.orchestra.run.smithi135.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-26T21:57:51.619 INFO:teuthology.orchestra.run.smithi135.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 706 kB/s | 54 kB 00:00 2024-04-26T21:57:51.626 INFO:teuthology.orchestra.run.smithi139.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 584 kB/s | 1.3 MB 00:02 2024-04-26T21:57:51.653 INFO:teuthology.orchestra.run.smithi135.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-04-26T21:57:51.678 INFO:teuthology.orchestra.run.smithi135.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 751 kB/s | 44 kB 00:00 2024-04-26T21:57:51.720 INFO:teuthology.orchestra.run.smithi135.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T21:57:51.746 INFO:teuthology.orchestra.run.smithi135.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 535 kB/s | 36 kB 00:00 2024-04-26T21:57:51.787 INFO:teuthology.orchestra.run.smithi135.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-26T21:57:51.821 INFO:teuthology.orchestra.run.smithi135.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 305 kB 00:00 2024-04-26T21:57:51.855 INFO:teuthology.orchestra.run.smithi135.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-04-26T21:57:51.888 INFO:teuthology.orchestra.run.smithi135.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-26T21:57:51.914 INFO:teuthology.orchestra.run.smithi135.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T21:57:51.947 INFO:teuthology.orchestra.run.smithi135.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-04-26T21:57:52.081 INFO:teuthology.orchestra.run.smithi135.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-04-26T21:57:52.332 INFO:teuthology.orchestra.run.smithi135.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-04-26T21:57:52.515 INFO:teuthology.orchestra.run.smithi135.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-26T21:57:52.590 INFO:teuthology.orchestra.run.smithi135.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-26T21:57:52.651 INFO:teuthology.orchestra.run.smithi135.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 974 kB/s | 309 kB 00:00 2024-04-26T21:57:52.749 INFO:teuthology.orchestra.run.smithi135.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-26T21:57:52.825 INFO:teuthology.orchestra.run.smithi135.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-04-26T21:57:53.009 INFO:teuthology.orchestra.run.smithi135.stdout:(109/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-26T21:57:53.034 INFO:teuthology.orchestra.run.smithi135.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-04-26T21:57:53.176 INFO:teuthology.orchestra.run.smithi135.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-04-26T21:57:53.260 INFO:teuthology.orchestra.run.smithi135.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-26T21:57:53.327 INFO:teuthology.orchestra.run.smithi135.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 475 kB/s | 151 kB 00:00 2024-04-26T21:57:53.341 INFO:teuthology.orchestra.run.smithi139.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 641 kB/s | 2.4 MB 00:03 2024-04-26T21:57:53.348 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:57:53.348 INFO:teuthology.orchestra.run.smithi139.stdout:Total 11 MB/s | 222 MB 00:21 2024-04-26T21:57:53.369 INFO:teuthology.orchestra.run.smithi135.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-26T21:57:53.394 INFO:teuthology.orchestra.run.smithi135.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-04-26T21:57:53.436 INFO:teuthology.orchestra.run.smithi135.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-04-26T21:57:53.512 INFO:teuthology.orchestra.run.smithi135.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T21:57:53.537 INFO:teuthology.orchestra.run.smithi135.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-26T21:57:53.587 INFO:teuthology.orchestra.run.smithi135.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-26T21:57:53.704 INFO:teuthology.orchestra.run.smithi135.stdout:(120/152): libarrow-9.0.0-11.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-04-26T21:57:53.730 INFO:teuthology.orchestra.run.smithi135.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.4 MB/s | 206 kB 00:00 2024-04-26T21:57:53.805 INFO:teuthology.orchestra.run.smithi135.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-26T21:57:53.830 INFO:teuthology.orchestra.run.smithi135.stdout:(123/152): python3-colorama-0.4.4-12.el9.noarch 120 kB/s | 35 kB 00:00 2024-04-26T21:57:53.881 INFO:teuthology.orchestra.run.smithi135.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-26T21:57:53.906 INFO:teuthology.orchestra.run.smithi135.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-26T21:57:53.948 INFO:teuthology.orchestra.run.smithi135.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-04-26T21:57:53.974 INFO:teuthology.orchestra.run.smithi135.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-04-26T21:57:54.024 INFO:teuthology.orchestra.run.smithi135.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-04-26T21:57:54.041 INFO:teuthology.orchestra.run.smithi135.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-26T21:57:54.083 INFO:teuthology.orchestra.run.smithi135.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 789 kB/s | 46 kB 00:00 2024-04-26T21:57:54.108 INFO:teuthology.orchestra.run.smithi135.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-26T21:57:54.150 INFO:teuthology.orchestra.run.smithi135.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-26T21:57:54.167 INFO:teuthology.orchestra.run.smithi135.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-04-26T21:57:54.192 INFO:teuthology.orchestra.run.smithi135.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T21:57:54.218 INFO:teuthology.orchestra.run.smithi135.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-04-26T21:57:54.243 INFO:teuthology.orchestra.run.smithi135.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T21:57:54.269 INFO:teuthology.orchestra.run.smithi135.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 371 kB/s | 28 kB 00:00 2024-04-26T21:57:54.294 INFO:teuthology.orchestra.run.smithi135.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 405 kB/s | 31 kB 00:00 2024-04-26T21:57:54.328 INFO:teuthology.orchestra.run.smithi135.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T21:57:54.353 INFO:teuthology.orchestra.run.smithi135.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 703 kB/s | 59 kB 00:00 2024-04-26T21:57:54.379 INFO:teuthology.orchestra.run.smithi135.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-26T21:57:54.404 INFO:teuthology.orchestra.run.smithi135.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-26T21:57:54.424 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-04-26T21:57:54.429 INFO:teuthology.orchestra.run.smithi135.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T21:57:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T21:57:54.488 INFO:teuthology.orchestra.run.smithi135.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-26T21:57:54.514 INFO:teuthology.orchestra.run.smithi135.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-26T21:57:54.539 INFO:teuthology.orchestra.run.smithi135.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-26T21:57:54.565 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-04-26T21:57:54.565 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-04-26T21:57:54.714 INFO:teuthology.orchestra.run.smithi135.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-26T21:57:55.098 INFO:teuthology.orchestra.run.smithi135.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-26T21:57:55.382 INFO:teuthology.orchestra.run.smithi135.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.9 MB/s | 3.4 MB 00:00 2024-04-26T21:57:56.559 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-04-26T21:57:56.560 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-04-26T21:57:56.749 INFO:teuthology.orchestra.run.smithi135.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 679 kB/s | 1.3 MB 00:02 2024-04-26T21:57:58.207 INFO:teuthology.orchestra.run.smithi135.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 788 kB/s | 2.4 MB 00:03 2024-04-26T21:57:58.210 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:57:58.210 INFO:teuthology.orchestra.run.smithi135.stdout:Total 8.7 MB/s | 222 MB 00:25 2024-04-26T21:57:59.333 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-26T21:57:59.476 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-26T21:57:59.476 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-26T21:58:01.300 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-26T21:58:01.301 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-26T21:58:02.587 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-04-26T21:58:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:58:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:58:02.761 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:58:02.795 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:58:02.821 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:58:02.863 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:58:03.191 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:58:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:58:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:58:03.515 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:58:03.521 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:58:03.552 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:58:03.704 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:58:03.748 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:58:03.776 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:58:03.862 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:58:03.894 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:58:03.939 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:58:03.961 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:58:03.986 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:58:04.177 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:58:04.211 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:58:04.317 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:58:04.406 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:58:04.443 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:58:04.478 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:58:04.542 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:58:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:58:04.733 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:58:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:58:04.847 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:58:04.916 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:58:04.986 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:58:05.017 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:58:05.051 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:58:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:58:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:58:05.205 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:58:05.234 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:58:05.304 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:58:05.349 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:58:05.424 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:58:05.558 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:58:05.611 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:58:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:58:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:58:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:58:06.311 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:58:06.348 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:58:06.390 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:58:06.551 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:58:06.617 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:58:06.672 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:58:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:58:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:58:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:58:06.860 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:58:06.895 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:58:06.938 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:58:06.980 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:58:07.117 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:58:07.170 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:58:07.208 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:58:07.249 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:58:07.327 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-26T21:58:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:58:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:58:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:58:07.439 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:58:07.448 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:58:07.472 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:58:07.483 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:58:07.537 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:58:07.564 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:58:07.571 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:58:07.597 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:58:07.639 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:58:07.690 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:58:07.721 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:58:07.749 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:58:07.783 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:58:07.823 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:58:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:58:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:58:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:58:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:58:07.961 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:58:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:58:08.036 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:58:08.074 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:58:08.123 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:58:08.166 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:58:08.180 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:58:08.214 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:58:08.217 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:58:08.261 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:58:08.308 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:58:08.338 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:58:08.359 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:58:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:58:08.485 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:58:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:58:08.564 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:58:08.666 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:58:08.690 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:58:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:58:08.766 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:58:08.785 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:58:08.983 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:58:09.016 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:58:09.123 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:58:09.142 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:58:09.195 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:58:09.226 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:58:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:58:09.266 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:58:09.300 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:58:09.330 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:58:09.364 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:58:09.387 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:58:09.408 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:58:09.452 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:58:09.475 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:58:09.485 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:58:09.513 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:58:09.514 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:58:09.582 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:58:09.635 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:58:09.666 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:58:09.689 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:58:09.702 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:58:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:58:09.769 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:58:09.797 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:58:09.805 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:58:09.848 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:58:09.877 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:58:09.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:58:09.992 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:58:10.066 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:58:10.199 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:58:10.253 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:58:10.446 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:58:10.560 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:58:10.780 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:58:10.871 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:58:10.903 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:58:10.938 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:58:10.985 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:58:11.018 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:58:11.168 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:58:11.223 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:58:11.277 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:58:11.349 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:58:11.382 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:58:11.418 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:58:11.465 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:58:11.501 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:58:11.544 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:58:11.587 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:58:11.722 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:58:11.776 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:58:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:58:11.807 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:58:11.848 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:58:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:58:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:58:11.934 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:58:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:58:11.974 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:58:12.014 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:58:12.040 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:58:12.065 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:58:12.140 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:58:12.245 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:58:12.267 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:58:12.288 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:58:12.298 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:58:12.318 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:58:12.353 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:58:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:58:12.392 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:58:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:58:12.423 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:58:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:58:12.454 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:58:12.488 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:58:12.522 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:58:12.563 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:58:12.599 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:58:12.701 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:58:12.746 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:58:12.785 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:58:12.832 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:58:12.922 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:58:12.930 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:58:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:58:12.979 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:58:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:58:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:58:13.698 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:58:13.744 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:58:13.778 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:58:13.877 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:58:13.928 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:58:14.007 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:58:14.056 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:58:14.233 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:58:14.262 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:58:14.341 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:58:14.982 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:58:15.095 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:58:15.323 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:58:15.355 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:58:15.894 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:58:15.933 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:58:16.022 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:58:16.022 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:58:16.023 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:58:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:58:16.328 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:58:16.413 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:58:16.441 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:58:16.474 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:58:16.789 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:58:16.829 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:58:16.920 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:58:16.951 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:58:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:58:17.466 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:58:17.491 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:58:17.594 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:58:20.047 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:58:20.083 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:58:20.625 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:58:20.671 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:58:20.762 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:58:20.762 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:58:20.763 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:58:20.763 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:20.789 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-04-26T21:58:27.288 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:27.470 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:58:27.486 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:58:27.486 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:27.486 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:58:27.486 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:28.965 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:58:28.990 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:58:28.990 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:28.990 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:58:28.990 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:58:28.991 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:58:28.991 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:29.215 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:58:29.238 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:58:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:58:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:58:29.238 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:58:29.239 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:29.682 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:58:29.705 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:58:29.705 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:29.705 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:58:29.705 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:58:29.705 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:58:29.705 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:29.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:58:29.861 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:58:29.925 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:58:30.112 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:58:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:58:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:58:30.325 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:58:30.466 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:58:30.507 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:58:30.539 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:58:30.567 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:58:30.969 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:58:30.999 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:58:31.031 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:58:31.101 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:58:31.138 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:58:32.166 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:58:32.207 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-04-26T21:58:32.280 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:32.454 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:58:32.469 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:58:32.469 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:32.469 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:58:32.469 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:33.170 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:58:33.207 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:58:33.308 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:58:33.478 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:58:33.509 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:58:33.532 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:58:33.532 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:33.532 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:58:33.532 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:58:33.532 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:58:33.532 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:33.577 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:58:33.588 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:58:33.685 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:58:33.733 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:58:33.924 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:58:33.950 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:58:33.950 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:33.950 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:58:33.951 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:58:33.951 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:58:33.951 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:58:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:58:34.220 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:34.220 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:58:34.220 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:58:34.221 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:58:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:34.652 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:58:34.667 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:58:34.687 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:58:34.690 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:58:34.690 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:34.691 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:58:34.691 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:58:34.691 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:58:34.691 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:34.710 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:58:34.710 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:34.710 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:58:34.710 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:58:34.710 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:58:34.710 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:34.754 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:58:34.777 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:58:34.777 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:34.777 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:58:34.777 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:34.806 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:58:34.822 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:58:34.844 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:58:34.845 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:34.845 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:58:34.845 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:58:34.845 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:58:34.845 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:34.854 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:58:34.910 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:58:35.095 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:58:35.144 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:58:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:58:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:58:35.170 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:35.170 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:58:35.170 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:58:35.170 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:58:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:35.214 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:58:35.310 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:58:35.441 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:58:35.479 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:58:35.511 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:58:35.539 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:58:35.993 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:58:36.022 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:58:36.054 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:58:36.124 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:58:36.154 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:58:37.181 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:58:37.214 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:58:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:58:38.198 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:58:38.300 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:58:38.476 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:58:38.518 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:58:38.541 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:58:38.541 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:38.541 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:58:38.541 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:58:38.542 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:58:38.542 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:38.585 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:58:38.597 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:58:38.694 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:58:38.742 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:58:39.668 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:58:39.706 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:58:39.729 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:58:39.729 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:39.729 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:58:39.729 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:58:39.729 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:58:39.729 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:39.759 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:58:39.775 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:58:39.795 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:58:39.795 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:39.795 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:58:39.795 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:39.801 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:58:39.841 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:58:39.842 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:58:39.863 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:58:39.864 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:39.864 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:58:39.864 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:58:39.864 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:58:39.864 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:39.874 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:58:39.902 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:58:40.079 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:58:40.130 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:58:40.160 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:58:40.166 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:58:40.189 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:58:40.189 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:58:40.189 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:58:40.189 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:58:40.189 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:58:40.189 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:40.784 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:58:40.824 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:58:40.844 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:58:41.681 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:58:41.761 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:58:41.803 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:58:41.886 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:58:41.915 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:58:41.915 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:58:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:58:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:58:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:58:42.138 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:58:42.206 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:58:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:58:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:58:44.886 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:58:44.919 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:58:44.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:58:45.131 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:58:45.169 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:58:45.199 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:58:45.794 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:58:45.832 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:58:45.852 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:58:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:58:46.766 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:58:46.815 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:58:46.896 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:58:46.928 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:58:46.928 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:58:47.012 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:58:47.045 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:58:47.045 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:58:47.139 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:58:47.203 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:58:48.098 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:58:48.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:58:48.100 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:58:48.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:58:48.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:58:48.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:58:50.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:58:50.104 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:50.104 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-04-26T21:58:50.104 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.104 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.104 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.105 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.106 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:58:50.107 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.108 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:58:50.109 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:58:50.110 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:58:50.111 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:58:50.112 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:58:50.113 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:58:50.114 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.115 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:58:50.116 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:58:50.117 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:50.117 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-26T21:58:50.475 DEBUG:teuthology.parallel:result is None 2024-04-26T21:58:53.428 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:58:53.428 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:58:53.428 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:58:53.428 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:58:53.428 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:58:53.429 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:58:53.430 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:58:53.441 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:58:53.441 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:58:53.441 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:58:53.441 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:58:53.442 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:58:53.443 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:58:53.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:58:53.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:58:53.446 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:58:53.447 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:58:53.448 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:58:53.449 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:58:53.450 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:58:55.491 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.492 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:58:55.493 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:58:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:58:55.496 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:58:55.500 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:58:55.502 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:58:55.503 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:58:55.503 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:55.503 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-26T21:58:55.924 DEBUG:teuthology.parallel:result is None 2024-04-26T21:58:55.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:58:56.066 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:58:56.088 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:58:56.088 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:58:56.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:58:56.226 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi139.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:58:56.252 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:58:56.252 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:58:56.254 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:58:56.254 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:56.254 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:58:56.281 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:56.281 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:58:56.322 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:58:56.322 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:56.322 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:58:56.346 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:58:56.409 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:56.409 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:58:56.435 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:58:56.501 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:58:56.501 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:56.501 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:58:56.524 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:58:56.587 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:56.588 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:58:56.612 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:58:56.678 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:58:56.678 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:56.678 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:58:56.701 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:58:56.765 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:56.765 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:58:56.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:58:56.857 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T21:58:56.943 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T21:58:56.944 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2024-04-26T21:58:56.946 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2024-04-26T21:58:56.971 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T21:58:56.971 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:58:57.010 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:58:57.035 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T21:58:57.035 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:58:57.074 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:58:57.105 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T21:58:57.105 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T21:58:57.105 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T21:58:57.106 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:57.106 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:58:57.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:58:57.262 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:58:57.285 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:57.285 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:58:57.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:58:57.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:58:57.443 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T21:58:57.443 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T21:58:57.443 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}}', 'email': None, 'first_in_suite': False, 'job_id': '7675239', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}, 'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi135.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSRV7yFXnXoPmx6i1rOJ/GZGdRyjHl9zExFiX7zxM6J8PBhpIElPz9pKRIDq2ILxIOSPO4RLKvmqhFS56vmXM4=', 'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSO+8fZrkKoUsg+LCSjg08A1kMkizaMwDod6+Fl9CDNXbRdmn3s8sz/UWUt+rRx0Dc4Q0EtupklDqA1nGTlqgo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T21:58:57.444 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:58:57.446 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:58:57.492 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:58:57.500 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:58:57.556 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:57.557 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:58:57.578 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:58:57.578 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-04-26T21:58:57.632 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:58:57.632 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-26 21:58:47.556247455 +0000 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-26 21:56:37.220044656 +0000 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-26 21:56:37.220044656 +0000 2024-04-26T21:58:57.633 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-26 21:56:37.220044656 +0000 2024-04-26T21:58:57.633 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:58:57.694 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-26T21:58:57.695 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-26T21:58:57.695 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000222043 s, 2.3 MB/s 2024-04-26T21:58:57.696 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:58:57.751 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-04-26T21:58:57.805 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:58:57.805 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-26 21:58:47.557247435 +0000 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-26 21:56:37.561037229 +0000 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-26 21:56:37.561037229 +0000 2024-04-26T21:58:57.806 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-26 21:56:37.561037229 +0000 2024-04-26T21:58:57.806 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:58:57.867 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-26T21:58:57.867 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-26T21:58:57.868 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000205337 s, 2.5 MB/s 2024-04-26T21:58:57.869 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:58:57.924 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-04-26T21:58:57.978 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-26 21:58:47.557247435 +0000 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-26 21:56:37.905029737 +0000 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-26 21:56:37.905029737 +0000 2024-04-26T21:58:57.979 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-26 21:56:37.905029737 +0000 2024-04-26T21:58:57.979 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:58:58.040 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-26T21:58:58.040 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-26T21:58:58.041 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000175202 s, 2.9 MB/s 2024-04-26T21:58:58.041 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:58:58.097 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-04-26T21:58:58.150 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-26 21:58:47.557247435 +0000 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-26 21:56:38.222022832 +0000 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-26 21:56:38.222022832 +0000 2024-04-26T21:58:58.151 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-26 21:56:38.222022832 +0000 2024-04-26T21:58:58.152 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:58:58.212 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-26T21:58:58.212 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-26T21:58:58.212 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000207092 s, 2.5 MB/s 2024-04-26T21:58:58.214 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:58:58.268 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:58:58.269 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:58.269 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:58:58.284 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:58:58.284 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-26 21:58:42.579719548 +0000 2024-04-26T21:58:58.339 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-26 21:56:33.894887917 +0000 2024-04-26T21:58:58.340 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-26 21:56:33.894887917 +0000 2024-04-26T21:58:58.340 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-26 21:56:33.894887917 +0000 2024-04-26T21:58:58.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:58:58.401 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-26T21:58:58.401 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-26T21:58:58.401 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000184815 s, 2.8 MB/s 2024-04-26T21:58:58.403 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:58:58.458 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-26 21:58:42.580719528 +0000 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-26 21:56:34.236880147 +0000 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-26 21:56:34.236880147 +0000 2024-04-26T21:58:58.512 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-26 21:56:34.236880147 +0000 2024-04-26T21:58:58.513 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:58:58.573 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-26T21:58:58.573 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-26T21:58:58.573 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000197184 s, 2.6 MB/s 2024-04-26T21:58:58.574 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:58:58.630 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-26 21:58:42.580719528 +0000 2024-04-26T21:58:58.685 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-26 21:56:34.577872400 +0000 2024-04-26T21:58:58.686 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-26 21:56:34.577872400 +0000 2024-04-26T21:58:58.686 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-26 21:56:34.577872400 +0000 2024-04-26T21:58:58.686 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:58:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-26T21:58:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-26T21:58:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000187293 s, 2.7 MB/s 2024-04-26T21:58:58.749 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:58:58.805 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-26 21:58:42.580719528 +0000 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-26 21:56:34.907864903 +0000 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-26 21:56:34.907864903 +0000 2024-04-26T21:58:58.859 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-26 21:56:34.907864903 +0000 2024-04-26T21:58:58.860 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:58:58.920 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-26T21:58:58.920 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-26T21:58:58.920 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000162358 s, 3.2 MB/s 2024-04-26T21:58:58.921 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:58:58.977 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:58:58.977 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:58:58.977 INFO:tasks.ceph:Generating config... 2024-04-26T21:58:58.978 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T21:58:58.978 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-26T21:58:58.979 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T21:58:58.979 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T21:58:58.979 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T21:58:58.979 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T21:58:58.979 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T21:58:58.979 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T21:58:58.980 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T21:58:58.980 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T21:58:58.980 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T21:58:58.980 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T21:58:58.981 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T21:58:58.982 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-26T21:58:58.982 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T21:58:58.982 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-26T21:58:58.982 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T21:58:58.982 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T21:58:58.982 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T21:58:58.982 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T21:58:59.046 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T21:58:59.049 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T21:58:59.091 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:58:59.154 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.135'), ('mon.b', '172.21.15.139'), ('mon.c', '[v2:172.21.15.139:3301,v1:172.21.15.139:6790]')] 2024-04-26T21:58:59.155 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.135,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T21:58:59.156 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:59.156 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:58:59.210 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.135 --add b 172.21.15.139 --addv c '[v2:172.21.15.139:3301,v1:172.21.15.139:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: generated fsid c1882db8-5327-4bd0-9178-0a7c5b1ec94f 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:setting min_mon_release = quincy 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:epoch 0 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:fsid c1882db8-5327-4bd0-9178-0a7c5b1ec94f 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:last_changed 2024-04-26T21:58:59.296171+0000 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:created 2024-04-26T21:58:59.296171+0000 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:min_mon_release 17 (quincy) 2024-04-26T21:58:59.295 INFO:teuthology.orchestra.run.smithi135.stdout:election_strategy: 1 2024-04-26T21:58:59.296 INFO:teuthology.orchestra.run.smithi135.stdout:0: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.a 2024-04-26T21:58:59.296 INFO:teuthology.orchestra.run.smithi135.stdout:1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-04-26T21:58:59.296 INFO:teuthology.orchestra.run.smithi135.stdout:2: [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] mon.c 2024-04-26T21:58:59.296 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T21:58:59.297 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:58:59.311 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c1882db8-5327-4bd0-9178-0a7c5b1ec94f... 2024-04-26T21:58:59.312 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:58:59.354 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:58:59.391 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-04-26T21:58:59.392 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-04-26T21:58:59.393 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-04-26T21:58:59.394 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.135,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]" 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = c1882db8-5327-4bd0-9178-0a7c5b1ec94f 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.395 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:58:59.396 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: bdev async discard = True 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: bdev enable discard = True 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore allocator = bitmap 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 1/20 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 1/20 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 4/10 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: filestore flush min = 0 2024-04-26T21:58:59.397 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: osd blocked scrub grace period = 3600 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: osd max markdown count = 1000 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:58:59.398 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-04-26T21:58:59.399 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-04-26T21:58:59.400 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: client_shutdown_timeout = 120 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-04-26T21:58:59.401 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: mds bal fragment size max = 10000 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: mds bal merge size = 5 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: mds bal split bits = 3 2024-04-26T21:58:59.402 INFO:teuthology.orchestra.run.smithi135.stdout: mds bal split size = 100 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = True 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = True 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-04-26T21:58:59.403 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-04-26T21:58:59.404 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-04-26T21:58:59.405 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-04-26T21:58:59.406 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-04-26T21:58:59.407 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.135,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]" 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on skipped message = False 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = c1882db8-5327-4bd0-9178-0a7c5b1ec94f 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-04-26T21:58:59.408 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.409 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: bdev async discard = True 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: bdev enable discard = True 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore allocator = bitmap 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 1/20 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 1/20 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 4/10 2024-04-26T21:58:59.410 INFO:teuthology.orchestra.run.smithi139.stdout: filestore flush min = 0 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: osd blocked scrub grace period = 3600 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: osd max markdown count = 1000 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-26T21:58:59.411 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-04-26T21:58:59.412 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:58:59.413 INFO:teuthology.orchestra.run.smithi139.stdout: mon down mkfs grace = 300 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: client_shutdown_timeout = 120 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-26T21:58:59.414 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal fragment size max = 10000 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal merge size = 5 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal split bits = 3 2024-04-26T21:58:59.415 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal split size = 100 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = True 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = True 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-26T21:58:59.416 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-26T21:58:59.416 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T21:58:59.416 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T21:58:59.499 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T21:58:59.499 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:59.499 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T21:58:59.513 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:59.513 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T21:58:59.568 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T21:58:59.568 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:59.568 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:58:59.568 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:58:59.640 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:58:59.640 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:58:59.695 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T21:58:59.695 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:59.695 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:58:59.695 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:58:59.728 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:58:59.728 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:58:59.782 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T21:58:59.782 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T21:58:59.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:58:59.830 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:58:59.832 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:58:59.887 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:58:59.890 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T21:58:59.890 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:58:59.941 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T21:58:59.964 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:59:00.052 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:59:00.054 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T21:59:00.077 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:59:00.125 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:59:00.128 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T21:59:00.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:59:00.238 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:59:00.241 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T21:59:00.264 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T21:59:00.265 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:59:00.304 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T21:59:00.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T21:59:00.356 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T21:59:00.367 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T21:59:00.367 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:59:00.368 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T21:59:00.391 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:00.391 INFO:tasks.ceph:role: osd.0 2024-04-26T21:59:00.391 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T21:59:00.391 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:00.461 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:00.462 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:00.462 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:00.462 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:00.962 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-26T21:59:00.964 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-26T21:59:00.964 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T21:59:01.084 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T21:59:01.114 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T21:59:01.179 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:01.179 INFO:tasks.ceph:role: osd.1 2024-04-26T21:59:01.179 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T21:59:01.180 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:01.252 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:01.713 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-26T21:59:01.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-26T21:59:01.716 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T21:59:01.801 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T21:59:01.828 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T21:59:01.892 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:01.892 INFO:tasks.ceph:role: osd.2 2024-04-26T21:59:01.892 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T21:59:01.892 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:01.963 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:02.445 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-26T21:59:02.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-26T21:59:02.447 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T21:59:02.582 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T21:59:02.611 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T21:59:02.674 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:02.675 INFO:tasks.ceph:role: osd.3 2024-04-26T21:59:02.675 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T21:59:02.675 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:59:02.745 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:02.745 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:02.745 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:02.745 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:02.745 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:02.746 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:02.746 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:02.746 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:02.746 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:02.746 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:03.257 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-26T21:59:03.260 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-26T21:59:03.260 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T21:59:03.409 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T21:59:03.440 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:03.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:03.535+0000 7f7438018640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T21:59:03.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:03.536+0000 7f7438018640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T21:59:03.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:03.536+0000 7f7438018640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:59:03.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:03.536+0000 7f7438018640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:59:03.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:03.536+0000 7f7438018640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:59:03.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:03.536+0000 7f7438018640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T21:59:06.395 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T21:59:06.423 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:06.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:06.511+0000 7fec4f825640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T21:59:06.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:06.511+0000 7fec4f825640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T21:59:06.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:06.511+0000 7fec4f825640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:59:06.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:06.511+0000 7fec4f825640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:59:06.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:06.511+0000 7fec4f825640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:59:06.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:06.512+0000 7fec4f825640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T21:59:09.361 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T21:59:09.387 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:09.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:09.474+0000 7f4aba418640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T21:59:09.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:09.474+0000 7f4aba418640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T21:59:09.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:09.474+0000 7f4aba418640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:59:09.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:09.474+0000 7f4aba418640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:59:09.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:09.474+0000 7f4aba418640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:59:09.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:09.474+0000 7f4aba418640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T21:59:12.321 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T21:59:12.348 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:12.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:12.436+0000 7ff3c1557640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T21:59:12.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:12.437+0000 7ff3c1557640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T21:59:12.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:12.437+0000 7ff3c1557640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:59:12.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:12.437+0000 7ff3c1557640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:59:12.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:12.437+0000 7ff3c1557640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:59:12.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:12.437+0000 7ff3c1557640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T21:59:15.294 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T21:59:15.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T21:59:15.351 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:15.351 INFO:tasks.ceph:role: osd.4 2024-04-26T21:59:15.351 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T21:59:15.351 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:15.426 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:15.427 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:15.427 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:15.937 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-26T21:59:15.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-26T21:59:15.940 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T21:59:16.044 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T21:59:16.080 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T21:59:16.147 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:16.147 INFO:tasks.ceph:role: osd.5 2024-04-26T21:59:16.148 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T21:59:16.148 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:59:16.221 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:16.221 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:16.222 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:16.222 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:16.222 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:16.682 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-26T21:59:16.684 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-26T21:59:16.684 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T21:59:16.776 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T21:59:16.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T21:59:16.873 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:16.874 INFO:tasks.ceph:role: osd.6 2024-04-26T21:59:16.874 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T21:59:16.874 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:16.946 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:16.947 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:16.947 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:17.433 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-26T21:59:17.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-26T21:59:17.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T21:59:17.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T21:59:17.608 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T21:59:17.673 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:59:17.673 INFO:tasks.ceph:role: osd.7 2024-04-26T21:59:17.673 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T21:59:17.673 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:59:17.745 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:59:17.746 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:59:17.746 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:59:17.746 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:59:18.257 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-26T21:59:18.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-26T21:59:18.259 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T21:59:18.404 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T21:59:18.438 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:18.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:18.528+0000 7ff1ee20f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T21:59:18.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:18.529+0000 7ff1ee20f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T21:59:18.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:18.529+0000 7ff1ee20f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:59:18.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:18.529+0000 7ff1ee20f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:59:18.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:18.529+0000 7ff1ee20f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:59:18.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:18.529+0000 7ff1ee20f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T21:59:21.386 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T21:59:21.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:21.502+0000 7f7b38c16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T21:59:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:21.502+0000 7f7b38c16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T21:59:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:21.502+0000 7f7b38c16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:59:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:21.502+0000 7f7b38c16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:59:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:21.502+0000 7f7b38c16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:59:21.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:21.502+0000 7f7b38c16640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T21:59:24.354 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T21:59:24.381 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:24.469+0000 7f57bc1d4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T21:59:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:24.469+0000 7f57bc1d4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T21:59:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:24.469+0000 7f57bc1d4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:59:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:24.469+0000 7f57bc1d4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:59:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:24.469+0000 7f57bc1d4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:59:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:24.470+0000 7f57bc1d4640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T21:59:27.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T21:59:27.348 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:27.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:27.435+0000 7fb557625640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:27.435+0000 7fb557625640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:27.435+0000 7fb557625640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:27.435+0000 7fb557625640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:27.435+0000 7fb557625640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:27.436+0000 7fb557625640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T21:59:27.445 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:59:27.448 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:59:30.281 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T21:59:30.308 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T21:59:30.309 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.309 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T21:59:30.338 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.338 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T21:59:30.407 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.407 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T21:59:30.474 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.474 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T21:59:30.541 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.541 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T21:59:30.607 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.607 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T21:59:30.672 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:30.672 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T21:59:30.737 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:30.737 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T21:59:30.766 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:30.766 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T21:59:30.832 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:30.833 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T21:59:30.899 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:30.899 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T21:59:30.964 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:30.964 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T21:59:31.030 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:31.031 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T21:59:31.096 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:31.096 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T21:59:31.160 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:31.160 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T21:59:31.177 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:31.177 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T21:59:31.217 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T21:59:31.217 DEBUG:teuthology.orchestra.run.smithi135:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:59:31.220 DEBUG:teuthology.orchestra.run.smithi139:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.y] 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCjIyxmyR+MMRAAy1ORZyt2EnIcZrAK4yBOuQ== 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCjIyxmoRMFOBAAh2+dYdPIvwIb9fCa+ZSo6g== 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCkIyxmQHcwAxAAIVIumNMyZ1wpiIM+tO2LYA== 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.0] 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCnIyxmtLoKIBAAu2T5itVd/eAAvrdIwBy7Pg== 2024-04-26T21:59:31.260 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.1] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCqIyxmwImZHhAAht5KbWCdBvv5OgmP/ZvTPw== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.2] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCtIyxmaVZZHBAA57ZXXn8s1mhcqKdygfwYpw== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.3] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCwIyxmSbIXGhAAB9VdXU9S+RyFQ9/pxnWsRA== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.x] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCjIyxmcsz/NBAA5WANsIn3uKu2nP6fuwEP5A== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCkIyxmEBmSBxAA6ymQLPhg3Sp2pVFphU0pfw== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCkIyxmSGVLDhAAs8/MHaJxIlYKl2urTLGfGw== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.4] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQC2Iyxm+FSlHxAAY205i9tRnnMx3gzY0Npmqg== 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.5] 2024-04-26T21:59:31.261 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQC5IyxmhH77HRAAx523Dhxh5T39W7mjjXBzvA== 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.6] 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQC8Iyxmu1sKHBAAmYbm0NezJPVOih7H7vlmCg== 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.7] 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQC/Iyxm5wkCGhAAv6MT8r1eHV1/8GAsSzopNQ== 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCkIyxmiAwyEhAAtnYEZQH6o0ktxxOKW9vXKA== 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2024-04-26T21:59:31.262 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCkIyxm9whKFRAA+mhwyof5cZaYwPMcdqHy0Q== 2024-04-26T21:59:31.279 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.y] 2024-04-26T21:59:31.279 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCjIyxmyR+MMRAAy1ORZyt2EnIcZrAK4yBOuQ== 2024-04-26T21:59:31.279 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-26T21:59:31.279 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCjIyxmoRMFOBAAh2+dYdPIvwIb9fCa+ZSo6g== 2024-04-26T21:59:31.279 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCkIyxmQHcwAxAAIVIumNMyZ1wpiIM+tO2LYA== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.0] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCnIyxmtLoKIBAAu2T5itVd/eAAvrdIwBy7Pg== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.1] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCqIyxmwImZHhAAht5KbWCdBvv5OgmP/ZvTPw== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.2] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCtIyxmaVZZHBAA57ZXXn8s1mhcqKdygfwYpw== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.3] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCwIyxmSbIXGhAAB9VdXU9S+RyFQ9/pxnWsRA== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.x] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCjIyxmcsz/NBAA5WANsIn3uKu2nP6fuwEP5A== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCkIyxmEBmSBxAA6ymQLPhg3Sp2pVFphU0pfw== 2024-04-26T21:59:31.280 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCkIyxmSGVLDhAAs8/MHaJxIlYKl2urTLGfGw== 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.4] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC2Iyxm+FSlHxAAY205i9tRnnMx3gzY0Npmqg== 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.5] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC5IyxmhH77HRAAx523Dhxh5T39W7mjjXBzvA== 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.6] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC8Iyxmu1sKHBAAmYbm0NezJPVOih7H7vlmCg== 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.7] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/Iyxm5wkCGhAAv6MT8r1eHV1/8GAsSzopNQ== 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCkIyxmiAwyEhAAtnYEZQH6o0ktxxOKW9vXKA== 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:[client.1] 2024-04-26T21:59:31.281 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCkIyxm9whKFRAA+mhwyof5cZaYwPMcdqHy0Q== 2024-04-26T21:59:31.282 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:59:31.306 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:59:31.373 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.427 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.429 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.482 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.531 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.533 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.581 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.583 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.633 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.635 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.683 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:59:31.685 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:59:31.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.736 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.787 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.789 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:59:31.840 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.842 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.890 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.893 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.941 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.943 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.992 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:31.994 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:59:32.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:59:32.046 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:59:32.097 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:59:32.099 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:59:32.151 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T21:59:32.152 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T21:59:32.175 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:59:32.269 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T21:59:32.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T21:59:32.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:59:32.410 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T21:59:32.433 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T21:59:32.497 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:59:32.592 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T21:59:32.615 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:32.617 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:59:32.670 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T21:59:32.670 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T21:59:32.671 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T21:59:32.673 INFO:tasks.ceph.mon.a:Started 2024-04-26T21:59:32.673 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T21:59:32.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T21:59:32.713 INFO:tasks.ceph.mon.b:Started 2024-04-26T21:59:32.713 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T21:59:32.713 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T21:59:32.715 INFO:tasks.ceph.mon.c:Started 2024-04-26T21:59:32.715 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T21:59:32.716 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T21:59:32.716 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T21:59:32.718 INFO:tasks.ceph.mgr.y:Started 2024-04-26T21:59:32.718 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T21:59:32.718 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T21:59:32.720 INFO:tasks.ceph.mgr.x:Started 2024-04-26T21:59:32.720 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T21:59:32.721 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T21:59:32.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 Processor -- start 2024-04-26T21:59:32.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 -- start start 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd63c153160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 0x7fd63c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c153bb0 con 0x7fd63c0c3530 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd63c0c85f0 con 0x7fd63c1536a0 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd64274d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c153e90 con 0x7fd63c06ee20 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd640cea640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 0x7fd63c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.819+0000 7fd640cea640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 0x7fd63c14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53260/0 (socket says 172.21.15.135:53260) 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd640cea640 1 -- 172.21.15.135:0/608256506 learned_addr learned my addr 172.21.15.135:0/608256506 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd640cea640 1 -- 172.21.15.135:0/608256506 >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 msgr2=0x7fd63c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd640cea640 1 -- 172.21.15.135:0/608256506 >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 msgr2=0x7fd63c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd640cea640 1 --2- 172.21.15.135:0/608256506 >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 0x7fd63c14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd640cea640 1 --2- 172.21.15.135:0/608256506 >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 0x7fd63c14cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd63b7fe640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd63c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd63b7fe640 1 -- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 msgr2=0x7fd63c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd63b7fe640 1 -- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 msgr2=0x7fd63c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd63b7fe640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd63c153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:59:32.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.820+0000 7fd63b7fe640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd63c153160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:59:32.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630064473 0 0) 0x7fd63c153bb0 con 0x7fd63c0c3530 2024-04-26T21:59:32.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd624003520 con 0x7fd63c0c3530 2024-04-26T21:59:32.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 908889714 0 0) 0x7fd624003520 con 0x7fd63c0c3530 2024-04-26T21:59:32.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 msgr2=0x7fd63c153160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd63c153160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 msgr2=0x7fd63c14cdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 --2- 172.21.15.135:0/608256506 >> v2:172.21.15.135:3300/0 conn(0x7fd63c1536a0 0x7fd63c14cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c0c76e0 con 0x7fd63c0c3530 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fd62c001670 con 0x7fd63c0c3530 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.971+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 >> v1:172.21.15.139:6789/0 conn(0x7fd63c0c3530 legacy=0x7fd63c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63affd640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63affd640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd624004800 0x7fd624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63affd640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd6240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0c85f0 con 0x7fd6240039b0 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c153e90 con 0x7fd624004800 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd62400bb50 con 0x7fd63c06ee20 2024-04-26T21:59:32.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63bfff640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63b7fe640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd624004800 0x7fd624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd640cea640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd6240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63bfff640 1 -- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 msgr2=0x7fd6240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63bfff640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c06ee20 0x7fd6240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63bfff640 1 -- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd624004800 msgr2=0x7fd624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63bfff640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd624004800 0x7fd624004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.972+0000 7fd63bfff640 1 -- 172.21.15.135:0/608256506 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624003520 con 0x7fd6240039b0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.973+0000 7fd63bfff640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd624003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c160 tx=0x7fd624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa4b940fe6c39f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.973+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c036070 con 0x7fd6240039b0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.973+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c031600 con 0x7fd6240039b0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.973+0000 7fd63affd640 1 -- 172.21.15.135:0/608256506 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c007c60 con 0x7fd6240039b0 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd64274d640 1 -- 172.21.15.135:0/608256506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 msgr2=0x7fd624003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:32.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd64274d640 1 --2- 172.21.15.135:0/608256506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd624003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c160 tx=0x7fd624005830 comp rx=0 tx=0).stop 2024-04-26T21:59:32.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd640cea640 1 -- 172.21.15.135:0/608256506 reap_dead start 2024-04-26T21:59:32.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd64274d640 1 -- 172.21.15.135:0/608256506 shutdown_connections 2024-04-26T21:59:32.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd64274d640 1 -- 172.21.15.135:0/608256506 >> 172.21.15.135:0/608256506 conn(0x7fd63c0aa630 msgr2=0x7fd63c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:32.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd64274d640 1 -- 172.21.15.135:0/608256506 shutdown_connections 2024-04-26T21:59:32.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.974+0000 7fd64274d640 1 -- 172.21.15.135:0/608256506 wait complete. 2024-04-26T21:59:32.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.975+0000 7fd64274d640 1 Processor -- start 2024-04-26T21:59:32.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 -- start start 2024-04-26T21:59:32.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd63c0c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c154550 0x7fd63c1488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c0c21f0 0x7fd63c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0c3a90 con 0x7fd6240039b0 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0c3910 con 0x7fd63c154550 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd64274d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c0c6960 con 0x7fd63c0c21f0 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd63bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd63c0c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c154550 0x7fd63c1488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd63bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd63c0c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53278/0 (socket says 172.21.15.135:53278) 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.976+0000 7fd63bfff640 1 -- 172.21.15.135:0/3564228495 learned_addr learned my addr 172.21.15.135:0/3564228495 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd640cea640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c0c21f0 0x7fd63c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd63bfff640 1 -- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c0c21f0 msgr2=0x7fd63c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd63bfff640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c0c21f0 0x7fd63c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd63bfff640 1 -- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c154550 msgr2=0x7fd63c1488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:32.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd63bfff640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c154550 0x7fd63c1488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd63bfff640 1 -- 172.21.15.135:0/3564228495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c1ed530 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd63bfff640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd63c0c1cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd62c000c00 tx=0x7fd63c153d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c041020 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c037040 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c1ed810 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c036070 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1480b0 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.977+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd62c044020 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.978+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd62c03d480 con 0x7fd6240039b0 2024-04-26T21:59:32.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.978+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c151550 con 0x7fd6240039b0 2024-04-26T21:59:32.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:32.980+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd63c151550 con 0x7fd6240039b0 2024-04-26T21:59:33.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.046+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd63c14c890 con 0x7fd6240039b0 2024-04-26T21:59:33.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.047+0000 7fd638ff9640 1 -- 172.21.15.135:0/3564228495 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd63c14c890 con 0x7fd6240039b0 2024-04-26T21:59:33.047 INFO:teuthology.orchestra.run.smithi135.stderr:adjusted tunables profile to default 2024-04-26T21:59:33.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 msgr2=0x7fd63c0c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd63c0c1cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd62c000c00 tx=0x7fd63c153d80 comp rx=0 tx=0).stop 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 shutdown_connections 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd63c0c21f0 0x7fd63c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c154550 0x7fd63c1488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 --2- 172.21.15.135:0/3564228495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6240039b0 0x7fd63c0c1cb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 >> 172.21.15.135:0/3564228495 conn(0x7fd63c0aa630 msgr2=0x7fd63c0680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 shutdown_connections 2024-04-26T21:59:33.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:33.048+0000 7fd64274d640 1 -- 172.21.15.135:0/3564228495 wait complete. 2024-04-26T21:59:33.059 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T21:59:33.059 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T21:59:33.059 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:33.060 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T21:59:33.083 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:33.083 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T21:59:33.084 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:33.084+0000 7f84fb0f7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:59:33.084 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:33.084+0000 7fc6f3530040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:59:33.106 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:33.107 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T21:59:33.169 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T21:59:33.170 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T21:59:33.233 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:33.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T21:59:33.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:33.241+0000 7fc6f3530040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:59:33.256 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:33.256 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T21:59:33.266 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:33.266+0000 7f84fb0f7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:59:33.296 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:33.296+0000 7fc6f3530040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:59:33.319 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:33.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T21:59:33.320 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:33.320+0000 7f84fb0f7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:59:33.383 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T21:59:33.383 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T21:59:33.446 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new c2a884f4-36b9-4e32-a4dd-0a9e19967ac8 0 2024-04-26T21:59:33.554 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:33.554+0000 7fc6f3530040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:59:33.570 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:33.570+0000 7f84fb0f7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:59:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.585+0000 7fcc41f71640 1 Processor -- start 2024-04-26T21:59:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.585+0000 7fcc41f71640 1 -- start start 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc41f71640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcc3c150e90 0x7fcc3c074370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc41f71640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcc3c070c80 0x7fcc3c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc41f71640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3c071090 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc41f71640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c06fc50 con 0x7fcc3c150e90 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc41f71640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c072e60 con 0x7fcc3c070c80 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcc3c070c80 0x7fcc3c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcc3c070c80 0x7fcc3c06d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51896/0 (socket says 172.21.15.139:51896) 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 -- 172.21.15.139:0/703387577 learned_addr learned my addr 172.21.15.139:0/703387577 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3b7fe640 1 --2- 172.21.15.139:0/703387577 >> v2:172.21.15.135:3300/0 conn(0x7fcc3c150e90 0x7fcc3c074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 -- 172.21.15.139:0/703387577 >> v2:172.21.15.135:3300/0 conn(0x7fcc3c150e90 msgr2=0x7fcc3c074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 --2- 172.21.15.139:0/703387577 >> v2:172.21.15.135:3300/0 conn(0x7fcc3c150e90 0x7fcc3c074370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 -- 172.21.15.139:0/703387577 >> v1:172.21.15.135:6789/0 conn(0x7fcc3c0c3c70 legacy=0x7fcc3c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3a7fc640 1 -- 172.21.15.139:0/703387577 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591029336 0 0) 0x7fcc2c001110 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.586+0000 7fcc3bfff640 1 -- 172.21.15.139:0/703387577 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c154580 con 0x7fcc3c070c80 2024-04-26T21:59:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.587+0000 7fcc3bfff640 1 --2- 172.21.15.139:0/703387577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c070c80 0x7fcc3c06d320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c070800 tx=0x7fcc3402a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd92224b93cea435 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.587+0000 7fcc3a7fc640 1 -- 172.21.15.139:0/703387577 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34036070 con 0x7fcc3c070c80 2024-04-26T21:59:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc3a7fc640 1 -- 172.21.15.139:0/703387577 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc34031600 con 0x7fcc3c070c80 2024-04-26T21:59:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc3a7fc640 1 -- 172.21.15.139:0/703387577 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340319d0 con 0x7fcc3c070c80 2024-04-26T21:59:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 -- 172.21.15.139:0/703387577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c070c80 msgr2=0x7fcc3c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 --2- 172.21.15.139:0/703387577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c070c80 0x7fcc3c06d320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c070800 tx=0x7fcc3402a610 comp rx=0 tx=0).stop 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 -- 172.21.15.139:0/703387577 shutdown_connections 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 --2- 172.21.15.139:0/703387577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c070c80 0x7fcc3c06d320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 --2- 172.21.15.139:0/703387577 >> v2:172.21.15.135:3300/0 conn(0x7fcc3c150e90 0x7fcc3c074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 -- 172.21.15.139:0/703387577 >> 172.21.15.139:0/703387577 conn(0x7fcc3c0c5d70 msgr2=0x7fcc3c0c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 -- 172.21.15.139:0/703387577 shutdown_connections 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.588+0000 7fcc41f71640 1 -- 172.21.15.139:0/703387577 wait complete. 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.589+0000 7fcc41f71640 1 Processor -- start 2024-04-26T21:59:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.589+0000 7fcc41f71640 1 -- start start 2024-04-26T21:59:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc41f71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc3c070c80 0x7fcc3c0ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc41f71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 0x7fcc3c1e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc41f71640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc3c150e90 0x7fcc3c1e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc41f71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c154230 con 0x7fcc3c070c80 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc41f71640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c072af0 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc41f71640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3c151270 con 0x7fcc3c150e90 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 0x7fcc3c1e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc3c150e90 0x7fcc3c1e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 0x7fcc3c1e84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51900/0 (socket says 172.21.15.139:51900) 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc3affd640 1 -- 172.21.15.139:0/2653985071 learned_addr learned my addr 172.21.15.139:0/2653985071 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.590+0000 7fcc3b7fe640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc3c070c80 0x7fcc3c0ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3affd640 1 -- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc3c150e90 msgr2=0x7fcc3c1e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3affd640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc3c150e90 0x7fcc3c1e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3affd640 1 -- 172.21.15.139:0/2653985071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc3c070c80 msgr2=0x7fcc3c0ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3affd640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc3c070c80 0x7fcc3c0ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3affd640 1 -- 172.21.15.139:0/2653985071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c1de670 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3bfff640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc3c150e90 0x7fcc3c1e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc3affd640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 0x7fcc3c1e84c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00c440 tx=0x7fcc2c00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c017070 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c005480 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c1ebbd0 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c003830 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c0adb90 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc2c01a070 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.591+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fcc2c012740 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.592+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c0aed50 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.594+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcc2c021080 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.603 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:33.603+0000 7fc6f3530040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:59:33.616 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:33.616+0000 7f84fb0f7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:59:33.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.667+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c2a884f4-36b9-4e32-a4dd-0a9e19967ac8", "id": 0} v 0) -- 0x7fcc3c0507a0 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.670+0000 7fcc23fff640 1 -- 172.21.15.139:0/2653985071 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2a884f4-36b9-4e32-a4dd-0a9e19967ac8", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fcc3c071090 con 0x7fcc3c0c3c70 2024-04-26T21:59:33.670 INFO:teuthology.orchestra.run.smithi139.stdout:0 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 msgr2=0x7fcc3c1e84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 0x7fcc3c1e84c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00c440 tx=0x7fcc2c00c900 comp rx=0 tx=0).stop 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 shutdown_connections 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc3c150e90 0x7fcc3c1e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc3c0c3c70 0x7fcc3c1e84c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 --2- 172.21.15.139:0/2653985071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc3c070c80 0x7fcc3c0ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 >> 172.21.15.139:0/2653985071 conn(0x7fcc3c0c5d70 msgr2=0x7fcc3c14b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 shutdown_connections 2024-04-26T21:59:33.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.671+0000 7fcc41f71640 1 -- 172.21.15.139:0/2653985071 wait complete. 2024-04-26T21:59:33.681 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 366143a9-a0ba-4cde-85d6-c871f36fc1b7 1 2024-04-26T21:59:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.777+0000 7f258763e640 1 Processor -- start 2024-04-26T21:59:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.778+0000 7f258763e640 1 -- start start 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.778+0000 7f258763e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25800c5430 0x7f2580157720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.778+0000 7f258763e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258015a230 con 0x7f258013fe20 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.778+0000 7f258763e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258015a510 con 0x7f25800c3c70 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.778+0000 7f258763e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2580157c60 con 0x7f25800c5430 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2585bdb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25800c5430 0x7f2580157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2585bdb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25800c5430 0x7f2580157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51902/0 (socket says 172.21.15.139:51902) 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2584bd9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f25800c3c70 0x7f2580157010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:43392/0 (socket says 172.21.15.139:43392) 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2585bdb640 1 -- 172.21.15.139:0/3231992669 learned_addr learned my addr 172.21.15.139:0/3231992669 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2577fff640 1 -- 172.21.15.139:0/3231992669 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639400146 0 0) 0x7f258015a230 con 0x7f258013fe20 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2585bdb640 1 -- 172.21.15.139:0/3231992669 >> v1:172.21.15.139:6789/0 conn(0x7f25800c3c70 legacy=0x7f2580157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2585bdb640 1 -- 172.21.15.139:0/3231992669 >> v1:172.21.15.135:6789/0 conn(0x7f258013fe20 legacy=0x7f258006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.779+0000 7f2585bdb640 1 -- 172.21.15.139:0/3231992669 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258015a7f0 con 0x7f25800c5430 2024-04-26T21:59:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.780+0000 7f2585bdb640 1 --2- 172.21.15.139:0/3231992669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25800c5430 0x7f2580157720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2578009e50 tx=0x7f257802a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e143891d2cb33295 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.780+0000 7f2577fff640 1 -- 172.21.15.139:0/3231992669 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578036070 con 0x7f25800c5430 2024-04-26T21:59:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.780+0000 7f2577fff640 1 -- 172.21.15.139:0/3231992669 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2578031600 con 0x7f25800c5430 2024-04-26T21:59:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.780+0000 7f2577fff640 1 -- 172.21.15.139:0/3231992669 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25780319d0 con 0x7f25800c5430 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.780+0000 7f258763e640 1 -- 172.21.15.139:0/3231992669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25800c5430 msgr2=0x7f2580157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.780+0000 7f258763e640 1 --2- 172.21.15.139:0/3231992669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25800c5430 0x7f2580157720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2578009e50 tx=0x7f257802a6d0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.781+0000 7f258763e640 1 -- 172.21.15.139:0/3231992669 shutdown_connections 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.781+0000 7f258763e640 1 --2- 172.21.15.139:0/3231992669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25800c5430 0x7f2580157720 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.781+0000 7f258763e640 1 -- 172.21.15.139:0/3231992669 >> 172.21.15.139:0/3231992669 conn(0x7f25800c5d70 msgr2=0x7f25800c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.781+0000 7f258763e640 1 -- 172.21.15.139:0/3231992669 shutdown_connections 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.781+0000 7f258763e640 1 -- 172.21.15.139:0/3231992669 wait complete. 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 Processor -- start 2024-04-26T21:59:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 -- start start 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 0x7f2580073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25800c5430 0x7f25801ec510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 0x7f25801eca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f258015b120 con 0x7f25800c5430 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f258015adb0 con 0x7f258013fe20 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.782+0000 7f258763e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2580140200 con 0x7f25800c3c70 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 0x7f2580073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2585bdb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 0x7f25801eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 0x7f2580073950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:40294/0 (socket says 172.21.15.139:40294) 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 -- 172.21.15.139:0/4084611028 learned_addr learned my addr 172.21.15.139:0/4084611028 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2585bdb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 0x7f25801eca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51912/0 (socket says 172.21.15.139:51912) 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2584bd9640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25800c5430 0x7f25801ec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 -- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 msgr2=0x7f25801eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 0x7f25801eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 -- 172.21.15.139:0/4084611028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25800c5430 msgr2=0x7f25801ec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25800c5430 0x7f25801ec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 -- 172.21.15.139:0/4084611028 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258013c160 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2585bdb640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 0x7f25801eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f25853da640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 0x7f2580073950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f257000c410 tx=0x7f257000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570017070 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25700035f0 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25801efa80 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25700039e0 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801f0040 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.783+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f257001a070 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.784+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f2570012d60 con 0x7f25800c3c70 2024-04-26T21:59:33.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.784+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25800507a0 con 0x7f25800c3c70 2024-04-26T21:59:33.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.786+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2570012400 con 0x7f25800c3c70 2024-04-26T21:59:33.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.859+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "366143a9-a0ba-4cde-85d6-c871f36fc1b7", "id": 1} v 0) -- 0x7f25800c2bf0 con 0x7f25800c3c70 2024-04-26T21:59:33.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.862+0000 7f2575ffb640 1 -- 172.21.15.139:0/4084611028 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "366143a9-a0ba-4cde-85d6-c871f36fc1b7", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f25700125e0 con 0x7f25800c3c70 2024-04-26T21:59:33.862 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 msgr2=0x7f2580073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 0x7f2580073950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f257000c410 tx=0x7f257000c8d0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 shutdown_connections 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f258013fe20 0x7f25801eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25800c5430 0x7f25801ec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 --2- 172.21.15.139:0/4084611028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25800c3c70 0x7f2580073950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 >> 172.21.15.139:0/4084611028 conn(0x7f25800c5d70 msgr2=0x7f25800c2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 shutdown_connections 2024-04-26T21:59:33.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.863+0000 7f258763e640 1 -- 172.21.15.139:0/4084611028 wait complete. 2024-04-26T21:59:33.873 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346 2 2024-04-26T21:59:33.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.971+0000 7f7b073f0640 1 Processor -- start 2024-04-26T21:59:33.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.971+0000 7f7b073f0640 1 -- start start 2024-04-26T21:59:33.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b073f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b0013fe20 0x7f7b000c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b073f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7b00157010 0x7f7b001574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b073f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0015a230 con 0x7f7b000c3c70 2024-04-26T21:59:33.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b073f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7b000c39b0 con 0x7f7b00157010 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b073f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0015b120 con 0x7f7b0013fe20 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0498b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7b000c3c70 0x7f7b0006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:43396/0 (socket says 172.21.15.139:43396) 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0498b640 1 -- 172.21.15.139:0/812397004 learned_addr learned my addr 172.21.15.139:0/812397004 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0598d640 1 --2- 172.21.15.139:0/812397004 >> v2:172.21.15.135:3300/0 conn(0x7f7b00157010 0x7f7b001574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0518c640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b0013fe20 0x7f7b000c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0598d640 1 -- 172.21.15.139:0/812397004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b0013fe20 msgr2=0x7f7b000c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0598d640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b0013fe20 0x7f7b000c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0598d640 1 -- 172.21.15.139:0/812397004 >> v1:172.21.15.139:6789/0 conn(0x7f7b000c3c70 legacy=0x7f7b0006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0598d640 1 -- 172.21.15.139:0/812397004 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0015a510 con 0x7f7b00157010 2024-04-26T21:59:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.973+0000 7f7b0518c640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b0013fe20 0x7f7b000c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.974+0000 7f7b0598d640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b00157010 0x7f7b001574a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009d90 tx=0x7f7af802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb93871196c6c356 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7af7fff640 1 -- 172.21.15.139:0/812397004 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8007740 con 0x7f7b00157010 2024-04-26T21:59:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7af7fff640 1 -- 172.21.15.139:0/812397004 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af80078e0 con 0x7f7b00157010 2024-04-26T21:59:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7af7fff640 1 -- 172.21.15.139:0/812397004 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8007cb0 con 0x7f7b00157010 2024-04-26T21:59:33.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 -- 172.21.15.139:0/812397004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b00157010 msgr2=0x7f7b001574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b00157010 0x7f7b001574a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009d90 tx=0x7f7af802a660 comp rx=0 tx=0).stop 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 -- 172.21.15.139:0/812397004 shutdown_connections 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b00157010 0x7f7b001574a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 --2- 172.21.15.139:0/812397004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b0013fe20 0x7f7b000c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 -- 172.21.15.139:0/812397004 >> 172.21.15.139:0/812397004 conn(0x7f7b000c5d70 msgr2=0x7f7b000c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 -- 172.21.15.139:0/812397004 shutdown_connections 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.976+0000 7f7b073f0640 1 -- 172.21.15.139:0/812397004 wait complete. 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.977+0000 7f7b073f0640 1 Processor -- start 2024-04-26T21:59:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.977+0000 7f7b073f0640 1 -- start start 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b073f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b000c3c70 0x7f7b00155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b073f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0013fe20 0x7f7b000c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b073f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 0x7f7b000c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b073f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0015adb0 con 0x7f7b000c3c70 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b073f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0015aad0 con 0x7f7b0013fe20 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b073f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7b00140200 con 0x7f7b00157010 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0598d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 0x7f7b000c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0498b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0013fe20 0x7f7b000c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0598d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 0x7f7b000c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:40314/0 (socket says 172.21.15.139:40314) 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0598d640 1 -- 172.21.15.139:0/862529244 learned_addr learned my addr 172.21.15.139:0/862529244 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0518c640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b000c3c70 0x7f7b00155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0498b640 1 -- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 msgr2=0x7f7b000c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0498b640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 0x7f7b000c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0498b640 1 -- 172.21.15.139:0/862529244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b000c3c70 msgr2=0x7f7b00155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0498b640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b000c3c70 0x7f7b00155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.978+0000 7f7b0498b640 1 -- 172.21.15.139:0/862529244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b001ec8d0 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7b0598d640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 0x7f7b000c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7b0498b640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0013fe20 0x7f7b000c7140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7af000c370 tx=0x7f7af000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0017070 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af0005ce0 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0007310 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b001ecbb0 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7af001a070 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b001ed170 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.979+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7af0012820 con 0x7f7b0013fe20 2024-04-26T21:59:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.980+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b001ed4b0 con 0x7f7b0013fe20 2024-04-26T21:59:33.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:33.982+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7b001ed4b0 con 0x7f7b0013fe20 2024-04-26T21:59:34.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.055+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346", "id": 2} v 0) -- 0x7f7b000507a0 con 0x7f7b0013fe20 2024-04-26T21:59:34.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.059+0000 7f7af5ffb640 1 -- 172.21.15.139:0/862529244 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7af0022020 con 0x7f7b0013fe20 2024-04-26T21:59:34.059 INFO:teuthology.orchestra.run.smithi139.stdout:2 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0013fe20 msgr2=0x7f7b000c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0013fe20 0x7f7b000c7140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7af000c370 tx=0x7f7af000c830 comp rx=0 tx=0).stop 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 shutdown_connections 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b00157010 0x7f7b000c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0013fe20 0x7f7b000c7140 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 --2- 172.21.15.139:0/862529244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b000c3c70 0x7f7b00155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 >> 172.21.15.139:0/862529244 conn(0x7f7b000c5d70 msgr2=0x7f7b00158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 shutdown_connections 2024-04-26T21:59:34.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.060+0000 7f7b073f0640 1 -- 172.21.15.139:0/862529244 wait complete. 2024-04-26T21:59:34.070 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 5ef0dd21-2c67-4f9a-86e0-9febe0b401ac 3 2024-04-26T21:59:34.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.167+0000 7eff7767b640 1 Processor -- start 2024-04-26T21:59:34.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.167+0000 7eff7767b640 1 -- start start 2024-04-26T21:59:34.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff7767b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff7013ddd0 0x7eff70154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff7767b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7eff70071cc0 0x7eff70073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff7767b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7eff70073c40 0x7eff70070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff7767b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7eff700707e0 con 0x7eff70071cc0 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff7767b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7eff70155450 con 0x7eff70073c40 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff7767b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7eff70155730 con 0x7eff7013ddd0 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff75c18640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7eff70073c40 0x7eff70070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff75417640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff7013ddd0 0x7eff70154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff75c18640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7eff70073c40 0x7eff70070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51934/0 (socket says 172.21.15.139:51934) 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff75c18640 1 -- 172.21.15.139:0/1874501026 learned_addr learned my addr 172.21.15.139:0/1874501026 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.168+0000 7eff74c16640 1 --2- 172.21.15.139:0/1874501026 >> v2:172.21.15.135:3300/0 conn(0x7eff70071cc0 0x7eff70073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75c18640 1 -- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff7013ddd0 msgr2=0x7eff70154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75c18640 1 --2- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff7013ddd0 0x7eff70154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75c18640 1 -- 172.21.15.139:0/1874501026 >> v2:172.21.15.135:3300/0 conn(0x7eff70071cc0 msgr2=0x7eff70073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75c18640 1 --2- 172.21.15.139:0/1874501026 >> v2:172.21.15.135:3300/0 conn(0x7eff70071cc0 0x7eff70073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75c18640 1 -- 172.21.15.139:0/1874501026 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff70155a10 con 0x7eff70073c40 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75417640 1 --2- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff7013ddd0 0x7eff70154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff74c16640 1 --2- 172.21.15.139:0/1874501026 >> v2:172.21.15.135:3300/0 conn(0x7eff70071cc0 0x7eff70073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff75c18640 1 --2- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff70073c40 0x7eff70070f70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff700ca280 tx=0x7eff6802a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2469b598a180d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff67fff640 1 -- 172.21.15.139:0/1874501026 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68036070 con 0x7eff70073c40 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff67fff640 1 -- 172.21.15.139:0/1874501026 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff68031600 con 0x7eff70073c40 2024-04-26T21:59:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.169+0000 7eff67fff640 1 -- 172.21.15.139:0/1874501026 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff680319d0 con 0x7eff70073c40 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 -- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff70073c40 msgr2=0x7eff70070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 --2- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff70073c40 0x7eff70070f70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff700ca280 tx=0x7eff6802a610 comp rx=0 tx=0).stop 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 -- 172.21.15.139:0/1874501026 shutdown_connections 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 --2- 172.21.15.139:0/1874501026 >> v2:172.21.15.135:3300/0 conn(0x7eff70071cc0 0x7eff70073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 --2- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff70073c40 0x7eff70070f70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 --2- 172.21.15.139:0/1874501026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff7013ddd0 0x7eff70154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 -- 172.21.15.139:0/1874501026 >> 172.21.15.139:0/1874501026 conn(0x7eff700ca2f0 msgr2=0x7eff700ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 -- 172.21.15.139:0/1874501026 shutdown_connections 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.170+0000 7eff7767b640 1 -- 172.21.15.139:0/1874501026 wait complete. 2024-04-26T21:59:34.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.171+0000 7eff7767b640 1 Processor -- start 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.171+0000 7eff7767b640 1 -- start start 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff7767b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff70071cc0 0x7eff701edac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff7767b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff70073c40 0x7eff701df710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff7767b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 0x7eff701dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff7767b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff701551c0 con 0x7eff70071cc0 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff7767b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff70155040 con 0x7eff7013ddd0 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff7767b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7eff700ab470 con 0x7eff70073c40 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 0x7eff701dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff74c16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff70073c40 0x7eff701df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 0x7eff701dfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51936/0 (socket says 172.21.15.139:51936) 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 -- 172.21.15.139:0/417556104 learned_addr learned my addr 172.21.15.139:0/417556104 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75417640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff70071cc0 0x7eff701edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 -- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff70073c40 msgr2=0x7eff701df710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff70073c40 0x7eff701df710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 -- 172.21.15.139:0/417556104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff70071cc0 msgr2=0x7eff701edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff70071cc0 0x7eff701edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 -- 172.21.15.139:0/417556104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701e0190 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.172+0000 7eff75c18640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 0x7eff701dfc50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7eff68009f40 tx=0x7eff68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6803f020 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff68037040 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701e1e90 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68036070 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff70156380 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff68042020 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.173+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7eff68044b10 con 0x7eff7013ddd0 2024-04-26T21:59:34.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.174+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff700507a0 con 0x7eff7013ddd0 2024-04-26T21:59:34.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.176+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7eff68049030 con 0x7eff7013ddd0 2024-04-26T21:59:34.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.248+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5ef0dd21-2c67-4f9a-86e0-9febe0b401ac", "id": 3} v 0) -- 0x7eff70074110 con 0x7eff7013ddd0 2024-04-26T21:59:34.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff65ffb640 1 -- 172.21.15.139:0/417556104 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ef0dd21-2c67-4f9a-86e0-9febe0b401ac", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7eff701e0190 con 0x7eff7013ddd0 2024-04-26T21:59:34.252 INFO:teuthology.orchestra.run.smithi139.stdout:3 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 msgr2=0x7eff701dfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 0x7eff701dfc50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7eff68009f40 tx=0x7eff68004a40 comp rx=0 tx=0).stop 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 shutdown_connections 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff7013ddd0 0x7eff701dfc50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff70073c40 0x7eff701df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 --2- 172.21.15.139:0/417556104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff70071cc0 0x7eff701edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.252+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 >> 172.21.15.139:0/417556104 conn(0x7eff700ca2f0 msgr2=0x7eff700c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.253+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 shutdown_connections 2024-04-26T21:59:34.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.253+0000 7eff7767b640 1 -- 172.21.15.139:0/417556104 wait complete. 2024-04-26T21:59:34.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 16e214f8-07e4-425a-a636-9841b787e311 4 2024-04-26T21:59:34.305 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.305+0000 7f84fb0f7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.360+0000 7ff62e7da640 1 Processor -- start 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.360+0000 7ff62e7da640 1 -- start start 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62e7da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff628070f70 0x7ff6280713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62e7da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6280707e0 con 0x7ff628071cc0 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62e7da640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff628155450 con 0x7ff62813ddd0 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62e7da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff628155210 con 0x7ff628070f70 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff627fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff62813ddd0 0x7ff628154b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:43400/0 (socket says 172.21.15.139:43400) 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff627fff640 1 -- 172.21.15.139:0/1618067665 learned_addr learned my addr 172.21.15.139:0/1618067665 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62cd77640 1 --2- 172.21.15.139:0/1618067665 >> v2:172.21.15.135:3300/0 conn(0x7ff628070f70 0x7ff6280713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889843247 0 0) 0x7ff628155450 con 0x7ff62813ddd0 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff610003520 con 0x7ff62813ddd0 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734335693 0 0) 0x7ff6280707e0 con 0x7ff628071cc0 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff628155450 con 0x7ff628071cc0 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62cd77640 1 -- 172.21.15.139:0/1618067665 >> v1:172.21.15.139:6789/0 conn(0x7ff62813ddd0 legacy=0x7ff628154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62cd77640 1 -- 172.21.15.139:0/1618067665 >> v1:172.21.15.135:6789/0 conn(0x7ff628071cc0 legacy=0x7ff628073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.361+0000 7ff62cd77640 1 -- 172.21.15.139:0/1618067665 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff628155730 con 0x7ff628070f70 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.362+0000 7ff62cd77640 1 --2- 172.21.15.139:0/1618067665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628070f70 0x7ff6280713c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff620009d90 tx=0x7ff62002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53632f45dbddd85e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.362+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620007740 con 0x7ff628070f70 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.362+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6200078e0 con 0x7ff628070f70 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.362+0000 7ff626ffd640 1 -- 172.21.15.139:0/1618067665 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620007cb0 con 0x7ff628070f70 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 -- 172.21.15.139:0/1618067665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628070f70 msgr2=0x7ff6280713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 --2- 172.21.15.139:0/1618067665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628070f70 0x7ff6280713c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff620009d90 tx=0x7ff62002a660 comp rx=0 tx=0).stop 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 -- 172.21.15.139:0/1618067665 shutdown_connections 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 --2- 172.21.15.139:0/1618067665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628070f70 0x7ff6280713c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 -- 172.21.15.139:0/1618067665 >> 172.21.15.139:0/1618067665 conn(0x7ff6280ca2f0 msgr2=0x7ff6280ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 -- 172.21.15.139:0/1618067665 shutdown_connections 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.363+0000 7ff62e7da640 1 -- 172.21.15.139:0/1618067665 wait complete. 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.364+0000 7ff62e7da640 1 Processor -- start 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.364+0000 7ff62e7da640 1 -- start start 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62e7da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 0x7ff62806c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62e7da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628071cc0 0x7ff6281f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62e7da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 0x7ff628153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62e7da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff628073e10 con 0x7ff628071cc0 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62e7da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff628071900 con 0x7ff62813ddd0 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62e7da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff628073f90 con 0x7ff628070f70 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff627fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 0x7ff62806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 0x7ff628153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff627fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 0x7ff62806c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:40352/0 (socket says 172.21.15.139:40352) 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 0x7ff628153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51950/0 (socket says 172.21.15.139:51950) 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff627fff640 1 -- 172.21.15.139:0/3939514618 learned_addr learned my addr 172.21.15.139:0/3939514618 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff6277fe640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628071cc0 0x7ff6281f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 -- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 msgr2=0x7ff62806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 0x7ff62806c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 -- 172.21.15.139:0/3939514618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628071cc0 msgr2=0x7ff6281f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628071cc0 0x7ff6281f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 -- 172.21.15.139:0/3939514618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62813e8f0 con 0x7ff62813ddd0 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff627fff640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 0x7ff62806c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.365+0000 7ff62cd77640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 0x7ff628153d60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff620031000 tx=0x7ff620004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620040070 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff620031c00 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281ec810 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620009450 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff628157db0 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff620039040 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.366+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7ff620045be0 con 0x7ff62813ddd0 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.367+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6280507a0 con 0x7ff62813ddd0 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.370+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff620053020 con 0x7ff62813ddd0 2024-04-26T21:59:34.380 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.381+0000 7fc6f3530040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:59:34.393 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.393+0000 7f84fb0f7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:59:34.436 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.436+0000 7f84fb0f7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:59:34.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.442+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "16e214f8-07e4-425a-a636-9841b787e311", "id": 4} v 0) -- 0x7ff628074110 con 0x7ff62813ddd0 2024-04-26T21:59:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.448+0000 7ff624ff9640 1 -- 172.21.15.139:0/3939514618 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "16e214f8-07e4-425a-a636-9841b787e311", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7ff620045e20 con 0x7ff62813ddd0 2024-04-26T21:59:34.448 INFO:teuthology.orchestra.run.smithi139.stdout:4 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 msgr2=0x7ff628153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 0x7ff628153d60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff620031000 tx=0x7ff620004a40 comp rx=0 tx=0).stop 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 shutdown_connections 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff62813ddd0 0x7ff628153d60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff628071cc0 0x7ff6281f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 --2- 172.21.15.139:0/3939514618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff628070f70 0x7ff62806c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 >> 172.21.15.139:0/3939514618 conn(0x7ff6280ca2f0 msgr2=0x7ff6280af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 shutdown_connections 2024-04-26T21:59:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.449+0000 7ff62e7da640 1 -- 172.21.15.139:0/3939514618 wait complete. 2024-04-26T21:59:34.459 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new d5a4f44c-f092-499b-8b8a-a122bee206e9 5 2024-04-26T21:59:34.474 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.474+0000 7fc6f3530040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:59:34.519 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.519+0000 7fc6f3530040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:59:34.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 Processor -- start 2024-04-26T21:59:34.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 -- start start 2024-04-26T21:59:34.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1e013fe20 0x7fe1e00c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe1e0157010 0x7fe1e0157440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e015a230 con 0x7fe1e00c3c70 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00c39b0 con 0x7fe1e0157010 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1e6514640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e015b120 con 0x7fe1e013fe20 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1dffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1e013fe20 0x7fe1e00c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1dffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1e013fe20 0x7fe1e00c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51964/0 (socket says 172.21.15.139:51964) 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.557+0000 7fe1dffff640 1 -- 172.21.15.139:0/81780528 learned_addr learned my addr 172.21.15.139:0/81780528 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1e4ab1640 1 --2- 172.21.15.139:0/81780528 >> v2:172.21.15.135:3300/0 conn(0x7fe1e0157010 0x7fe1e0157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1dffff640 1 -- 172.21.15.139:0/81780528 >> v2:172.21.15.135:3300/0 conn(0x7fe1e0157010 msgr2=0x7fe1e0157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1dffff640 1 --2- 172.21.15.139:0/81780528 >> v2:172.21.15.135:3300/0 conn(0x7fe1e0157010 0x7fe1e0157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1dffff640 1 -- 172.21.15.139:0/81780528 >> v1:172.21.15.135:6789/0 conn(0x7fe1e00c3c70 legacy=0x7fe1e006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1dffff640 1 -- 172.21.15.139:0/81780528 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e015a510 con 0x7fe1e013fe20 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1dffff640 1 --2- 172.21.15.139:0/81780528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c5430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00c2750 tx=0x7fe1d002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3b8f2c397d4114 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.558+0000 7fe1deffd640 1 -- 172.21.15.139:0/81780528 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0037070 con 0x7fe1e013fe20 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1deffd640 1 -- 172.21.15.139:0/81780528 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d0032440 con 0x7fe1e013fe20 2024-04-26T21:59:34.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1deffd640 1 -- 172.21.15.139:0/81780528 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0032810 con 0x7fe1e013fe20 2024-04-26T21:59:34.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 -- 172.21.15.139:0/81780528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e013fe20 msgr2=0x7fe1e00c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 --2- 172.21.15.139:0/81780528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c5430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00c2750 tx=0x7fe1d002a270 comp rx=0 tx=0).stop 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 -- 172.21.15.139:0/81780528 shutdown_connections 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 --2- 172.21.15.139:0/81780528 >> v2:172.21.15.135:3300/0 conn(0x7fe1e0157010 0x7fe1e0157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 --2- 172.21.15.139:0/81780528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c5430 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 -- 172.21.15.139:0/81780528 >> 172.21.15.139:0/81780528 conn(0x7fe1e00c5d70 msgr2=0x7fe1e00c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 -- 172.21.15.139:0/81780528 shutdown_connections 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.559+0000 7fe1e6514640 1 -- 172.21.15.139:0/81780528 wait complete. 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.560+0000 7fe1e6514640 1 Processor -- start 2024-04-26T21:59:34.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 -- start start 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1e00c3c70 0x7fe1e0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e0157010 0x7fe1e01efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e015adb0 con 0x7fe1e013fe20 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e015aad0 con 0x7fe1e0157010 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e6514640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e0140200 con 0x7fe1e00c3c70 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1dffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1e00c3c70 0x7fe1e0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1e4ab1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e0157010 0x7fe1e01efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1dffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1e00c3c70 0x7fe1e0073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:40366/0 (socket says 172.21.15.139:40366) 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1dffff640 1 -- 172.21.15.139:0/953270825 learned_addr learned my addr 172.21.15.139:0/953270825 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.561+0000 7fe1df7fe640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e4ab1640 1 -- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1e00c3c70 msgr2=0x7fe1e0073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e4ab1640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1e00c3c70 0x7fe1e0073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e4ab1640 1 -- 172.21.15.139:0/953270825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1e013fe20 msgr2=0x7fe1e00c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e4ab1640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e4ab1640 1 -- 172.21.15.139:0/953270825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0072a00 con 0x7fe1e0157010 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e4ab1640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e0157010 0x7fe1e01efee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007d20 tx=0x7fe1d800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800edd0 con 0x7fe1e0157010 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d8003990 con 0x7fe1e0157010 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e0074410 con 0x7fe1e0157010 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8003d80 con 0x7fe1e0157010 2024-04-26T21:59:34.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e00c7a20 con 0x7fe1e0157010 2024-04-26T21:59:34.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1d801b020 con 0x7fe1e0157010 2024-04-26T21:59:34.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.562+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe1d801d410 con 0x7fe1e0157010 2024-04-26T21:59:34.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.563+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e00c5a30 con 0x7fe1e0157010 2024-04-26T21:59:34.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.565+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe1e00c5a30 con 0x7fe1e0157010 2024-04-26T21:59:34.603 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.603+0000 7f84fb0f7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:59:34.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.636+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d5a4f44c-f092-499b-8b8a-a122bee206e9", "id": 5} v 0) -- 0x7fe1e00507a0 con 0x7fe1e0157010 2024-04-26T21:59:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.641+0000 7fe1dcff9640 1 -- 172.21.15.139:0/953270825 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5a4f44c-f092-499b-8b8a-a122bee206e9", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe1d8014420 con 0x7fe1e0157010 2024-04-26T21:59:34.641 INFO:teuthology.orchestra.run.smithi139.stdout:5 2024-04-26T21:59:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e0157010 msgr2=0x7fe1e01efee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e0157010 0x7fe1e01efee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007d20 tx=0x7fe1d800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 shutdown_connections 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1e0157010 0x7fe1e01efee0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1e013fe20 0x7fe1e00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 --2- 172.21.15.139:0/953270825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1e00c3c70 0x7fe1e0073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 >> 172.21.15.139:0/953270825 conn(0x7fe1e00c5d70 msgr2=0x7fe1e0158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 shutdown_connections 2024-04-26T21:59:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.642+0000 7fe1e6514640 1 -- 172.21.15.139:0/953270825 wait complete. 2024-04-26T21:59:34.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 76127437-7143-4255-b484-d3e40f69bc25 6 2024-04-26T21:59:34.653 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.653+0000 7f84fb0f7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:59:34.693 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.693+0000 7fc6f3530040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:59:34.704 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.704+0000 7f84fb0f7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:59:34.747 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.748+0000 7fc6f3530040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:59:34.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.789+0000 7f7cd7b7f640 1 Processor -- start 2024-04-26T21:59:34.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.790+0000 7f7cd7b7f640 1 -- start start 2024-04-26T21:59:34.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.790+0000 7f7cd7b7f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7cd0074620 0x7f7cd01582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.790+0000 7f7cd7b7f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0158830 0x7f7cd015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.790+0000 7f7cd7b7f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd0154920 con 0x7f7cd0151e10 2024-04-26T21:59:34.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd7b7f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd0074e90 con 0x7f7cd0074620 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd7b7f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00c6b60 con 0x7f7cd0158830 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd591b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7cd0151e10 0x7f7cd00ac4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:33992/0 (socket says 172.21.15.139:33992) 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd611c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0158830 0x7f7cd015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd591b640 1 -- 172.21.15.139:0/4183596182 learned_addr learned my addr 172.21.15.139:0/4183596182 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd511a640 1 --2- 172.21.15.139:0/4183596182 >> v2:172.21.15.135:3300/0 conn(0x7f7cd0074620 0x7f7cd01582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd4919640 1 -- 172.21.15.139:0/4183596182 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407788389 0 0) 0x7f7cd0154920 con 0x7f7cd0151e10 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd4919640 1 -- 172.21.15.139:0/4183596182 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cbc003520 con 0x7f7cd0151e10 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd611c640 1 -- 172.21.15.139:0/4183596182 >> v2:172.21.15.135:3300/0 conn(0x7f7cd0074620 msgr2=0x7f7cd01582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd611c640 1 --2- 172.21.15.139:0/4183596182 >> v2:172.21.15.135:3300/0 conn(0x7f7cd0074620 0x7f7cd01582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd611c640 1 -- 172.21.15.139:0/4183596182 >> v1:172.21.15.139:6789/0 conn(0x7f7cd0151e10 legacy=0x7f7cd00ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.791+0000 7f7cd611c640 1 -- 172.21.15.139:0/4183596182 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd00c62c0 con 0x7f7cd0158830 2024-04-26T21:59:34.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.792+0000 7f7cd611c640 1 --2- 172.21.15.139:0/4183596182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0158830 0x7f7cd015f2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009d90 tx=0x7f7cc802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df7fff399a9dea83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.792+0000 7f7cd4919640 1 -- 172.21.15.139:0/4183596182 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8007740 con 0x7f7cd0158830 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.792+0000 7f7cd4919640 1 -- 172.21.15.139:0/4183596182 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc80078e0 con 0x7f7cd0158830 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.792+0000 7f7cd4919640 1 -- 172.21.15.139:0/4183596182 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8007cb0 con 0x7f7cd0158830 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/4183596182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0158830 msgr2=0x7f7cd015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/4183596182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0158830 0x7f7cd015f2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009d90 tx=0x7f7cc802a660 comp rx=0 tx=0).stop 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/4183596182 shutdown_connections 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/4183596182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0158830 0x7f7cd015f2a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/4183596182 >> v2:172.21.15.135:3300/0 conn(0x7f7cd0074620 0x7f7cd01582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/4183596182 >> 172.21.15.139:0/4183596182 conn(0x7f7cd00ca2f0 msgr2=0x7f7cd00ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/4183596182 shutdown_connections 2024-04-26T21:59:34.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.793+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/4183596182 wait complete. 2024-04-26T21:59:34.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.794+0000 7f7cd7b7f640 1 Processor -- start 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.794+0000 7f7cd7b7f640 1 -- start start 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0074620 0x7f7cd014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd0151e10 0x7f7cd01e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd0158830 0x7f7cd00af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00c5f50 con 0x7f7cd0151e10 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00c6880 con 0x7f7cd0158830 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00acc00 con 0x7f7cd0074620 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd591b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0074620 0x7f7cd014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd591b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0074620 0x7f7cd014ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:50204/0 (socket says 172.21.15.139:50204) 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd591b640 1 -- 172.21.15.139:0/2016555576 learned_addr learned my addr 172.21.15.139:0/2016555576 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd0158830 0x7f7cd00af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd511a640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd0151e10 0x7f7cd01e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 -- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0074620 msgr2=0x7f7cd014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0074620 0x7f7cd014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 -- 172.21.15.139:0/2016555576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd0151e10 msgr2=0x7f7cd01e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd0151e10 0x7f7cd01e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 -- 172.21.15.139:0/2016555576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd014e7a0 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd611c640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd0158830 0x7f7cd00af110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8031000 tx=0x7f7cc8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc803f070 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8031c00 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.795+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd01e1b00 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.796+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8009450 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.796+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd00af650 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.796+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cc8044460 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.796+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f7cc803baa0 con 0x7f7cd0158830 2024-04-26T21:59:34.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.796+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd01e1910 con 0x7f7cd0158830 2024-04-26T21:59:34.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.798+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7cc8063ec0 con 0x7f7cd0158830 2024-04-26T21:59:34.803 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.803+0000 7fc6f3530040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:59:34.817 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:34.818+0000 7f84fb0f7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:59:34.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.871+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "76127437-7143-4255-b484-d3e40f69bc25", "id": 6} v 0) -- 0x7f7cd00766c0 con 0x7f7cd0158830 2024-04-26T21:59:34.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.876+0000 7f7cc67fc640 1 -- 172.21.15.139:0/2016555576 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "76127437-7143-4255-b484-d3e40f69bc25", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f7cc8052070 con 0x7f7cd0158830 2024-04-26T21:59:34.876 INFO:teuthology.orchestra.run.smithi139.stdout:6 2024-04-26T21:59:34.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd0158830 msgr2=0x7f7cd00af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd0158830 0x7f7cd00af110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8031000 tx=0x7f7cc8004a40 comp rx=0 tx=0).stop 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 shutdown_connections 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd0158830 0x7f7cd00af110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd0151e10 0x7f7cd01e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 --2- 172.21.15.139:0/2016555576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd0074620 0x7f7cd014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 >> 172.21.15.139:0/2016555576 conn(0x7f7cd00ca2f0 msgr2=0x7f7cd00ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 shutdown_connections 2024-04-26T21:59:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.877+0000 7f7cd7b7f640 1 -- 172.21.15.139:0/2016555576 wait complete. 2024-04-26T21:59:34.887 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 1f27f035-a26f-481c-9677-46400f4d8871 7 2024-04-26T21:59:34.921 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:34.922+0000 7fc6f3530040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:59:34.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.984+0000 7fcbda693640 1 Processor -- start 2024-04-26T21:59:34.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.984+0000 7fcbda693640 1 -- start start 2024-04-26T21:59:34.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbda693640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd40c3720 0x7fcbd40c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbda693640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd41577b0 con 0x7fcbd40c46c0 2024-04-26T21:59:34.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbda693640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd40ab5f0 con 0x7fcbd413ff20 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbda693640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd4157560 con 0x7fcbd40c3720 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbd8c30640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd40c3720 0x7fcbd40c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbd3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcbd413ff20 0x7fcbd415a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:33996/0 (socket says 172.21.15.139:33996) 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbd3fff640 1 -- 172.21.15.139:0/103654256 learned_addr learned my addr 172.21.15.139:0/103654256 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2922263287 0 0) 0x7fcbd40ab5f0 con 0x7fcbd413ff20 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.985+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbbc003520 con 0x7fcbd413ff20 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429315005 0 0) 0x7fcbd41577b0 con 0x7fcbd40c46c0 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbd40ab5f0 con 0x7fcbd40c46c0 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd8c30640 1 -- 172.21.15.139:0/103654256 >> v1:172.21.15.139:6789/0 conn(0x7fcbd413ff20 legacy=0x7fcbd415a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd8c30640 1 -- 172.21.15.139:0/103654256 >> v1:172.21.15.135:6789/0 conn(0x7fcbd40c46c0 legacy=0x7fcbd415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd8c30640 1 -- 172.21.15.139:0/103654256 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4140300 con 0x7fcbd40c3720 2024-04-26T21:59:34.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd8c30640 1 --2- 172.21.15.139:0/103654256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd40c3720 0x7fcbd40c3b70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009dc0 tx=0x7fcbcc02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a595535f46df259 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc02ae60 con 0x7fcbd40c3720 2024-04-26T21:59:34.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbcc007740 con 0x7fcbd40c3720 2024-04-26T21:59:34.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.986+0000 7fcbd2ffd640 1 -- 172.21.15.139:0/103654256 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc007b10 con 0x7fcbd40c3720 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 -- 172.21.15.139:0/103654256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd40c3720 msgr2=0x7fcbd40c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 --2- 172.21.15.139:0/103654256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd40c3720 0x7fcbd40c3b70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009dc0 tx=0x7fcbcc02a640 comp rx=0 tx=0).stop 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 -- 172.21.15.139:0/103654256 shutdown_connections 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 --2- 172.21.15.139:0/103654256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd40c3720 0x7fcbd40c3b70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 -- 172.21.15.139:0/103654256 >> 172.21.15.139:0/103654256 conn(0x7fcbd40ca2f0 msgr2=0x7fcbd40ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 -- 172.21.15.139:0/103654256 shutdown_connections 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.987+0000 7fcbda693640 1 -- 172.21.15.139:0/103654256 wait complete. 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.988+0000 7fcbda693640 1 Processor -- start 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.988+0000 7fcbda693640 1 -- start start 2024-04-26T21:59:34.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbda693640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd40c3720 0x7fcbd41564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbda693640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 0x7fcbd413aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbda693640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbd40c8ad0 0x7fcbd40c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbda693640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40ac710 con 0x7fcbd40c3720 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbda693640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40ab960 con 0x7fcbd40c8ad0 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbda693640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd40c40b0 con 0x7fcbd413ff20 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbd37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 0x7fcbd413aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbd8c30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbd40c8ad0 0x7fcbd40c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbd37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 0x7fcbd413aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:50226/0 (socket says 172.21.15.139:50226) 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbd37fe640 1 -- 172.21.15.139:0/2079053070 learned_addr learned my addr 172.21.15.139:0/2079053070 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.989+0000 7fcbd3fff640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd40c3720 0x7fcbd41564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd37fe640 1 -- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbd40c8ad0 msgr2=0x7fcbd40c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd37fe640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbd40c8ad0 0x7fcbd40c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd37fe640 1 -- 172.21.15.139:0/2079053070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd40c3720 msgr2=0x7fcbd41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd37fe640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd40c3720 0x7fcbd41564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:34.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd37fe640 1 -- 172.21.15.139:0/2079053070 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4159ef0 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd8c30640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbd40c8ad0 0x7fcbd40c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd37fe640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 0x7fcbd413aed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c420 tx=0x7fcbc400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4017070 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc40052d0 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4003810 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd40aed30 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd40aea50 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcbc401a070 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.990+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fcbc4011c20 con 0x7fcbd413ff20 2024-04-26T21:59:34.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.991+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd40766c0 con 0x7fcbd413ff20 2024-04-26T21:59:34.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:34.993+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcbc4021080 con 0x7fcbd413ff20 2024-04-26T21:59:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.065+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1f27f035-a26f-481c-9677-46400f4d8871", "id": 7} v 0) -- 0x7fcbd4156bf0 con 0x7fcbd413ff20 2024-04-26T21:59:35.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.074+0000 7fcbd0ff9640 1 -- 172.21.15.139:0/2079053070 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1f27f035-a26f-481c-9677-46400f4d8871", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fcbc4011590 con 0x7fcbd413ff20 2024-04-26T21:59:35.074 INFO:teuthology.orchestra.run.smithi139.stdout:7 2024-04-26T21:59:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 msgr2=0x7fcbd413aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 0x7fcbd413aed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c420 tx=0x7fcbc400c8e0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 shutdown_connections 2024-04-26T21:59:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbd40c8ad0 0x7fcbd40c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbd413ff20 0x7fcbd413aed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 --2- 172.21.15.139:0/2079053070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbd40c3720 0x7fcbd41564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 >> 172.21.15.139:0/2079053070 conn(0x7fcbd40ca2f0 msgr2=0x7fcbd40c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 shutdown_connections 2024-04-26T21:59:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T21:59:35.075+0000 7fcbda693640 1 -- 172.21.15.139:0/2079053070 wait complete. 2024-04-26T21:59:35.085 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T21:59:35.086 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T21:59:35.088 INFO:tasks.ceph.osd.0:Started 2024-04-26T21:59:35.088 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T21:59:35.088 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T21:59:35.092 INFO:tasks.ceph.osd.1:Started 2024-04-26T21:59:35.092 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T21:59:35.092 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T21:59:35.094 INFO:tasks.ceph.osd.2:Started 2024-04-26T21:59:35.094 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T21:59:35.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T21:59:35.097 INFO:tasks.ceph.osd.3:Started 2024-04-26T21:59:35.097 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T21:59:35.097 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T21:59:35.128 INFO:tasks.ceph.osd.4:Started 2024-04-26T21:59:35.128 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T21:59:35.128 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T21:59:35.132 INFO:tasks.ceph.osd.5:Started 2024-04-26T21:59:35.133 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T21:59:35.133 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T21:59:35.135 INFO:tasks.ceph.osd.6:Started 2024-04-26T21:59:35.136 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T21:59:35.136 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T21:59:35.138 INFO:tasks.ceph.osd.7:Started 2024-04-26T21:59:35.138 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:35.231 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:35.232+0000 7f84fb0f7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:59:35.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.246+0000 7f2146859640 1 Processor -- start 2024-04-26T21:59:35.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.246+0000 7f2146859640 1 -- start start 2024-04-26T21:59:35.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f2146859640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f21400c5520 0x7f2140156c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:35.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f2146859640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f214015a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f2146859640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214006f0c0 con 0x7f21400c5140 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f2146859640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f214015a190 con 0x7f21400c5520 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f2146859640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f214015a470 con 0x7f21400ac620 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f213f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f21400c5520 0x7f2140156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f213f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f21400c5520 0x7f2140156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53304/0 (socket says 172.21.15.135:53304) 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f213f7fe640 1 -- 172.21.15.135:0/162839217 learned_addr learned my addr 172.21.15.135:0/162839217 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f2144df6640 1 --2- 172.21.15.135:0/162839217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f214015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f213f7fe640 1 -- 172.21.15.135:0/162839217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 msgr2=0x7f214015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.247+0000 7f213f7fe640 1 --2- 172.21.15.135:0/162839217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f214015a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.248+0000 7f213f7fe640 1 -- 172.21.15.135:0/162839217 >> v1:172.21.15.139:6789/0 conn(0x7f21400c5140 legacy=0x7f214006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:35.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.248+0000 7f213f7fe640 1 -- 172.21.15.135:0/162839217 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214006f3a0 con 0x7f21400c5520 2024-04-26T21:59:35.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.248+0000 7f213f7fe640 1 --2- 172.21.15.135:0/162839217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5520 0x7f2140156c00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2130009870 tx=0x7f213002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba21755ceccd364 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:35.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.248+0000 7f213effd640 1 -- 172.21.15.135:0/162839217 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130037070 con 0x7f21400c5520 2024-04-26T21:59:35.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.248+0000 7f213effd640 1 -- 172.21.15.135:0/162839217 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2130032440 con 0x7f21400c5520 2024-04-26T21:59:35.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.248+0000 7f213effd640 1 -- 172.21.15.135:0/162839217 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130032810 con 0x7f21400c5520 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 -- 172.21.15.135:0/162839217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5520 msgr2=0x7f2140156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 --2- 172.21.15.135:0/162839217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5520 0x7f2140156c00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2130009870 tx=0x7f213002a1c0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 -- 172.21.15.135:0/162839217 shutdown_connections 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 --2- 172.21.15.135:0/162839217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f214015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 --2- 172.21.15.135:0/162839217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5520 0x7f2140156c00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 -- 172.21.15.135:0/162839217 >> 172.21.15.135:0/162839217 conn(0x7f214006e090 msgr2=0x7f214006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 -- 172.21.15.135:0/162839217 shutdown_connections 2024-04-26T21:59:35.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.249+0000 7f2146859640 1 -- 172.21.15.135:0/162839217 wait complete. 2024-04-26T21:59:35.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.250+0000 7f2146859640 1 Processor -- start 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.250+0000 7f2146859640 1 -- start start 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2146859640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f21401ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2146859640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 0x7f214014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2146859640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21400c5520 0x7f21401edd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2146859640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f214015afc0 con 0x7f21400c5140 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2146859640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f214015ae40 con 0x7f21400c5520 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2146859640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2140153f30 con 0x7f21400ac620 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 0x7f214014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 0x7f214014de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53318/0 (socket says 172.21.15.135:53318) 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 -- 172.21.15.135:0/487100914 learned_addr learned my addr 172.21.15.135:0/487100914 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213ffff640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f21401ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f2144df6640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21400c5520 0x7f21401edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 -- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 msgr2=0x7f21401ec8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f21401ec8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 -- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21400c5520 msgr2=0x7f21401edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21400c5520 0x7f21401edd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 -- 172.21.15.135:0/487100914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214014a0a0 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.251+0000 7f213f7fe640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 0x7f214014de10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f213002a200 tx=0x7f21300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130041030 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2130030440 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214014a340 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130037040 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214014a900 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2130044070 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.252+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f213003dd20 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.253+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21400507a0 con 0x7f21400c5140 2024-04-26T21:59:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.255+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f213004b070 con 0x7f21400c5140 2024-04-26T21:59:35.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.314+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21400681f0 con 0x7f21400c5140 2024-04-26T21:59:35.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.315+0000 7f213cff9640 1 -- 172.21.15.135:0/487100914 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f213003d650 con 0x7f21400c5140 2024-04-26T21:59:35.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:35.315 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":10,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:35.067593+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 msgr2=0x7f214014de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 0x7f214014de10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f213002a200 tx=0x7f21300046c0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 shutdown_connections 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21400c5520 0x7f21401edd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21400c5140 0x7f214014de10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 --2- 172.21.15.135:0/487100914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21400ac620 0x7f21401ec8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 >> 172.21.15.135:0/487100914 conn(0x7f214006e090 msgr2=0x7f21400c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 shutdown_connections 2024-04-26T21:59:35.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:35.316+0000 7f2146859640 1 -- 172.21.15.135:0/487100914 wait complete. 2024-04-26T21:59:35.327 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T21:59:35.328 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T21:59:35.344 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:35.344+0000 7fc6f3530040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:59:35.496 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:35.497+0000 7f84fb0f7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:59:35.548 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:35.549+0000 7f84fb0f7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:59:35.601 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:35.601+0000 7fc6f3530040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:59:35.652 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:35.653+0000 7fc6f3530040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:59:35.665 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:35.665+0000 7f84fb0f7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:59:35.701 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-26T21:59:35.701+0000 7fa6cf553640 -1 Falling back to public interface 2024-04-26T21:59:35.706 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-26T21:59:35.706+0000 7f75dffa3640 -1 Falling back to public interface 2024-04-26T21:59:35.710 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-26T21:59:35.710+0000 7f37bd65d640 -1 Falling back to public interface 2024-04-26T21:59:35.719 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-26T21:59:35.719+0000 7f92fd812640 -1 Falling back to public interface 2024-04-26T21:59:35.726 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-04-26T21:59:35.726+0000 7ff42c418640 -1 Falling back to public interface 2024-04-26T21:59:35.734 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-04-26T21:59:35.734+0000 7f3fbf214640 -1 Falling back to public interface 2024-04-26T21:59:35.738 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-04-26T21:59:35.738+0000 7f04e8419640 -1 Falling back to public interface 2024-04-26T21:59:35.750 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-04-26T21:59:35.751+0000 7f38a9031640 -1 Falling back to public interface 2024-04-26T21:59:35.776 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:35.776+0000 7fc6f3530040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:59:35.946 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:35.946+0000 7f84fb0f7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:59:35.994 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:35.995+0000 7f84fb0f7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:59:36.055 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.055+0000 7fc6f3530040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:59:36.101 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.102+0000 7fc6f3530040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:59:36.134 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:36.152 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:36.152+0000 7f84fb0f7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:59:36.224 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.224+0000 7fd4659e4640 1 Processor -- start 2024-04-26T21:59:36.224 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.224+0000 7fd4659e4640 1 -- start start 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd4659e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 0x7fd460070a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd4659e4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd460156dd0 con 0x7fd4600743a0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd4659e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4600ab780 con 0x7fd4600c3ab0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd4659e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd460157070 con 0x7fd460157870 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd45effd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd4600743a0 0x7fd460073270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35538/0 (socket says 172.21.15.135:35538) 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd45effd640 1 -- 172.21.15.135:0/3651618272 learned_addr learned my addr 172.21.15.135:0/3651618272 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd45f7fe640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 0x7fd460070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514997556 0 0) 0x7fd460156dd0 con 0x7fd4600743a0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.225+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd44c003520 con 0x7fd4600743a0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838569343 0 0) 0x7fd4600ab780 con 0x7fd4600c3ab0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd460156dd0 con 0x7fd4600c3ab0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1663579254 0 0) 0x7fd44c003520 con 0x7fd4600743a0 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 msgr2=0x7fd460070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:36.228 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 0x7fd460070a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 >> v1:172.21.15.139:6789/0 conn(0x7fd4600c3ab0 legacy=0x7fd4600702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd460157350 con 0x7fd4600743a0 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fd450001670 con 0x7fd4600743a0 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 >> v1:172.21.15.135:6789/0 conn(0x7fd4600743a0 legacy=0x7fd460073270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 0x7fd44c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd44c004120 0x7fd44c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44c004d40 0x7fd44c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd460157070 con 0x7fd44c004120 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c00bb80 con 0x7fd44c004d40 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c00bd80 con 0x7fd460157870 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45e7fc640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd44c004120 0x7fd44c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45f7fe640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44c004d40 0x7fd44c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45effd640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 0x7fd44c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45e7fc640 1 -- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 msgr2=0x7fd44c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:36.229 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45e7fc640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd460157870 0x7fd44c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45e7fc640 1 -- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44c004d40 msgr2=0x7fd44c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45e7fc640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44c004d40 0x7fd44c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45e7fc640 1 -- 172.21.15.135:0/3651618272 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c003520 con 0x7fd44c004120 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.226+0000 7fd45f7fe640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44c004d40 0x7fd44c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.227+0000 7fd45e7fc640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd44c004120 0x7fd44c004800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd4480018a0 tx=0x7fd44c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b068a7cc8c26ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.227+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480014f0 con 0x7fd44c004120 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.227+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4480052d0 con 0x7fd44c004120 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.227+0000 7fd45dffb640 1 -- 172.21.15.135:0/3651618272 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44802ac60 con 0x7fd44c004120 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd4659e4640 1 -- 172.21.15.135:0/3651618272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd44c004120 msgr2=0x7fd44c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd4659e4640 1 --2- 172.21.15.135:0/3651618272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd44c004120 0x7fd44c004800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd4480018a0 tx=0x7fd44c005830 comp rx=0 tx=0).stop 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd45f7fe640 1 -- 172.21.15.135:0/3651618272 reap_dead start 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd4659e4640 1 -- 172.21.15.135:0/3651618272 shutdown_connections 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd4659e4640 1 -- 172.21.15.135:0/3651618272 >> 172.21.15.135:0/3651618272 conn(0x7fd46006fa50 msgr2=0x7fd460075700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd4659e4640 1 -- 172.21.15.135:0/3651618272 shutdown_connections 2024-04-26T21:59:36.230 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.228+0000 7fd4659e4640 1 -- 172.21.15.135:0/3651618272 wait complete. 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.229+0000 7fd4659e4640 1 Processor -- start 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.229+0000 7fd4659e4640 1 -- start start 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd4659e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd44c004120 0x7fd46013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd4659e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd460157f70 0x7fd460158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd4659e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 0x7fd460153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd4659e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4600aba60 con 0x7fd4601594a0 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd4659e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd460074780 con 0x7fd460157f70 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd4659e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd4600ab430 con 0x7fd44c004120 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 0x7fd460153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 0x7fd460153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53338/0 (socket says 172.21.15.135:53338) 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 -- 172.21.15.135:0/3465503141 learned_addr learned my addr 172.21.15.135:0/3465503141 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45e7fc640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd460157f70 0x7fd460158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.233 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45effd640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd44c004120 0x7fd46013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 -- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd44c004120 msgr2=0x7fd46013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd44c004120 0x7fd46013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 -- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd460157f70 msgr2=0x7fd460158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd460157f70 0x7fd460158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 -- 172.21.15.135:0/3465503141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4601ec4c0 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45e7fc640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd460157f70 0x7fd460158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd45f7fe640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 0x7fd460153b10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd46015b250 tx=0x7fd45800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458009c10 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd458003990 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.230+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458003d80 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.231+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4601f0440 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.231+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4601ec740 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.231+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4580042b0 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.231+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd458015bb0 con 0x7fd4601594a0 2024-04-26T21:59:36.234 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.232+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4600506c0 con 0x7fd4601594a0 2024-04-26T21:59:36.235 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.233+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd4601ec4c0 con 0x7fd4601594a0 2024-04-26T21:59:36.257 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.257+0000 7fc6f3530040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:59:36.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.295+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd46013d440 con 0x7fd4601594a0 2024-04-26T21:59:36.295 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:36.296+0000 7f84fb0f7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:59:36.296 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.296+0000 7fd457fff640 1 -- 172.21.15.135:0/3465503141 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd458019030 con 0x7fd4601594a0 2024-04-26T21:59:36.296 INFO:teuthology.misc.health.smithi135.stdout: 2024-04-26T21:59:36.296 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":10,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:35.067593+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 msgr2=0x7fd460153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 0x7fd460153b10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd46015b250 tx=0x7fd45800f580 comp rx=0 tx=0).stop 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 shutdown_connections 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4601594a0 0x7fd460153b10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd460157f70 0x7fd460158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 --2- 172.21.15.135:0/3465503141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd44c004120 0x7fd46013d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 >> 172.21.15.135:0/3465503141 conn(0x7fd46006fa50 msgr2=0x7fd4600681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:36.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 shutdown_connections 2024-04-26T21:59:36.298 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:36.297+0000 7fd4659e4640 1 -- 172.21.15.135:0/3465503141 wait complete. 2024-04-26T21:59:36.306 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:59:36.350 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:36.347+0000 7f84fb0f7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:59:36.395 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.395+0000 7fc6f3530040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:59:36.417 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:36.417+0000 7f84fb0f7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:59:36.444 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.444+0000 7fc6f3530040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:59:36.462 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:36.463+0000 7f84fb0f7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:59:36.513 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.513+0000 7fc6f3530040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:59:36.558 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:36.558+0000 7fc6f3530040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:59:36.941 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:36.941+0000 7f84fb0f7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:59:37.033 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:37.033+0000 7fc6f3530040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:59:37.058 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:37.058+0000 7f84fb0f7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:59:37.154 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-04-26T21:59:37.154+0000 7fc6f3530040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:59:38.073 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:38.073+0000 7f849bce5640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T21:59:38.073 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-26T21:59:38.073+0000 7f849bce5640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:59:39.201 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-26T21:59:39.201+0000 7fa6cf553640 -1 osd.3 0 log_to_monitors true 2024-04-26T21:59:39.207 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-04-26T21:59:39.207+0000 7ff42c418640 -1 osd.5 0 log_to_monitors true 2024-04-26T21:59:39.225 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-26T21:59:39.225+0000 7f75dffa3640 -1 osd.1 0 log_to_monitors true 2024-04-26T21:59:39.249 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-04-26T21:59:39.249+0000 7f3fbf214640 -1 osd.4 0 log_to_monitors true 2024-04-26T21:59:39.257 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-26T21:59:39.257+0000 7f92fd812640 -1 osd.2 0 log_to_monitors true 2024-04-26T21:59:39.270 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-26T21:59:39.270+0000 7f37bd65d640 -1 osd.0 0 log_to_monitors true 2024-04-26T21:59:39.284 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-04-26T21:59:39.285+0000 7f04e8419640 -1 osd.6 0 log_to_monitors true 2024-04-26T21:59:39.308 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-04-26T21:59:39.308+0000 7f38a9031640 -1 osd.7 0 log_to_monitors true 2024-04-26T21:59:43.082 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-26T21:59:43.081+0000 7f37b95f3640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:59:43.095 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-26T21:59:43.095+0000 7f37b1413640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.112 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:43.120 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-26T21:59:43.121+0000 7fa6cbcfd640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:59:43.121 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-26T21:59:43.122+0000 7f75dbf39640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:59:43.132 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-26T21:59:43.133+0000 7fa6c3309640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.133 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-26T21:59:43.134+0000 7f75d3d59640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.146 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-04-26T21:59:43.146+0000 7f04e3d0c640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:59:43.148 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-04-26T21:59:43.148+0000 7f38a4580640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:59:43.160 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-04-26T21:59:43.160+0000 7f3fbabe7640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:59:43.163 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-04-26T21:59:43.163+0000 7f04dbb2c640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.163 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-04-26T21:59:43.163+0000 7f389c3b1640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.170 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-26T21:59:43.171+0000 7f92f91df640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:59:43.171 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-04-26T21:59:43.171+0000 7f3fb2a07640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.179 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-26T21:59:43.180+0000 7f92f0fff640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.190 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-04-26T21:59:43.191+0000 7ff427d80640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:59:43.204 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-04-26T21:59:43.205+0000 7ff41fba0640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:59:43.208 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.209+0000 7f1e6f151640 1 Processor -- start 2024-04-26T21:59:43.209 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.209+0000 7f1e6f151640 1 -- start start 2024-04-26T21:59:43.210 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6f151640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e680c7470 0x7f1e68162520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.210 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6f151640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e68156e60 con 0x7f1e68154270 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6f151640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e680abc40 con 0x7f1e680cbcd0 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6f151640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1e680cb1f0 con 0x7f1e680c7470 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6ceed640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f1e68154270 0x7f1e6806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51904/0 (socket says 172.21.15.135:51904) 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6ceed640 1 -- 172.21.15.135:0/2924939463 learned_addr learned my addr 172.21.15.135:0/2924939463 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6d6ee640 1 --2- 172.21.15.135:0/2924939463 >> v2:172.21.15.139:3300/0 conn(0x7f1e680c7470 0x7f1e68162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085402035 0 0) 0x7f1e68156e60 con 0x7f1e68154270 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e58003520 con 0x7f1e68154270 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877860867 0 0) 0x7f1e680abc40 con 0x7f1e680cbcd0 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e68156e60 con 0x7f1e680cbcd0 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6d6ee640 1 -- 172.21.15.135:0/2924939463 >> v1:172.21.15.139:6789/0 conn(0x7f1e680cbcd0 legacy=0x7f1e680c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6d6ee640 1 -- 172.21.15.135:0/2924939463 >> v1:172.21.15.135:6789/0 conn(0x7f1e68154270 legacy=0x7f1e6806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.210+0000 7f1e6d6ee640 1 -- 172.21.15.135:0/2924939463 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680cb4d0 con 0x7f1e680c7470 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6d6ee640 1 --2- 172.21.15.135:0/2924939463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e680c7470 0x7f1e68162520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009d90 tx=0x7f1e6002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a77f8b48af4abaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.211 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60007740 con 0x7f1e680c7470 2024-04-26T21:59:43.212 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e600078e0 con 0x7f1e680c7470 2024-04-26T21:59:43.212 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e677fe640 1 -- 172.21.15.135:0/2924939463 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60007cb0 con 0x7f1e680c7470 2024-04-26T21:59:43.214 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 -- 172.21.15.135:0/2924939463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e680c7470 msgr2=0x7f1e68162520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.214 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2924939463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e680c7470 0x7f1e68162520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009d90 tx=0x7f1e6002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 -- 172.21.15.135:0/2924939463 shutdown_connections 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2924939463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e680c7470 0x7f1e68162520 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 -- 172.21.15.135:0/2924939463 >> 172.21.15.135:0/2924939463 conn(0x7f1e680cc3e0 msgr2=0x7f1e680cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 -- 172.21.15.135:0/2924939463 shutdown_connections 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.211+0000 7f1e6f151640 1 -- 172.21.15.135:0/2924939463 wait complete. 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.212+0000 7f1e6f151640 1 Processor -- start 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.212+0000 7f1e6f151640 1 -- start start 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6f151640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 0x7f1e680aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6f151640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e680cbcd0 0x7f1e681eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6f151640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e68154270 0x7f1e681ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6f151640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e680cc130 con 0x7f1e680c7470 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6f151640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68154650 con 0x7f1e68154270 2024-04-26T21:59:43.215 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6f151640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1e680bfb90 con 0x7f1e680cbcd0 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6ceed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 0x7f1e680aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6ceed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 0x7f1e680aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58184/0 (socket says 172.21.15.135:58184) 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.213+0000 7f1e6ceed640 1 -- 172.21.15.135:0/2306969757 learned_addr learned my addr 172.21.15.135:0/2306969757 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6d6ee640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e68154270 0x7f1e681ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e67fff640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e680cbcd0 0x7f1e681eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6ceed640 1 -- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e680cbcd0 msgr2=0x7f1e681eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6ceed640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e680cbcd0 0x7f1e681eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6ceed640 1 -- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e68154270 msgr2=0x7f1e681ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6ceed640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e68154270 0x7f1e681ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6ceed640 1 -- 172.21.15.135:0/2306969757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e681f0500 con 0x7f1e680c7470 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6d6ee640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e68154270 0x7f1e681ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6ceed640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 0x7f1e680aff50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c410 tx=0x7f1e5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c017070 con 0x7f1e680c7470 2024-04-26T21:59:43.216 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c005ce0 con 0x7f1e680c7470 2024-04-26T21:59:43.217 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0072d0 con 0x7f1e680c7470 2024-04-26T21:59:43.217 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e680b1d50 con 0x7f1e680c7470 2024-04-26T21:59:43.217 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.214+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e681ef4f0 con 0x7f1e680c7470 2024-04-26T21:59:43.217 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.215+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e680b2270 con 0x7f1e680c7470 2024-04-26T21:59:43.218 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.215+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f1e5c01a070 con 0x7f1e680c7470 2024-04-26T21:59:43.218 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.216+0000 7f1e657fa640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1e4c06c860 0x7f1e4c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.218 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.216+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f1e5c089ed0 con 0x7f1e680c7470 2024-04-26T21:59:43.218 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.216+0000 7f1e67fff640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1e4c06c860 0x7f1e4c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.218 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.216+0000 7f1e67fff640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1e4c06c860 0x7f1e4c06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1e6814ddd0 tx=0x7f1e54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.220 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.220+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e5c058bb0 con 0x7f1e680c7470 2024-04-26T21:59:43.399 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.399+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e680506c0 con 0x7f1e680c7470 2024-04-26T21:59:43.400 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.400+0000 7f1e657fa640 1 -- 172.21.15.135:0/2306969757 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f1e5c05ceb0 con 0x7f1e680c7470 2024-04-26T21:59:43.400 INFO:teuthology.misc.health.smithi135.stdout: 2024-04-26T21:59:43.400 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":12,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:41.074883+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:43.402 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1e4c06c860 msgr2=0x7f1e4c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.402 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1e4c06c860 0x7f1e4c06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1e6814ddd0 tx=0x7f1e54009040 comp rx=0 tx=0).stop 2024-04-26T21:59:43.402 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 msgr2=0x7f1e680aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.402 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 0x7f1e680aff50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c410 tx=0x7f1e5c00c8d0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 shutdown_connections 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1e4c06c860 0x7f1e4c06ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e68154270 0x7f1e681ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e680cbcd0 0x7f1e681eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 --2- 172.21.15.135:0/2306969757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e680c7470 0x7f1e680aff50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 >> 172.21.15.135:0/2306969757 conn(0x7f1e680cc3e0 msgr2=0x7f1e681534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 shutdown_connections 2024-04-26T21:59:43.403 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:43.402+0000 7f1e6f151640 1 -- 172.21.15.135:0/2306969757 wait complete. 2024-04-26T21:59:43.412 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:59:50.219 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:50.319 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.319+0000 7fcc8abf7640 1 Processor -- start 2024-04-26T21:59:50.319 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.319+0000 7fcc8abf7640 1 -- start start 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc8abf7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 0x7fcc84160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc8abf7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc841522c0 con 0x7fcc84073b80 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc8abf7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc840ad910 con 0x7fcc84151ee0 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc8abf7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc840736f0 con 0x7fcc840adf90 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc83fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fcc84073b80 0x7fcc84154880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55724/0 (socket says 172.21.15.135:55724) 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc83fff640 1 -- 172.21.15.135:0/2846603167 learned_addr learned my addr 172.21.15.135:0/2846603167 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc89194640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 0x7fcc84160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2882529202 0 0) 0x7fcc841522c0 con 0x7fcc84073b80 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.320+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc6c003520 con 0x7fcc84073b80 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976278364 0 0) 0x7fcc840ad910 con 0x7fcc84151ee0 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc841522c0 con 0x7fcc84151ee0 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2924880048 0 0) 0x7fcc6c003520 con 0x7fcc84073b80 2024-04-26T21:59:50.323 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 msgr2=0x7fcc84160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 0x7fcc84160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 >> v1:172.21.15.139:6789/0 conn(0x7fcc84151ee0 legacy=0x7fcc84071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc840adbf0 con 0x7fcc84073b80 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fcc78001670 con 0x7fcc84073b80 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 >> v1:172.21.15.135:6789/0 conn(0x7fcc84073b80 legacy=0x7fcc84154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc6c003b70 0x7fcc6c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc6c004800 0x7fcc6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 0x7fcc6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc840736f0 con 0x7fcc6c004800 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c00ba70 con 0x7fcc6c003b70 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c00bc40 con 0x7fcc840adf90 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc83fff640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc6c004800 0x7fcc6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc88993640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc6c003b70 0x7fcc6c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.324 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc89194640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 0x7fcc6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc83fff640 1 -- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 msgr2=0x7fcc6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc83fff640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc840adf90 0x7fcc6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc83fff640 1 -- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc6c003b70 msgr2=0x7fcc6c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc83fff640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc6c003b70 0x7fcc6c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.321+0000 7fcc83fff640 1 -- 172.21.15.135:0/2846603167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c003520 con 0x7fcc6c004800 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc83fff640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc6c004800 0x7fcc6c004c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800c1c0 tx=0x7fcc6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5eb47c27bc53bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78007cc0 con 0x7fcc6c004800 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc78005ad0 con 0x7fcc6c004800 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc837fe640 1 -- 172.21.15.135:0/2846603167 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78007070 con 0x7fcc6c004800 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc8abf7640 1 -- 172.21.15.135:0/2846603167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc6c004800 msgr2=0x7fcc6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/2846603167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc6c004800 0x7fcc6c004c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800c1c0 tx=0x7fcc6c005830 comp rx=0 tx=0).stop 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.322+0000 7fcc89194640 1 -- 172.21.15.135:0/2846603167 reap_dead start 2024-04-26T21:59:50.325 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.323+0000 7fcc8abf7640 1 -- 172.21.15.135:0/2846603167 shutdown_connections 2024-04-26T21:59:50.326 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.323+0000 7fcc8abf7640 1 -- 172.21.15.135:0/2846603167 >> 172.21.15.135:0/2846603167 conn(0x7fcc8406e090 msgr2=0x7fcc8406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.326 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.323+0000 7fcc8abf7640 1 -- 172.21.15.135:0/2846603167 shutdown_connections 2024-04-26T21:59:50.326 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.323+0000 7fcc8abf7640 1 -- 172.21.15.135:0/2846603167 wait complete. 2024-04-26T21:59:50.328 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.324+0000 7fcc8abf7640 1 Processor -- start 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.324+0000 7fcc8abf7640 1 -- start start 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc6c004800 0x7fcc841eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc841eeaa0 0x7fcc84156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 0x7fcc84156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84074640 con 0x7fcc840c7310 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84073380 con 0x7fcc841eeaa0 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc84071750 con 0x7fcc6c004800 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 0x7fcc84156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 0x7fcc84156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41020/0 (socket says 172.21.15.135:41020) 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 -- 172.21.15.135:0/925312428 learned_addr learned my addr 172.21.15.135:0/925312428 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc88993640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc6c004800 0x7fcc841eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc83fff640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc841eeaa0 0x7fcc84156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 -- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc6c004800 msgr2=0x7fcc841eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.329 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc6c004800 0x7fcc841eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 -- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc841eeaa0 msgr2=0x7fcc84156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc841eeaa0 0x7fcc84156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 -- 172.21.15.135:0/925312428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc841eef20 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc88993640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc6c004800 0x7fcc841eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc83fff640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc841eeaa0 0x7fcc84156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc89194640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 0x7fcc84156e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc840c2390 tx=0x7fcc7c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00fde0 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7c015430 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc841eab50 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c015820 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.325+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc84152b20 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.326+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc840506c0 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.327+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc7c015bd0 con 0x7fcc840c7310 2024-04-26T21:59:50.330 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.327+0000 7fcc817fa640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcc7006c8f0 0x7fcc7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.331 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.327+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcc7c08c6e0 con 0x7fcc840c7310 2024-04-26T21:59:50.331 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.327+0000 7fcc88993640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcc7006c8f0 0x7fcc7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.331 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.328+0000 7fcc88993640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcc7006c8f0 0x7fcc7006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc74005560 tx=0x7fcc74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.331 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.332+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc7c019030 con 0x7fcc840c7310 2024-04-26T21:59:50.508 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.508+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc84150810 con 0x7fcc840c7310 2024-04-26T21:59:50.509 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.509+0000 7fcc817fa640 1 -- 172.21.15.135:0/925312428 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7fcc7c010040 con 0x7fcc840c7310 2024-04-26T21:59:50.509 INFO:teuthology.misc.health.smithi135.stdout: 2024-04-26T21:59:50.509 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":16,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:47.089554+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:50.511 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcc7006c8f0 msgr2=0x7fcc7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.511 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcc7006c8f0 0x7fcc7006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc74005560 tx=0x7fcc74009040 comp rx=0 tx=0).stop 2024-04-26T21:59:50.511 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 msgr2=0x7fcc84156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.511 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 0x7fcc84156e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc840c2390 tx=0x7fcc7c00f580 comp rx=0 tx=0).stop 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 shutdown_connections 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcc7006c8f0 0x7fcc7006edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc840c7310 0x7fcc84156e20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc841eeaa0 0x7fcc84156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 --2- 172.21.15.135:0/925312428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc6c004800 0x7fcc841eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 >> 172.21.15.135:0/925312428 conn(0x7fcc8406e090 msgr2=0x7fcc8414a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 shutdown_connections 2024-04-26T21:59:50.512 INFO:teuthology.misc.health.smithi135.stderr:2024-04-26T21:59:50.511+0000 7fcc8abf7640 1 -- 172.21.15.135:0/925312428 wait complete. 2024-04-26T21:59:50.521 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T21:59:50.521 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T21:59:50.521 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:59:50.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:59:50.524 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:59:50.524 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:59:50.524 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:59:50.526 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:59:50.526 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:59:50.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:59:50.529 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:59:50.529 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:59:50.529 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:59:50.531 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:59:50.531 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T21:59:50.532 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:50.598 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-26T21:59:50.599 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-26T21:59:50.612 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-04-26T21:59:50.614 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-04-26T21:59:50.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.632+0000 7fbb83a7b640 1 Processor -- start 2024-04-26T21:59:50.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.632+0000 7fbb83a7b640 1 -- start start 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb83a7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb7c0ca5a0 0x7fbb7c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb83a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c15a1a0 0x7fbb7c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb83a7b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7c0c8df0 con 0x7fbb7c154160 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb83a7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbb7c0c9100 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb81817640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbb7c154160 0x7fbb7c0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55760/0 (socket says 172.21.15.135:55760) 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb83a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c154540 con 0x7fbb7c15a1a0 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb81817640 1 -- 172.21.15.135:0/1726636989 learned_addr learned my addr 172.21.15.135:0/1726636989 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb81016640 1 --2- 172.21.15.135:0/1726636989 >> v2:172.21.15.139:3300/0 conn(0x7fbb7c0ca5a0 0x7fbb7c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb82018640 1 --2- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c15a1a0 0x7fbb7c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.633+0000 7fbb80815640 1 -- 172.21.15.135:0/1726636989 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280523210 0 0) 0x7fbb78001110 con 0x7fbb7c154160 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb80815640 1 -- 172.21.15.135:0/1726636989 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb68003520 con 0x7fbb7c154160 2024-04-26T21:59:50.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb80815640 1 -- 172.21.15.135:0/1726636989 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1233648065 0 0) 0x7fbb68003520 con 0x7fbb7c154160 2024-04-26T21:59:50.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb81016640 1 -- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c15a1a0 msgr2=0x7fbb7c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:50.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb81016640 1 --2- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c15a1a0 0x7fbb7c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb81016640 1 -- 172.21.15.135:0/1726636989 >> v1:172.21.15.135:6789/0 conn(0x7fbb7c154160 legacy=0x7fbb7c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb81016640 1 -- 172.21.15.135:0/1726636989 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c15a910 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.634+0000 7fbb81016640 1 --2- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c156bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097a0 tx=0x7fbb6c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4aa91c2c20e4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb80815640 1 -- 172.21.15.135:0/1726636989 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c037070 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb80815640 1 -- 172.21.15.135:0/1726636989 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6c02aba0 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb80815640 1 -- 172.21.15.135:0/1726636989 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78001840 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 -- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0ca5a0 msgr2=0x7fbb7c156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c156bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097a0 tx=0x7fbb6c02a110 comp rx=0 tx=0).stop 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 -- 172.21.15.135:0/1726636989 shutdown_connections 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c15a1a0 0x7fbb7c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/1726636989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c156bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 -- 172.21.15.135:0/1726636989 >> 172.21.15.135:0/1726636989 conn(0x7fbb7c0ae010 msgr2=0x7fbb7c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 -- 172.21.15.135:0/1726636989 shutdown_connections 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.635+0000 7fbb83a7b640 1 -- 172.21.15.135:0/1726636989 wait complete. 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.636+0000 7fbb83a7b640 1 Processor -- start 2024-04-26T21:59:50.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 -- start start 2024-04-26T21:59:50.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c154160 0x7fbb7c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c15a1a0 0x7fbb7c148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c15a5f0 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c15aed0 con 0x7fbb7c15a1a0 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb83a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c0c7510 con 0x7fbb7c154160 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c1572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41054/0 (socket says 172.21.15.135:41054) 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 -- 172.21.15.135:0/3106533944 learned_addr learned my addr 172.21.15.135:0/3106533944 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb82018640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c15a1a0 0x7fbb7c148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81016640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c154160 0x7fbb7c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 -- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c154160 msgr2=0x7fbb7c152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c154160 0x7fbb7c152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 -- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c15a1a0 msgr2=0x7fbb7c148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c15a1a0 0x7fbb7c148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.637+0000 7fbb81817640 1 -- 172.21.15.135:0/3106533944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c143df0 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.638+0000 7fbb81817640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c1572c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c700 tx=0x7fbb7800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.638+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78017070 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.638+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c144060 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.638+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78005ad0 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.638+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78007070 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.638+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c149a30 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.639+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c143b70 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.639+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb7801a070 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.640+0000 7fbb727fc640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbb6006c920 0x7fbb6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.640+0000 7fbb81016640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbb6006c920 0x7fbb6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.640+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbb7808ada0 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.640+0000 7fbb81016640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbb6006c920 0x7fbb6006ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c034000 tx=0x7fbb6c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.644+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb7c143b70 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.820+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb7c076650 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.821+0000 7fbb727fc640 1 -- 172.21.15.135:0/3106533944 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7fbb7c0c8df0 con 0x7fbb7c0ca5a0 2024-04-26T21:59:50.820 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:50.821 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:47.089554+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbb6006c920 msgr2=0x7fbb6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbb6006c920 0x7fbb6006ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c034000 tx=0x7fbb6c02e000 comp rx=0 tx=0).stop 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 msgr2=0x7fbb7c1572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c1572c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c700 tx=0x7fbb7800cbc0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 shutdown_connections 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbb6006c920 0x7fbb6006ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c15a1a0 0x7fbb7c148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbb7c154160 0x7fbb7c152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 --2- 172.21.15.135:0/3106533944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb7c0ca5a0 0x7fbb7c1572c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 >> 172.21.15.135:0/3106533944 conn(0x7fbb7c0ae010 msgr2=0x7fbb7c0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 shutdown_connections 2024-04-26T21:59:50.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.823+0000 7fbb83a7b640 1 -- 172.21.15.135:0/3106533944 wait complete. 2024-04-26T21:59:50.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:59:50.834 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:59:50.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.933+0000 7fe44c178640 1 Processor -- start 2024-04-26T21:59:50.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.933+0000 7fe44c178640 1 -- start start 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe44c178640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe4440af140 0x7fe4440c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe44c178640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe444154e10 con 0x7fe44406d590 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe44c178640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4441550b0 con 0x7fe444070120 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe44c178640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe4441559e0 con 0x7fe4440af140 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe449f14640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe44406d590 0x7fe444158d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55774/0 (socket says 172.21.15.135:55774) 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe449f14640 1 -- 172.21.15.135:0/2718808735 learned_addr learned my addr 172.21.15.135:0/2718808735 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:50.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe44a715640 1 --2- 172.21.15.135:0/2718808735 >> v2:172.21.15.135:3300/0 conn(0x7fe4440af140 0x7fe4440c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe448f12640 1 -- 172.21.15.135:0/2718808735 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325403031 0 0) 0x7fe444154e10 con 0x7fe44406d590 2024-04-26T21:59:50.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.934+0000 7fe448f12640 1 -- 172.21.15.135:0/2718808735 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe430003520 con 0x7fe44406d590 2024-04-26T21:59:50.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe44a715640 1 -- 172.21.15.135:0/2718808735 >> v1:172.21.15.139:6789/0 conn(0x7fe444070120 legacy=0x7fe4440aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:50.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe44a715640 1 -- 172.21.15.135:0/2718808735 >> v1:172.21.15.135:6789/0 conn(0x7fe44406d590 legacy=0x7fe444158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe44a715640 1 -- 172.21.15.135:0/2718808735 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe444155390 con 0x7fe4440af140 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe44a715640 1 --2- 172.21.15.135:0/2718808735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4440af140 0x7fe4440c3be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097a0 tx=0x7fe43c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eece4f689aacb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe448f12640 1 -- 172.21.15.135:0/2718808735 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c037070 con 0x7fe4440af140 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe448f12640 1 -- 172.21.15.135:0/2718808735 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c02ae50 con 0x7fe4440af140 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.935+0000 7fe448f12640 1 -- 172.21.15.135:0/2718808735 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c032660 con 0x7fe4440af140 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 -- 172.21.15.135:0/2718808735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4440af140 msgr2=0x7fe4440c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 --2- 172.21.15.135:0/2718808735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4440af140 0x7fe4440c3be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097a0 tx=0x7fe43c02a140 comp rx=0 tx=0).stop 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 -- 172.21.15.135:0/2718808735 shutdown_connections 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 --2- 172.21.15.135:0/2718808735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4440af140 0x7fe4440c3be0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 -- 172.21.15.135:0/2718808735 >> 172.21.15.135:0/2718808735 conn(0x7fe4440c5ce0 msgr2=0x7fe4440c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 -- 172.21.15.135:0/2718808735 shutdown_connections 2024-04-26T21:59:50.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.936+0000 7fe44c178640 1 -- 172.21.15.135:0/2718808735 wait complete. 2024-04-26T21:59:50.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.937+0000 7fe44c178640 1 Processor -- start 2024-04-26T21:59:50.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.937+0000 7fe44c178640 1 -- start start 2024-04-26T21:59:50.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 0x7fe4440c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe444070120 0x7fe444157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4440af140 0x7fe444150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe44406daf0 con 0x7fe44406d590 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe44406d970 con 0x7fe4440af140 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440c4120 con 0x7fe444070120 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 0x7fe4440c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 0x7fe4440c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41072/0 (socket says 172.21.15.135:41072) 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 -- 172.21.15.135:0/1851701881 learned_addr learned my addr 172.21.15.135:0/1851701881 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449713640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe444070120 0x7fe444157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44a715640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4440af140 0x7fe444150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 -- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe444070120 msgr2=0x7fe444157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe444070120 0x7fe444157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 -- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4440af140 msgr2=0x7fe444150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4440af140 0x7fe444150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:50.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 -- 172.21.15.135:0/1851701881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe444158450 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449713640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe444070120 0x7fe444157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44a715640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4440af140 0x7fe444150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe449f14640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 0x7fe4440c2cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe44000cb70 tx=0x7fe44000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440017070 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44000ad40 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4441586c0 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440012620 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.938+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe444158c80 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.939+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe444068dc0 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.940+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe44001a070 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.940+0000 7fe43a7fc640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe42806c920 0x7fe42806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.940+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe44008b600 con 0x7fe44406d590 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.940+0000 7fe449713640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe42806c920 0x7fe42806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:50.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.941+0000 7fe449713640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe42806c920 0x7fe42806ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4440c9060 tx=0x7fe434009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:50.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:50.945+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe440059d80 con 0x7fe44406d590 2024-04-26T21:59:51.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.119+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4440507a0 con 0x7fe44406d590 2024-04-26T21:59:51.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.119+0000 7fe43a7fc640 1 -- 172.21.15.135:0/1851701881 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fe440059680 con 0x7fe44406d590 2024-04-26T21:59:51.119 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T21:59:51.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe42806c920 msgr2=0x7fe42806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe42806c920 0x7fe42806ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4440c9060 tx=0x7fe434009040 comp rx=0 tx=0).stop 2024-04-26T21:59:51.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 msgr2=0x7fe4440c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 0x7fe4440c2cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe44000cb70 tx=0x7fe44000a430 comp rx=0 tx=0).stop 2024-04-26T21:59:51.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 shutdown_connections 2024-04-26T21:59:51.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe42806c920 0x7fe42806ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4440af140 0x7fe444150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe444070120 0x7fe444157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 --2- 172.21.15.135:0/1851701881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe44406d590 0x7fe4440c2cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 >> 172.21.15.135:0/1851701881 conn(0x7fe4440c5ce0 msgr2=0x7fe444070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 shutdown_connections 2024-04-26T21:59:51.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.121+0000 7fe44c178640 1 -- 172.21.15.135:0/1851701881 wait complete. 2024-04-26T21:59:51.132 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:51.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.228+0000 7f383e139640 1 Processor -- start 2024-04-26T21:59:51.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.228+0000 7f383e139640 1 -- start start 2024-04-26T21:59:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f383e139640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38380aecf0 0x7f38380c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f383e139640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f38380c3d40 0x7f3838074180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f383e139640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3838156f60 con 0x7f38380c4820 2024-04-26T21:59:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f383e139640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3838157240 con 0x7f38380c3d40 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f383e139640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3838157520 con 0x7f38380aecf0 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f38380c3d40 0x7f3838074180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f38380c3d40 0x7f3838074180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41076/0 (socket says 172.21.15.135:41076) 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 -- 172.21.15.135:0/2007044029 learned_addr learned my addr 172.21.15.135:0/2007044029 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f38377fe640 1 --2- 172.21.15.135:0/2007044029 >> v2:172.21.15.139:3300/0 conn(0x7f38380aecf0 0x7f38380c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 -- 172.21.15.135:0/2007044029 >> v2:172.21.15.139:3300/0 conn(0x7f38380aecf0 msgr2=0x7f38380c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 --2- 172.21.15.135:0/2007044029 >> v2:172.21.15.139:3300/0 conn(0x7f38380aecf0 0x7f38380c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 -- 172.21.15.135:0/2007044029 >> v1:172.21.15.139:6789/0 conn(0x7f38380c4820 legacy=0x7f3838073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f3837fff640 1 -- 172.21.15.135:0/2007044029 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3838157800 con 0x7f38380c3d40 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.229+0000 7f38377fe640 1 --2- 172.21.15.135:0/2007044029 >> v2:172.21.15.139:3300/0 conn(0x7f38380aecf0 0x7f38380c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f3837fff640 1 --2- 172.21.15.135:0/2007044029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f3838074180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38380cb340 tx=0x7f383002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bc65f79ca47c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f38367fc640 1 -- 172.21.15.135:0/2007044029 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830036070 con 0x7f38380c3d40 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f38367fc640 1 -- 172.21.15.135:0/2007044029 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3830031600 con 0x7f38380c3d40 2024-04-26T21:59:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f38367fc640 1 -- 172.21.15.135:0/2007044029 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300319d0 con 0x7f38380c3d40 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 -- 172.21.15.135:0/2007044029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 msgr2=0x7f3838074180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 --2- 172.21.15.135:0/2007044029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f3838074180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38380cb340 tx=0x7f383002a660 comp rx=0 tx=0).stop 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 -- 172.21.15.135:0/2007044029 shutdown_connections 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 --2- 172.21.15.135:0/2007044029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f3838074180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 --2- 172.21.15.135:0/2007044029 >> v2:172.21.15.139:3300/0 conn(0x7f38380aecf0 0x7f38380c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 -- 172.21.15.135:0/2007044029 >> 172.21.15.135:0/2007044029 conn(0x7f38380c6ed0 msgr2=0x7f38380c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 -- 172.21.15.135:0/2007044029 shutdown_connections 2024-04-26T21:59:51.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.230+0000 7f383e139640 1 -- 172.21.15.135:0/2007044029 wait complete. 2024-04-26T21:59:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.231+0000 7f383e139640 1 Processor -- start 2024-04-26T21:59:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 -- start start 2024-04-26T21:59:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38380aecf0 0x7f383814e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f38381efd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38380c4820 0x7f38381f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3838156c10 con 0x7f38380c3d40 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f38380746c0 con 0x7f38380aecf0 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f383e139640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f38380af0d0 con 0x7f38380c4820 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f38381efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f38381efd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41082/0 (socket says 172.21.15.135:41082) 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.232+0000 7f3836ffd640 1 -- 172.21.15.135:0/2940535249 learned_addr learned my addr 172.21.15.135:0/2940535249 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f38377fe640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38380aecf0 0x7f383814e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3837fff640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38380c4820 0x7f38381f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3836ffd640 1 -- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38380c4820 msgr2=0x7f38381f02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3836ffd640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38380c4820 0x7f38381f02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3836ffd640 1 -- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38380aecf0 msgr2=0x7f383814e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3836ffd640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38380aecf0 0x7f383814e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3836ffd640 1 -- 172.21.15.135:0/2940535249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383814cf00 con 0x7f38380c3d40 2024-04-26T21:59:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f3836ffd640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f38381efd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b3f0 tx=0x7f382c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c015020 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f382c011830 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381ec900 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c011c20 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.233+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381ecec0 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.234+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38380507a0 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.235+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f382c018020 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.235+0000 7f381ffff640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f382406c8f0 0x7f382406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.235+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f382c08b260 con 0x7f38380c3d40 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.235+0000 7f38377fe640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f382406c8f0 0x7f382406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.235+0000 7f38377fe640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f382406c8f0 0x7f382406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f382800ab60 tx=0x7f3828008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:51.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.239+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f382c01f030 con 0x7f38380c3d40 2024-04-26T21:59:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.449+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3838073b10 con 0x7f38380c3d40 2024-04-26T21:59:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.449+0000 7f381ffff640 1 -- 172.21.15.135:0/2940535249 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f382c059420 con 0x7f38380c3d40 2024-04-26T21:59:51.450 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:51.450 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:59:51.451 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 2 2024-04-26T21:59:51.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f382406c8f0 msgr2=0x7f382406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f382406c8f0 0x7f382406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f382800ab60 tx=0x7f3828008040 comp rx=0 tx=0).stop 2024-04-26T21:59:51.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 msgr2=0x7f38381efd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:51.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f38381efd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b3f0 tx=0x7f382c00b8b0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 shutdown_connections 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f382406c8f0 0x7f382406edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38380c4820 0x7f38381f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38380c3d40 0x7f38381efd90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 --2- 172.21.15.135:0/2940535249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38380aecf0 0x7f383814e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 >> 172.21.15.135:0/2940535249 conn(0x7f38380c6ed0 msgr2=0x7f3838155660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 shutdown_connections 2024-04-26T21:59:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:51.451+0000 7f383e139640 1 -- 172.21.15.135:0/2940535249 wait complete. 2024-04-26T21:59:53.464 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:53.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 Processor -- start 2024-04-26T21:59:53.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 -- start start 2024-04-26T21:59:53.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c154160 0x7fb13c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb13c15a1a0 0x7fb13c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb13c0c8df0 con 0x7fb13c0ca5a0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb13c0c9100 con 0x7fb13c15a1a0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb1418d3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c154540 con 0x7fb13c154160 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.572+0000 7fb13b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb13c15a1a0 0x7fb13c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb13c15a1a0 0x7fb13c0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41088/0 (socket says 172.21.15.135:41088) 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 -- 172.21.15.135:0/700255583 learned_addr learned my addr 172.21.15.135:0/700255583 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13affd640 1 --2- 172.21.15.135:0/700255583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c154160 0x7fb13c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 -- 172.21.15.135:0/700255583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c154160 msgr2=0x7fb13c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 --2- 172.21.15.135:0/700255583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c154160 0x7fb13c156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 -- 172.21.15.135:0/700255583 >> v1:172.21.15.139:6789/0 conn(0x7fb13c0ca5a0 legacy=0x7fb13c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 -- 172.21.15.135:0/700255583 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c15a910 con 0x7fb13c15a1a0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.573+0000 7fb13b7fe640 1 --2- 172.21.15.135:0/700255583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c15a1a0 0x7fb13c0c9590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb134009870 tx=0x7fb13402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b21e7e8517a7732 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb139ffb640 1 -- 172.21.15.135:0/700255583 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134037070 con 0x7fb13c15a1a0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb139ffb640 1 -- 172.21.15.135:0/700255583 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb134032440 con 0x7fb13c15a1a0 2024-04-26T21:59:53.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb139ffb640 1 -- 172.21.15.135:0/700255583 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134032810 con 0x7fb13c15a1a0 2024-04-26T21:59:53.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 -- 172.21.15.135:0/700255583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c15a1a0 msgr2=0x7fb13c0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 --2- 172.21.15.135:0/700255583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c15a1a0 0x7fb13c0c9590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb134009870 tx=0x7fb13402a270 comp rx=0 tx=0).stop 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 -- 172.21.15.135:0/700255583 shutdown_connections 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 --2- 172.21.15.135:0/700255583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c15a1a0 0x7fb13c0c9590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 --2- 172.21.15.135:0/700255583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c154160 0x7fb13c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 -- 172.21.15.135:0/700255583 >> 172.21.15.135:0/700255583 conn(0x7fb13c0ae010 msgr2=0x7fb13c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 -- 172.21.15.135:0/700255583 shutdown_connections 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.574+0000 7fb1418d3640 1 -- 172.21.15.135:0/700255583 wait complete. 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.575+0000 7fb1418d3640 1 Processor -- start 2024-04-26T21:59:53.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 -- start start 2024-04-26T21:59:53.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c0ca5a0 0x7fb13c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 0x7fb13c149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c15a1a0 0x7fb13c1ef900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c15a630 con 0x7fb13c154160 2024-04-26T21:59:53.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c15aed0 con 0x7fb13c0ca5a0 2024-04-26T21:59:53.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb1418d3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c0c7510 con 0x7fb13c15a1a0 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb13a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 0x7fb13c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb13a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 0x7fb13c149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41104/0 (socket says 172.21.15.135:41104) 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.576+0000 7fb13a7fc640 1 -- 172.21.15.135:0/1360558070 learned_addr learned my addr 172.21.15.135:0/1360558070 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13b7fe640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c15a1a0 0x7fb13c1ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13affd640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c0ca5a0 0x7fb13c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13a7fc640 1 -- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c15a1a0 msgr2=0x7fb13c1ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13a7fc640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c15a1a0 0x7fb13c1ef900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13a7fc640 1 -- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c0ca5a0 msgr2=0x7fb13c14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13a7fc640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c0ca5a0 0x7fb13c14b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13a7fc640 1 -- 172.21.15.135:0/1360558070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c157a00 con 0x7fb13c154160 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13affd640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c0ca5a0 0x7fb13c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb13a7fc640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 0x7fb13c149170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb13000c440 tx=0x7fb130001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300052d0 con 0x7fb13c154160 2024-04-26T21:59:53.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb130005470 con 0x7fb13c154160 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c159d60 con 0x7fb13c154160 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130007260 con 0x7fb13c154160 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.577+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c158040 con 0x7fb13c154160 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.578+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13c158240 con 0x7fb13c154160 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.578+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb130007400 con 0x7fb13c154160 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.579+0000 7fb123fff640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb12806c8f0 0x7fb12806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.579+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb13008ac90 con 0x7fb13c154160 2024-04-26T21:59:53.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.579+0000 7fb13affd640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb12806c8f0 0x7fb12806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.579+0000 7fb13affd640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb12806c8f0 0x7fb12806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb12c007520 tx=0x7fb12c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:53.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.583+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb13c158240 con 0x7fb13c154160 2024-04-26T21:59:53.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.794+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb13c076650 con 0x7fb13c154160 2024-04-26T21:59:53.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.795+0000 7fb123fff640 1 -- 172.21.15.135:0/1360558070 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7fb13c0c8df0 con 0x7fb13c154160 2024-04-26T21:59:53.796 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:53.796 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:59:53.796 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 3 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb12806c8f0 msgr2=0x7fb12806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb12806c8f0 0x7fb12806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb12c007520 tx=0x7fb12c00b040 comp rx=0 tx=0).stop 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 msgr2=0x7fb13c149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 0x7fb13c149170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb13000c440 tx=0x7fb130001e70 comp rx=0 tx=0).stop 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 shutdown_connections 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb12806c8f0 0x7fb12806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb13c15a1a0 0x7fb13c1ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb13c154160 0x7fb13c149170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 --2- 172.21.15.135:0/1360558070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb13c0ca5a0 0x7fb13c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 >> 172.21.15.135:0/1360558070 conn(0x7fb13c0ae010 msgr2=0x7fb13c0caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:53.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 shutdown_connections 2024-04-26T21:59:53.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.797+0000 7fb1418d3640 1 -- 172.21.15.135:0/1360558070 wait complete. 2024-04-26T21:59:53.809 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:53.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.904+0000 7fbc3f351640 1 Processor -- start 2024-04-26T21:59:53.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.904+0000 7fbc3f351640 1 -- start start 2024-04-26T21:59:53.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3f351640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbc380c6e20 0x7fbc380c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3f351640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 0x7fbc38156f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3f351640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc3815a530 con 0x7fbc380ca3a0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3f351640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbc38072930 con 0x7fbc380c6e20 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3f351640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3815a810 con 0x7fbc380740e0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3d8ee640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbc380ca3a0 0x7fbc38157460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55788/0 (socket says 172.21.15.135:55788) 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3d8ee640 1 -- 172.21.15.135:0/1299953277 learned_addr learned my addr 172.21.15.135:0/1299953277 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3c8ec640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 0x7fbc38156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 475305040 0 0) 0x7fbc3815a530 con 0x7fbc380ca3a0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc24003520 con 0x7fbc380ca3a0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3d0ed640 1 --2- 172.21.15.135:0/1299953277 >> v2:172.21.15.139:3300/0 conn(0x7fbc380c6e20 0x7fbc380c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2488955908 0 0) 0x7fbc24003520 con 0x7fbc380ca3a0 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 msgr2=0x7fbc38156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:53.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 0x7fbc38156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 >> v2:172.21.15.139:3300/0 conn(0x7fbc380c6e20 msgr2=0x7fbc380c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 --2- 172.21.15.135:0/1299953277 >> v2:172.21.15.139:3300/0 conn(0x7fbc380c6e20 0x7fbc380c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3815aaf0 con 0x7fbc380ca3a0 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fbc300019f0 con 0x7fbc380ca3a0 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 >> v1:172.21.15.135:6789/0 conn(0x7fbc380ca3a0 legacy=0x7fbc38157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc240039b0 0x7fbc24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 0x7fbc24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3815a810 con 0x7fbc24004d40 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2400b9c0 con 0x7fbc240039b0 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2400bb40 con 0x7fbc380740e0 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.905+0000 7fbc3d8ee640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d0ed640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc240039b0 0x7fbc24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3c8ec640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 0x7fbc24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d8ee640 1 -- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 msgr2=0x7fbc24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d8ee640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc380740e0 0x7fbc24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d8ee640 1 -- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc240039b0 msgr2=0x7fbc24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d8ee640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc240039b0 0x7fbc24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d8ee640 1 -- 172.21.15.135:0/1299953277 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc24003520 con 0x7fbc24004d40 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc3d8ee640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc240051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc300013d0 tx=0x7fbc24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9801494c18e37b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:53.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000a4e0 con 0x7fbc24004d40 2024-04-26T21:59:53.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.906+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3002f070 con 0x7fbc24004d40 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc2ffff640 1 -- 172.21.15.135:0/1299953277 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3002bce0 con 0x7fbc24004d40 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3f351640 1 -- 172.21.15.135:0/1299953277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 msgr2=0x7fbc240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3f351640 1 --2- 172.21.15.135:0/1299953277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc240051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc300013d0 tx=0x7fbc24005830 comp rx=0 tx=0).stop 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3d8ee640 1 -- 172.21.15.135:0/1299953277 reap_dead start 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3f351640 1 -- 172.21.15.135:0/1299953277 shutdown_connections 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3f351640 1 -- 172.21.15.135:0/1299953277 >> 172.21.15.135:0/1299953277 conn(0x7fbc380ae0b0 msgr2=0x7fbc380ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3f351640 1 -- 172.21.15.135:0/1299953277 shutdown_connections 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.907+0000 7fbc3f351640 1 -- 172.21.15.135:0/1299953277 wait complete. 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.908+0000 7fbc3f351640 1 Processor -- start 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.908+0000 7fbc3f351640 1 -- start start 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc38153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc38158f80 0x7fbc38158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc38146de0 0x7fbc38147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38154540 con 0x7fbc24004d40 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381543c0 con 0x7fbc38146de0 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3815a0c0 con 0x7fbc38158f80 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc38153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc38153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41112/0 (socket says 172.21.15.135:41112) 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 -- 172.21.15.135:0/3636645567 learned_addr learned my addr 172.21.15.135:0/3636645567 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3c8ec640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc38158f80 0x7fbc38158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d8ee640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc38146de0 0x7fbc38147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 -- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc38158f80 msgr2=0x7fbc38158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc38158f80 0x7fbc38158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 -- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc38146de0 msgr2=0x7fbc38147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc38146de0 0x7fbc38147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 -- 172.21.15.135:0/3636645567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc38149c70 con 0x7fbc24004d40 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3d0ed640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc38153860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc340119c0 tx=0x7fbc34011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34009590 con 0x7fbc24004d40 2024-04-26T21:59:53.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3400faf0 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34014440 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.909+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc38156410 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.910+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc381555b0 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.911+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc380766c0 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.911+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc34012040 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.911+0000 7fbc2dffb640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbc1c06c8f0 0x7fbc1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.912+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbc3408bd20 con 0x7fbc24004d40 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.912+0000 7fbc3c8ec640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbc1c06c8f0 0x7fbc1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.912+0000 7fbc3c8ec640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbc1c06c8f0 0x7fbc1c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc380b2460 tx=0x7fbc28009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:53.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:53.916+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc38149c70 con 0x7fbc24004d40 2024-04-26T21:59:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.095+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc380507a0 con 0x7fbc24004d40 2024-04-26T21:59:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.096+0000 7fbc2dffb640 1 -- 172.21.15.135:0/3636645567 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7fbc34019030 con 0x7fbc24004d40 2024-04-26T21:59:54.096 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:54.097 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:47.089554+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbc1c06c8f0 msgr2=0x7fbc1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbc1c06c8f0 0x7fbc1c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc380b2460 tx=0x7fbc28009190 comp rx=0 tx=0).stop 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 msgr2=0x7fbc38153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc38153860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc340119c0 tx=0x7fbc34011dc0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 shutdown_connections 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbc1c06c8f0 0x7fbc1c06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc38146de0 0x7fbc38147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc38158f80 0x7fbc38158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 --2- 172.21.15.135:0/3636645567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc24004d40 0x7fbc38153860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 >> 172.21.15.135:0/3636645567 conn(0x7fbc380ae0b0 msgr2=0x7fbc380c7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 shutdown_connections 2024-04-26T21:59:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.098+0000 7fbc3f351640 1 -- 172.21.15.135:0/3636645567 wait complete. 2024-04-26T21:59:54.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:59:54.110 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:59:54.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.211+0000 7fc952093640 1 Processor -- start 2024-04-26T21:59:54.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.211+0000 7fc952093640 1 -- start start 2024-04-26T21:59:54.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc952093640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c13ffb0 0x7fc94c0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc952093640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94c0c8a30 con 0x7fc94c15a280 2024-04-26T21:59:54.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc952093640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94c0c8d10 con 0x7fc94c158180 2024-04-26T21:59:54.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc952093640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c072be0 con 0x7fc94c13ffb0 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc94affd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc94c15a280 0x7fc94c072ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55790/0 (socket says 172.21.15.135:55790) 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc94affd640 1 -- 172.21.15.135:0/207110118 learned_addr learned my addr 172.21.15.135:0/207110118 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.212+0000 7fc94b7fe640 1 --2- 172.21.15.135:0/207110118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c13ffb0 0x7fc94c0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94a7fc640 1 -- 172.21.15.135:0/207110118 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447853642 0 0) 0x7fc94c0c8a30 con 0x7fc94c15a280 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94a7fc640 1 -- 172.21.15.135:0/207110118 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc934003520 con 0x7fc94c15a280 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94b7fe640 1 -- 172.21.15.135:0/207110118 >> v1:172.21.15.139:6789/0 conn(0x7fc94c158180 legacy=0x7fc94c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94a7fc640 1 -- 172.21.15.135:0/207110118 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35364159 0 0) 0x7fc94c0c8d10 con 0x7fc94c158180 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94b7fe640 1 -- 172.21.15.135:0/207110118 >> v1:172.21.15.135:6789/0 conn(0x7fc94c15a280 legacy=0x7fc94c072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94b7fe640 1 -- 172.21.15.135:0/207110118 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c0c8ff0 con 0x7fc94c13ffb0 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.213+0000 7fc94b7fe640 1 --2- 172.21.15.135:0/207110118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c13ffb0 0x7fc94c0c82a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0097d0 tx=0x7fc93c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e7516c558b77b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc94a7fc640 1 -- 172.21.15.135:0/207110118 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c037070 con 0x7fc94c13ffb0 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc94a7fc640 1 -- 172.21.15.135:0/207110118 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c032440 con 0x7fc94c13ffb0 2024-04-26T21:59:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc94a7fc640 1 -- 172.21.15.135:0/207110118 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0327f0 con 0x7fc94c13ffb0 2024-04-26T21:59:54.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 -- 172.21.15.135:0/207110118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c13ffb0 msgr2=0x7fc94c0c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 --2- 172.21.15.135:0/207110118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c13ffb0 0x7fc94c0c82a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0097d0 tx=0x7fc93c02a180 comp rx=0 tx=0).stop 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 -- 172.21.15.135:0/207110118 shutdown_connections 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 --2- 172.21.15.135:0/207110118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c13ffb0 0x7fc94c0c82a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 -- 172.21.15.135:0/207110118 >> 172.21.15.135:0/207110118 conn(0x7fc94c06dc10 msgr2=0x7fc94c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 -- 172.21.15.135:0/207110118 shutdown_connections 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.214+0000 7fc952093640 1 -- 172.21.15.135:0/207110118 wait complete. 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.215+0000 7fc952093640 1 Processor -- start 2024-04-26T21:59:54.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 -- start start 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc94c13ffb0 0x7fc94c1e99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c158180 0x7fc94c13dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 0x7fc94c0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c0c87e0 con 0x7fc94c15a280 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c0c9640 con 0x7fc94c13ffb0 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc952093640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c073700 con 0x7fc94c158180 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc94bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 0x7fc94c0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc94bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 0x7fc94c0add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41128/0 (socket says 172.21.15.135:41128) 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc94bfff640 1 -- 172.21.15.135:0/4024816316 learned_addr learned my addr 172.21.15.135:0/4024816316 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc94b7fe640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc94c13ffb0 0x7fc94c1e99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.216+0000 7fc94affd640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c158180 0x7fc94c13dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc94bfff640 1 -- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c158180 msgr2=0x7fc94c13dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc94bfff640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c158180 0x7fc94c13dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc94bfff640 1 -- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc94c13ffb0 msgr2=0x7fc94c1e99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc94bfff640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc94c13ffb0 0x7fc94c1e99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc94bfff640 1 -- 172.21.15.135:0/4024816316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c1e9f10 con 0x7fc94c15a280 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc94bfff640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 0x7fc94c0add50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c760 tx=0x7fc94400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944003da0 con 0x7fc94c15a280 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc944005070 con 0x7fc94c15a280 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c1557b0 con 0x7fc94c15a280 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944005460 con 0x7fc94c15a280 2024-04-26T21:59:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.217+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1ec8b0 con 0x7fc94c15a280 2024-04-26T21:59:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.218+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c076650 con 0x7fc94c15a280 2024-04-26T21:59:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.218+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc944005600 con 0x7fc94c15a280 2024-04-26T21:59:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.219+0000 7fc933fff640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c8f0 0x7fc93806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.219+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc94408b580 con 0x7fc94c15a280 2024-04-26T21:59:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.219+0000 7fc94b7fe640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c8f0 0x7fc93806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.219+0000 7fc94b7fe640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c8f0 0x7fc93806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc93c034000 tx=0x7fc93c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.223+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc94c1e9f10 con 0x7fc94c15a280 2024-04-26T21:59:54.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.398+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc94c0507a0 con 0x7fc94c15a280 2024-04-26T21:59:54.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.399+0000 7fc933fff640 1 -- 172.21.15.135:0/4024816316 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fc944059740 con 0x7fc94c15a280 2024-04-26T21:59:54.400 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.400+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c8f0 msgr2=0x7fc93806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.400+0000 7fc952093640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c8f0 0x7fc93806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc93c034000 tx=0x7fc93c02e000 comp rx=0 tx=0).stop 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.400+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 msgr2=0x7fc94c0add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.400+0000 7fc952093640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 0x7fc94c0add50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c760 tx=0x7fc94400cc20 comp rx=0 tx=0).stop 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.400+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 shutdown_connections 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.400+0000 7fc952093640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c8f0 0x7fc93806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.401+0000 7fc952093640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc94c15a280 0x7fc94c0add50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.401+0000 7fc952093640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc94c158180 0x7fc94c13dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.401+0000 7fc952093640 1 --2- 172.21.15.135:0/4024816316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc94c13ffb0 0x7fc94c1e99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.401+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 >> 172.21.15.135:0/4024816316 conn(0x7fc94c06dc10 msgr2=0x7fc94c13f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.401+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 shutdown_connections 2024-04-26T21:59:54.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.401+0000 7fc952093640 1 -- 172.21.15.135:0/4024816316 wait complete. 2024-04-26T21:59:54.412 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:59:54.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.508+0000 7f7a4e8d1640 1 Processor -- start 2024-04-26T21:59:54.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.508+0000 7f7a4e8d1640 1 -- start start 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a4e8d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4814d980 0x7f7a480c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a4e8d1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7a4806f1a0 0x7f7a48153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a4e8d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7a48154510 0x7f7a4806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a4e8d1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7a480c4d00 con 0x7f7a4806f1a0 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a4e8d1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7a480ae870 con 0x7f7a48154510 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a4e8d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7a481537f0 con 0x7f7a4814d980 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a477fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7a4806f1a0 0x7f7a48153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a477fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7a4806f1a0 0x7f7a48153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41136/0 (socket says 172.21.15.135:41136) 2024-04-26T21:59:54.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.509+0000 7f7a477fe640 1 -- 172.21.15.135:0/3432145250 learned_addr learned my addr 172.21.15.135:0/3432145250 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a47fff640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4814d980 0x7f7a480c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a4ce6e640 1 --2- 172.21.15.135:0/3432145250 >> v2:172.21.15.139:3300/0 conn(0x7f7a48154510 0x7f7a4806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a477fe640 1 -- 172.21.15.135:0/3432145250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4814d980 msgr2=0x7f7a480c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a477fe640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4814d980 0x7f7a480c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a477fe640 1 -- 172.21.15.135:0/3432145250 >> v2:172.21.15.139:3300/0 conn(0x7f7a48154510 msgr2=0x7f7a4806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a477fe640 1 --2- 172.21.15.135:0/3432145250 >> v2:172.21.15.139:3300/0 conn(0x7f7a48154510 0x7f7a4806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a477fe640 1 -- 172.21.15.135:0/3432145250 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a480c5690 con 0x7f7a4806f1a0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a47fff640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4814d980 0x7f7a480c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a477fe640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a4806f1a0 0x7f7a48153fd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a48074f00 tx=0x7f7a3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186eeff8d4de23b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a46ffd640 1 -- 172.21.15.135:0/3432145250 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c037070 con 0x7f7a4806f1a0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a46ffd640 1 -- 172.21.15.135:0/3432145250 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c032440 con 0x7f7a4806f1a0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.510+0000 7f7a46ffd640 1 -- 172.21.15.135:0/3432145250 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c032810 con 0x7f7a4806f1a0 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/3432145250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a4806f1a0 msgr2=0x7f7a48153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a4806f1a0 0x7f7a48153fd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a48074f00 tx=0x7f7a3c02a140 comp rx=0 tx=0).stop 2024-04-26T21:59:54.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/3432145250 shutdown_connections 2024-04-26T21:59:54.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a4806f1a0 0x7f7a48153fd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/3432145250 >> v2:172.21.15.139:3300/0 conn(0x7f7a48154510 0x7f7a4806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/3432145250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4814d980 0x7f7a480c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/3432145250 >> 172.21.15.135:0/3432145250 conn(0x7f7a480c5de0 msgr2=0x7f7a480c61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/3432145250 shutdown_connections 2024-04-26T21:59:54.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.511+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/3432145250 wait complete. 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.512+0000 7f7a4e8d1640 1 Processor -- start 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.512+0000 7f7a4e8d1640 1 -- start start 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4e8d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4806f1a0 0x7f7a481ee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4e8d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a4814d980 0x7f7a481eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4e8d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 0x7f7a48157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4e8d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480ac390 con 0x7f7a48074180 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4e8d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48153ab0 con 0x7f7a4814d980 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4e8d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7a480ac510 con 0x7f7a4806f1a0 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 0x7f7a48157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 0x7f7a48157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41152/0 (socket says 172.21.15.135:41152) 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 -- 172.21.15.135:0/1863861960 learned_addr learned my addr 172.21.15.135:0/1863861960 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a47fff640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4806f1a0 0x7f7a481ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a477fe640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a4814d980 0x7f7a481eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 -- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4806f1a0 msgr2=0x7f7a481ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4806f1a0 0x7f7a481ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 -- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a4814d980 msgr2=0x7f7a481eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a4814d980 0x7f7a481eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a4ce6e640 1 -- 172.21.15.135:0/1863861960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481ea700 con 0x7f7a48074180 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a47fff640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4806f1a0 0x7f7a481ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.513+0000 7f7a477fe640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a4814d980 0x7f7a481eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.514+0000 7f7a4ce6e640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 0x7f7a48157040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000cc30 tx=0x7f7a4000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.514+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000ce40 con 0x7f7a48074180 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.514+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a40003990 con 0x7f7a48074180 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.514+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a48157b80 con 0x7f7a48074180 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.514+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003d40 con 0x7f7a48074180 2024-04-26T21:59:54.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.514+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a48074960 con 0x7f7a48074180 2024-04-26T21:59:54.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.515+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a48152920 con 0x7f7a48074180 2024-04-26T21:59:54.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.515+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a40004000 con 0x7f7a48074180 2024-04-26T21:59:54.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.515+0000 7f7a44ff9640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a3406c8f0 0x7f7a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.516+0000 7f7a47fff640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a3406c8f0 0x7f7a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.516+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7a4008bcc0 con 0x7f7a48074180 2024-04-26T21:59:54.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.516+0000 7f7a47fff640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a3406c8f0 0x7f7a3406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a38006b10 tx=0x7f7a38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.520+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a48152920 con 0x7f7a48074180 2024-04-26T21:59:54.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.697+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7a480c4050 con 0x7f7a48074180 2024-04-26T21:59:54.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.697+0000 7f7a44ff9640 1 -- 172.21.15.135:0/1863861960 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f7a480c4050 con 0x7f7a48074180 2024-04-26T21:59:54.698 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:54.698 INFO:teuthology.orchestra.run.smithi135.stdout:[ 2024-04-26T21:59:54.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-26T21:59:54.699 INFO:teuthology.orchestra.run.smithi135.stdout: "poolnum": 1, 2024-04-26T21:59:54.699 INFO:teuthology.orchestra.run.smithi135.stdout: "poolname": ".mgr" 2024-04-26T21:59:54.699 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-26T21:59:54.699 INFO:teuthology.orchestra.run.smithi135.stdout:] 2024-04-26T21:59:54.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a3406c8f0 msgr2=0x7f7a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a3406c8f0 0x7f7a3406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a38006b10 tx=0x7f7a38008040 comp rx=0 tx=0).stop 2024-04-26T21:59:54.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 msgr2=0x7f7a48157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 0x7f7a48157040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000cc30 tx=0x7f7a4000a5c0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 shutdown_connections 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a3406c8f0 0x7f7a3406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a48074180 0x7f7a48157040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a4814d980 0x7f7a481eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 --2- 172.21.15.135:0/1863861960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a4806f1a0 0x7f7a481ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 >> 172.21.15.135:0/1863861960 conn(0x7f7a480c5de0 msgr2=0x7f7a48151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 shutdown_connections 2024-04-26T21:59:54.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.699+0000 7f7a4e8d1640 1 -- 172.21.15.135:0/1863861960 wait complete. 2024-04-26T21:59:54.711 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'a' 2024-04-26T21:59:54.711 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_metadata --pg_num_min 64 2024-04-26T21:59:54.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.811+0000 7f5836c46640 1 Processor -- start 2024-04-26T21:59:54.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.811+0000 7f5836c46640 1 -- start start 2024-04-26T21:59:54.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f5836c46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f5836c46640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5830157a00 con 0x7f5830073440 2024-04-26T21:59:54.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f5836c46640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58300ca550 con 0x7f5830150cb0 2024-04-26T21:59:54.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f5836c46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f58300ca830 con 0x7f58301508d0 2024-04-26T21:59:54.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5830073440 0x7f5830072930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55792/0 (socket says 172.21.15.135:55792) 2024-04-26T21:59:54.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582ffff640 1 -- 172.21.15.135:0/775644243 learned_addr learned my addr 172.21.15.135:0/775644243 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f58349e2640 1 --2- 172.21.15.135:0/775644243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129338735 0 0) 0x7f5830157a00 con 0x7f5830073440 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5818003520 con 0x7f5830073440 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692496182 0 0) 0x7f58300ca550 con 0x7f5830150cb0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5830157a00 con 0x7f5830150cb0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2367884598 0 0) 0x7f5818003520 con 0x7f5830073440 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f58349e2640 1 -- 172.21.15.135:0/775644243 >> v1:172.21.15.139:6789/0 conn(0x7f5830150cb0 legacy=0x7f58300c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.812+0000 7f58349e2640 1 -- 172.21.15.135:0/775644243 >> v1:172.21.15.135:6789/0 conn(0x7f5830073440 legacy=0x7f5830072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.813+0000 7f58349e2640 1 -- 172.21.15.135:0/775644243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300c6e60 con 0x7f58301508d0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.813+0000 7f58349e2640 1 --2- 172.21.15.135:0/775644243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301574c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5820009800 tx=0x7f582002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511a892185d921f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.813+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820037070 con 0x7f58301508d0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.813+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582002abc0 con 0x7f58301508d0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.813+0000 7f582f7fe640 1 -- 172.21.15.135:0/775644243 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824001670 con 0x7f58301508d0 2024-04-26T21:59:54.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 -- 172.21.15.135:0/775644243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 msgr2=0x7f58301574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 --2- 172.21.15.135:0/775644243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301574c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5820009800 tx=0x7f582002a130 comp rx=0 tx=0).stop 2024-04-26T21:59:54.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 -- 172.21.15.135:0/775644243 shutdown_connections 2024-04-26T21:59:54.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 --2- 172.21.15.135:0/775644243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301574c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 -- 172.21.15.135:0/775644243 >> 172.21.15.135:0/775644243 conn(0x7f58300ae070 msgr2=0x7f58300ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 -- 172.21.15.135:0/775644243 shutdown_connections 2024-04-26T21:59:54.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.814+0000 7f5836c46640 1 -- 172.21.15.135:0/775644243 wait complete. 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.815+0000 7f5836c46640 1 Processor -- start 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.815+0000 7f5836c46640 1 -- start start 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f5836c46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5830073440 0x7f58300c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f5836c46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f5836c46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 0x7f58301efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f5836c46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58300731c0 con 0x7f5830150cb0 2024-04-26T21:59:54.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f5836c46640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5830073040 con 0x7f5830073440 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f5836c46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f58300c9710 con 0x7f58301508d0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 0x7f58301efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 0x7f58301efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41154/0 (socket says 172.21.15.135:41154) 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 -- 172.21.15.135:0/1207450902 learned_addr learned my addr 172.21.15.135:0/1207450902 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f582ffff640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58349e2640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5830073440 0x7f58300c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 -- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 msgr2=0x7f58301ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 -- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5830073440 msgr2=0x7f58300c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5830073440 0x7f58300c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 -- 172.21.15.135:0/1207450902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300c5970 con 0x7f5830150cb0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.816+0000 7f58351e3640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 0x7f58301efd40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f582800c6b0 tx=0x7f582800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.817+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828003b70 con 0x7f5830150cb0 2024-04-26T21:59:54.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.817+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5828003d10 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.817+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58300c5c50 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.817+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828005310 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.817+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5830156f00 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.818+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58300694b0 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.818+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58280054b0 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.818+0000 7f582d7fa640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f581c06c8f0 0x7f581c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.819+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f582808b6b0 con 0x7f5830150cb0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.819+0000 7f58349e2640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f581c06c8f0 0x7f581c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.819+0000 7f58349e2640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f581c06c8f0 0x7f581c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5820034000 tx=0x7f582002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.823+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5828059e40 con 0x7f5830150cb0 2024-04-26T21:59:54.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:54.998+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64} v 0) -- 0x7f5830072ad0 con 0x7f5830150cb0 2024-04-26T21:59:55.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.620+0000 7f582d7fa640 1 -- 172.21.15.135:0/1207450902 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64}]=0 pool 'a_metadata' created v17) ==== 128+0+0 (secure 0 0 0) 0x7f5828059740 con 0x7f5830150cb0 2024-04-26T21:59:55.621 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'a_metadata' created 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f581c06c8f0 msgr2=0x7f581c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f581c06c8f0 0x7f581c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5820034000 tx=0x7f582002e000 comp rx=0 tx=0).stop 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 msgr2=0x7f58301efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 0x7f58301efd40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f582800c6b0 tx=0x7f582800cb70 comp rx=0 tx=0).stop 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 shutdown_connections 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f581c06c8f0 0x7f581c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5830150cb0 0x7f58301efd40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58301508d0 0x7f58301ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 --2- 172.21.15.135:0/1207450902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5830073440 0x7f58300c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 >> 172.21.15.135:0/1207450902 conn(0x7f58300ae070 msgr2=0x7f58300ad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 shutdown_connections 2024-04-26T21:59:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.623+0000 7f5836c46640 1 -- 172.21.15.135:0/1207450902 wait complete. 2024-04-26T21:59:55.639 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:59:55.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.744+0000 7f2c09c52640 1 Processor -- start 2024-04-26T21:59:55.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.744+0000 7f2c09c52640 1 -- start start 2024-04-26T21:59:55.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c09c52640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 0x7f2c0415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c09c52640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2c040ca2f0 0x7f2c0415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c09c52640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c04157840 con 0x7f2c040b1ac0 2024-04-26T21:59:55.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c09c52640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2c040b1ea0 con 0x7f2c040ca2f0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c09c52640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c040739f0 con 0x7f2c040bfa80 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c037fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2c040b1ac0 0x7f2c040c6d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55802/0 (socket says 172.21.15.135:55802) 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c037fe640 1 -- 172.21.15.135:0/1550074442 learned_addr learned my addr 172.21.15.135:0/1550074442 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c02ffd640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 0x7f2c0415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c03fff640 1 --2- 172.21.15.135:0/1550074442 >> v2:172.21.15.139:3300/0 conn(0x7f2c040ca2f0 0x7f2c0415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170437833 0 0) 0x7f2c04157840 con 0x7f2c040b1ac0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bec003520 con 0x7f2c040b1ac0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3058661673 0 0) 0x7f2bec003520 con 0x7f2c040b1ac0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 msgr2=0x7f2c0415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 0x7f2c0415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 >> v2:172.21.15.139:3300/0 conn(0x7f2c040ca2f0 msgr2=0x7f2c0415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 --2- 172.21.15.135:0/1550074442 >> v2:172.21.15.139:3300/0 conn(0x7f2c040ca2f0 0x7f2c0415a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0406db60 con 0x7f2c040b1ac0 2024-04-26T21:59:55.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2bf4001670 con 0x7f2c040b1ac0 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 >> v1:172.21.15.135:6789/0 conn(0x7f2c040b1ac0 legacy=0x7f2c040c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2bec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bec004800 0x7f2bec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 0x7f2bec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040b1ea0 con 0x7f2bec0039b0 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec00b9c0 con 0x7f2bec004800 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2bec00bb70 con 0x7f2c040bfa80 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.745+0000 7f2c037fe640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2bec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c02ffd640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bec004800 0x7f2bec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c03fff640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 0x7f2bec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c037fe640 1 -- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 msgr2=0x7f2bec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c037fe640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040bfa80 0x7f2bec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c037fe640 1 -- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bec004800 msgr2=0x7f2bec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c037fe640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bec004800 0x7f2bec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c037fe640 1 -- 172.21.15.135:0/1550074442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec003520 con 0x7f2bec0039b0 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.746+0000 7f2c037fe640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2bec003dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4007fd0 tx=0x7f2bec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7874ec10a69ab3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4007d50 con 0x7f2bec0039b0 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf4005ad0 con 0x7f2bec0039b0 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c027fc640 1 -- 172.21.15.135:0/1550074442 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4007070 con 0x7f2bec0039b0 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c09c52640 1 -- 172.21.15.135:0/1550074442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 msgr2=0x7f2bec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c09c52640 1 --2- 172.21.15.135:0/1550074442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2bec003dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4007fd0 tx=0x7f2bec005830 comp rx=0 tx=0).stop 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c03fff640 1 -- 172.21.15.135:0/1550074442 reap_dead start 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c09c52640 1 -- 172.21.15.135:0/1550074442 shutdown_connections 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c09c52640 1 -- 172.21.15.135:0/1550074442 >> 172.21.15.135:0/1550074442 conn(0x7f2c040cc7e0 msgr2=0x7f2c040ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c09c52640 1 -- 172.21.15.135:0/1550074442 shutdown_connections 2024-04-26T21:59:55.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.747+0000 7f2c09c52640 1 -- 172.21.15.135:0/1550074442 wait complete. 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.748+0000 7f2c09c52640 1 Processor -- start 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.748+0000 7f2c09c52640 1 -- start start 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c09c52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2c041ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c09c52640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040c2bd0 0x7f2c041506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c09c52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04150c10 0x7f2c040c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c09c52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0406e120 con 0x7f2bec0039b0 2024-04-26T21:59:55.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c09c52640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0406d910 con 0x7f2c04150c10 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c09c52640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c041574a0 con 0x7f2c040c2bd0 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2c041ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2c041ed2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41172/0 (socket says 172.21.15.135:41172) 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 -- 172.21.15.135:0/3570160858 learned_addr learned my addr 172.21.15.135:0/3570160858 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c02ffd640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040c2bd0 0x7f2c041506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c03fff640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04150c10 0x7f2c040c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 -- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040c2bd0 msgr2=0x7f2c041506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040c2bd0 0x7f2c041506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 -- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04150c10 msgr2=0x7f2c040c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04150c10 0x7f2c040c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c037fe640 1 -- 172.21.15.135:0/3570160858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c041ed7e0 con 0x7f2bec0039b0 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c02ffd640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040c2bd0 0x7f2c041506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:55.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.749+0000 7f2c03fff640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04150c10 0x7f2c040c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.750+0000 7f2c037fe640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2c041ed2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c2d0 tx=0x7f2c04156a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.750+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4004bf0 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.750+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf4043070 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.750+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c040c5cf0 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.750+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4007070 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.750+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c040c62b0 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.751+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c04076650 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.751+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf4007210 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.752+0000 7f2bebfff640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.752+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5222+0+0 (secure 0 0 0) 0x7f2bf40ac120 con 0x7f2bec0039b0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.752+0000 7f2c02ffd640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.752+0000 7f2c02ffd640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c040cbfb0 tx=0x7f2bf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.756+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2bf407a7c0 con 0x7f2bec0039b0 2024-04-26T21:59:55.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:55.934+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2c04073640 con 0x7f2bec0039b0 2024-04-26T21:59:56.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.627+0000 7f2bebfff640 1 -- 172.21.15.135:0/3570160858 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'a_data' created v18) ==== 160+0+0 (secure 0 0 0) 0x7f2bf407a0c0 con 0x7f2bec0039b0 2024-04-26T21:59:56.628 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'a_data' created 2024-04-26T21:59:56.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 msgr2=0x7f2bf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c040cbfb0 tx=0x7f2bf8008040 comp rx=0 tx=0).stop 2024-04-26T21:59:56.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 msgr2=0x7f2c041ed2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2c041ed2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c2d0 tx=0x7f2c04156a70 comp rx=0 tx=0).stop 2024-04-26T21:59:56.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 shutdown_connections 2024-04-26T21:59:56.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c040cbfb0 tx=0x7f2bf8008040 comp rx=0 tx=0).stop 2024-04-26T21:59:56.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04150c10 0x7f2c040c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c040c2bd0 0x7f2c041506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 --2- 172.21.15.135:0/3570160858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bec0039b0 0x7f2c041ed2a0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c2d0 tx=0x7f2c04156a70 comp rx=0 tx=0).stop 2024-04-26T21:59:56.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.631+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 >> 172.21.15.135:0/3570160858 conn(0x7f2c040cc7e0 msgr2=0x7f2c040cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:56.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.635+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 shutdown_connections 2024-04-26T21:59:56.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.637+0000 7f2c09c52640 1 -- 172.21.15.135:0/3570160858 wait complete. 2024-04-26T21:59:56.658 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new a a_metadata a_data 2024-04-26T21:59:56.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.760+0000 7f3e3c989640 1 Processor -- start 2024-04-26T21:59:56.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e3c989640 1 -- start start 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e3c989640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3e3813ddd0 0x7f3e38154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e3c989640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3e38071cc0 0x7f3e38073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e3c989640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e380707e0 con 0x7f3e38073c40 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e3c989640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3e38155450 con 0x7f3e38071cc0 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e3c989640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3e38155730 con 0x7f3e3813ddd0 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.761+0000 7f3e2ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3e38071cc0 0x7f3e38073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3e38071cc0 0x7f3e38073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41182/0 (socket says 172.21.15.135:41182) 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 -- 172.21.15.135:0/2629154014 learned_addr learned my addr 172.21.15.135:0/2629154014 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:56.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e3659c640 1 --2- 172.21.15.135:0/2629154014 >> v2:172.21.15.139:3300/0 conn(0x7f3e3813ddd0 0x7f3e38154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 -- 172.21.15.135:0/2629154014 >> v2:172.21.15.139:3300/0 conn(0x7f3e3813ddd0 msgr2=0x7f3e38154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 --2- 172.21.15.135:0/2629154014 >> v2:172.21.15.139:3300/0 conn(0x7f3e3813ddd0 0x7f3e38154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 -- 172.21.15.135:0/2629154014 >> v1:172.21.15.139:6789/0 conn(0x7f3e38073c40 legacy=0x7f3e38070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 -- 172.21.15.135:0/2629154014 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e38155a10 con 0x7f3e38071cc0 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.762+0000 7f3e2ffff640 1 --2- 172.21.15.135:0/2629154014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e38073700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e38075020 tx=0x7f3e2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5861263f42e3a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e35d9b640 1 -- 172.21.15.135:0/2629154014 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20037070 con 0x7f3e38071cc0 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e35d9b640 1 -- 172.21.15.135:0/2629154014 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e20032440 con 0x7f3e38071cc0 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e35d9b640 1 -- 172.21.15.135:0/2629154014 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20032810 con 0x7f3e38071cc0 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 -- 172.21.15.135:0/2629154014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 msgr2=0x7f3e38073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2629154014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e38073700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e38075020 tx=0x7f3e2002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 -- 172.21.15.135:0/2629154014 shutdown_connections 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2629154014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e38073700 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2629154014 >> v2:172.21.15.139:3300/0 conn(0x7f3e3813ddd0 0x7f3e38154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 -- 172.21.15.135:0/2629154014 >> 172.21.15.135:0/2629154014 conn(0x7f3e380ca2f0 msgr2=0x7f3e380ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:56.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 -- 172.21.15.135:0/2629154014 shutdown_connections 2024-04-26T21:59:56.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.763+0000 7f3e3c989640 1 -- 172.21.15.135:0/2629154014 wait complete. 2024-04-26T21:59:56.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.764+0000 7f3e3c989640 1 Processor -- start 2024-04-26T21:59:56.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.764+0000 7f3e3c989640 1 -- start start 2024-04-26T21:59:56.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3c989640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e380c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3c989640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38073c40 0x7f3e381ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3c989640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e3813ddd0 0x7f3e381ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3c989640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381551c0 con 0x7f3e38071cc0 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3c989640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38155040 con 0x7f3e3813ddd0 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3c989640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3e380ab470 con 0x7f3e38073c40 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e380c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e380c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41188/0 (socket says 172.21.15.135:41188) 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3659c640 1 -- 172.21.15.135:0/2872526225 learned_addr learned my addr 172.21.15.135:0/2872526225 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e36d9d640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e3813ddd0 0x7f3e381ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e2ffff640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38073c40 0x7f3e381ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3659c640 1 -- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38073c40 msgr2=0x7f3e381ecd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3659c640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38073c40 0x7f3e381ecd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.765+0000 7f3e3659c640 1 -- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e3813ddd0 msgr2=0x7f3e381ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e3659c640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e3813ddd0 0x7f3e381ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e3659c640 1 -- 172.21.15.135:0/2872526225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e381ee540 con 0x7f3e38071cc0 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e36d9d640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e3813ddd0 0x7f3e381ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:56.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e3659c640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e380c8760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800d830 tx=0x7f3e2800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800b6c0 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e28019070 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e381de2e0 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800ba80 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.766+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e380ab7d0 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.767+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e381eb750 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.767+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e2801e460 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.768+0000 7f3e2effd640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3e1806c8f0 0x7f3e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.768+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5599+0+0 (secure 0 0 0) 0x7f3e2808c130 con 0x7f3e38071cc0 2024-04-26T21:59:56.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.768+0000 7f3e2ffff640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3e1806c8f0 0x7f3e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.768+0000 7f3e2ffff640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3e1806c8f0 0x7f3e1806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e20034000 tx=0x7f3e2002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:56.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.772+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e2805a660 con 0x7f3e38071cc0 2024-04-26T21:59:56.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:56.983+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"} v 0) -- 0x7f3e38076650 con 0x7f3e38071cc0 2024-04-26T21:59:57.526 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:59:57.528 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:59:57.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.641+0000 7f3e2effd640 1 -- 172.21.15.135:0/2872526225 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 157+0+0 (secure 0 0 0) 0x7f3e28059f60 con 0x7f3e38071cc0 2024-04-26T21:59:57.647 INFO:teuthology.orchestra.run.smithi135.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:59:57.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.651+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3e1806c8f0 msgr2=0x7f3e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.651+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3e1806c8f0 0x7f3e1806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e20034000 tx=0x7f3e2002e000 comp rx=0 tx=0).stop 2024-04-26T21:59:57.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.651+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 msgr2=0x7f3e380c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.651+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e380c8760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800d830 tx=0x7f3e2800dcf0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.652+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 shutdown_connections 2024-04-26T21:59:57.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.652+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3e1806c8f0 0x7f3e1806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.652+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e3813ddd0 0x7f3e381ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.652+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38073c40 0x7f3e381ecd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.652+0000 7f3e3c989640 1 --2- 172.21.15.135:0/2872526225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e38071cc0 0x7f3e380c8760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.652+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 >> 172.21.15.135:0/2872526225 conn(0x7f3e380ca2f0 msgr2=0x7f3e38139a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:57.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.654+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 shutdown_connections 2024-04-26T21:59:57.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.654+0000 7f3e3c989640 1 -- 172.21.15.135:0/2872526225 wait complete. 2024-04-26T21:59:57.671 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:57.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.769+0000 7fad5e789640 1 Processor -- start 2024-04-26T21:59:57.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.769+0000 7fad5e789640 1 -- start start 2024-04-26T21:59:57.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad5e789640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad5813ddd0 0x7fad58154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad5e789640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad58070f70 0x7fad58071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad5e789640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad580707e0 con 0x7fad58071cc0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad5e789640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fad58155450 con 0x7fad5813ddd0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad5e789640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad58155730 con 0x7fad58070f70 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad5813ddd0 0x7fad58154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad5813ddd0 0x7fad58154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41194/0 (socket says 172.21.15.135:41194) 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad577fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fad58071cc0 0x7fad58073700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55818/0 (socket says 172.21.15.135:55818) 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 -- 172.21.15.135:0/3105058876 learned_addr learned my addr 172.21.15.135:0/3105058876 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad5cd26640 1 --2- 172.21.15.135:0/3105058876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad58070f70 0x7fad58071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 -- 172.21.15.135:0/3105058876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad58070f70 msgr2=0x7fad58071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 --2- 172.21.15.135:0/3105058876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad58070f70 0x7fad58071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 -- 172.21.15.135:0/3105058876 >> v1:172.21.15.135:6789/0 conn(0x7fad58071cc0 legacy=0x7fad58073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad56ffd640 1 -- 172.21.15.135:0/3105058876 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2460518970 0 0) 0x7fad580707e0 con 0x7fad58071cc0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.770+0000 7fad57fff640 1 -- 172.21.15.135:0/3105058876 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad58155a10 con 0x7fad5813ddd0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad57fff640 1 --2- 172.21.15.135:0/3105058876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad5813ddd0 0x7fad58154b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad48009870 tx=0x7fad4802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9687f80c173e3d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:57.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad56ffd640 1 -- 172.21.15.135:0/3105058876 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48037070 con 0x7fad5813ddd0 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad56ffd640 1 -- 172.21.15.135:0/3105058876 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48032440 con 0x7fad5813ddd0 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad56ffd640 1 -- 172.21.15.135:0/3105058876 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48032810 con 0x7fad5813ddd0 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad5e789640 1 -- 172.21.15.135:0/3105058876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad5813ddd0 msgr2=0x7fad58154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad5e789640 1 --2- 172.21.15.135:0/3105058876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad5813ddd0 0x7fad58154b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad48009870 tx=0x7fad4802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad5e789640 1 -- 172.21.15.135:0/3105058876 shutdown_connections 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad5e789640 1 --2- 172.21.15.135:0/3105058876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad58070f70 0x7fad58071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.771+0000 7fad5e789640 1 --2- 172.21.15.135:0/3105058876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad5813ddd0 0x7fad58154b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.772+0000 7fad5e789640 1 -- 172.21.15.135:0/3105058876 >> 172.21.15.135:0/3105058876 conn(0x7fad580ca2f0 msgr2=0x7fad580ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.772+0000 7fad5e789640 1 -- 172.21.15.135:0/3105058876 shutdown_connections 2024-04-26T21:59:57.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.772+0000 7fad5e789640 1 -- 172.21.15.135:0/3105058876 wait complete. 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 Processor -- start 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 -- start start 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58070f70 0x7fad58157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 0x7fad580bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad5813ddd0 0x7fad58153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad5813e330 con 0x7fad58071cc0 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad5813e1b0 con 0x7fad58070f70 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad5e789640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad58155040 con 0x7fad5813ddd0 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.773+0000 7fad577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 0x7fad580bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 0x7fad580bfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41206/0 (socket says 172.21.15.135:41206) 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 -- 172.21.15.135:0/2411343206 learned_addr learned my addr 172.21.15.135:0/2411343206 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad5cd26640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad5813ddd0 0x7fad58153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad57fff640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58070f70 0x7fad58157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 -- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad5813ddd0 msgr2=0x7fad58153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad5813ddd0 0x7fad58153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 -- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58070f70 msgr2=0x7fad58157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58070f70 0x7fad58157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 -- 172.21.15.135:0/2411343206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580c0420 con 0x7fad58071cc0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad57fff640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58070f70 0x7fad58157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad577fe640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 0x7fad580bfee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c4d0 tx=0x7fad4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40018070 con 0x7fad58071cc0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad40007500 con 0x7fad58071cc0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad58156010 con 0x7fad58071cc0 2024-04-26T21:59:57.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40004a20 con 0x7fad58071cc0 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.774+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad581565d0 con 0x7fad58071cc0 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.775+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad580507a0 con 0x7fad58071cc0 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.775+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad4001b070 con 0x7fad58071cc0 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.776+0000 7fad54ff9640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad3806c8f0 0x7fad3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.776+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5689+0+0 (secure 0 0 0) 0x7fad4008ad30 con 0x7fad58071cc0 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.776+0000 7fad57fff640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad3806c8f0 0x7fad3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.776+0000 7fad57fff640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad3806c8f0 0x7fad3806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad48034000 tx=0x7fad4802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:57.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.780+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad580c0420 con 0x7fad58071cc0 2024-04-26T21:59:57.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.956+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad58074110 con 0x7fad58071cc0 2024-04-26T21:59:57.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.957+0000 7fad54ff9640 1 -- 172.21.15.135:0/2411343206 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16646 (secure 0 0 0) 0x7fad40058bb0 con 0x7fad58071cc0 2024-04-26T21:59:57.959 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:57.959 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":19,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:56.985382+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:57.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad3806c8f0 msgr2=0x7fad3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad3806c8f0 0x7fad3806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad48034000 tx=0x7fad4802e000 comp rx=0 tx=0).stop 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 msgr2=0x7fad580bfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 0x7fad580bfee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c4d0 tx=0x7fad4000c990 comp rx=0 tx=0).stop 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 shutdown_connections 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad3806c8f0 0x7fad3806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad5813ddd0 0x7fad58153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad58071cc0 0x7fad580bfee0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.959+0000 7fad5e789640 1 --2- 172.21.15.135:0/2411343206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58070f70 0x7fad58157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.960+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 >> 172.21.15.135:0/2411343206 conn(0x7fad580ca2f0 msgr2=0x7fad5813d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.960+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 shutdown_connections 2024-04-26T21:59:57.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:57.960+0000 7fad5e789640 1 -- 172.21.15.135:0/2411343206 wait complete. 2024-04-26T21:59:57.972 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:58.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.067+0000 7faa1a3f0640 1 Processor -- start 2024-04-26T21:59:58.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.067+0000 7faa1a3f0640 1 -- start start 2024-04-26T21:59:58.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa1a3f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa14151e10 0x7faa140ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa1a3f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faa14074620 0x7faa141582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa1a3f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa14154920 con 0x7faa140aca30 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa1a3f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7faa14074e90 con 0x7faa14074620 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa1a3f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faa140c6b60 con 0x7faa14151e10 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faa14074620 0x7faa141582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faa14074620 0x7faa141582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41218/0 (socket says 172.21.15.135:41218) 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 -- 172.21.15.135:0/1935255432 learned_addr learned my addr 172.21.15.135:0/1935255432 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa13fff640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa14151e10 0x7faa140ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 -- 172.21.15.135:0/1935255432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa14151e10 msgr2=0x7faa140ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa14151e10 0x7faa140ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 -- 172.21.15.135:0/1935255432 >> v1:172.21.15.139:6789/0 conn(0x7faa140aca30 legacy=0x7faa14158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa137fe640 1 -- 172.21.15.135:0/1935255432 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa140c62c0 con 0x7faa14074620 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.068+0000 7faa13fff640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa14151e10 0x7faa140ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa137fe640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14074620 0x7faa141582f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009870 tx=0x7fa9fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac5b3f848404793 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa12ffd640 1 -- 172.21.15.135:0/1935255432 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc037070 con 0x7faa14074620 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa12ffd640 1 -- 172.21.15.135:0/1935255432 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9fc032440 con 0x7faa14074620 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa12ffd640 1 -- 172.21.15.135:0/1935255432 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc032810 con 0x7faa14074620 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1935255432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14074620 msgr2=0x7faa141582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14074620 0x7faa141582f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009870 tx=0x7fa9fc02a270 comp rx=0 tx=0).stop 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1935255432 shutdown_connections 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14074620 0x7faa141582f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1935255432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa14151e10 0x7faa140ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.069+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1935255432 >> 172.21.15.135:0/1935255432 conn(0x7faa140ca2f0 msgr2=0x7faa140ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.070+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1935255432 shutdown_connections 2024-04-26T21:59:58.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.070+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1935255432 wait complete. 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 Processor -- start 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 -- start start 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14074620 0x7faa141de900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa140aca30 0x7faa141dee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 0x7faa1414cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa140c5f50 con 0x7faa14151e10 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa140c6880 con 0x7faa14074620 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1a3f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faa14071f60 con 0x7faa140aca30 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.071+0000 7faa1898d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 0x7faa1414cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 0x7faa1414cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41230/0 (socket says 172.21.15.135:41230) 2024-04-26T21:59:58.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 -- 172.21.15.135:0/1963636879 learned_addr learned my addr 172.21.15.135:0/1963636879 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa137fe640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa140aca30 0x7faa141dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa13fff640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14074620 0x7faa141de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 -- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa140aca30 msgr2=0x7faa141dee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa140aca30 0x7faa141dee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 -- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14074620 msgr2=0x7faa141de900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14074620 0x7faa141de900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 -- 172.21.15.135:0/1963636879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa141e17d0 con 0x7faa14151e10 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1898d640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 0x7faa1414cae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c730 tx=0x7faa0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c003b90 con 0x7faa14151e10 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0c003d30 con 0x7faa14151e10 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00aa90 con 0x7faa14151e10 2024-04-26T21:59:58.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa141de120 con 0x7faa14151e10 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.072+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141e12b0 con 0x7faa14151e10 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.073+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa14155200 con 0x7faa14151e10 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.074+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa0c00ac30 con 0x7faa14151e10 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.074+0000 7faa10ff9640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa9f406c8f0 0x7fa9f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.074+0000 7faa13fff640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa9f406c8f0 0x7fa9f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.074+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5689+0+0 (secure 0 0 0) 0x7faa0c08b880 con 0x7faa14151e10 2024-04-26T21:59:58.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.075+0000 7faa13fff640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa9f406c8f0 0x7fa9f406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa04006fd0 tx=0x7faa04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.079+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa14155200 con 0x7faa14151e10 2024-04-26T21:59:58.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.254+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa141519c0 con 0x7faa14151e10 2024-04-26T21:59:58.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.256+0000 7faa10ff9640 1 -- 172.21.15.135:0/1963636879 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16646 (secure 0 0 0) 0x7faa141519c0 con 0x7faa14151e10 2024-04-26T21:59:58.257 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T21:59:58.257 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":19,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:56.985382+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa9f406c8f0 msgr2=0x7fa9f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa9f406c8f0 0x7fa9f406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa04006fd0 tx=0x7faa04008040 comp rx=0 tx=0).stop 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 msgr2=0x7faa1414cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 0x7faa1414cae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c730 tx=0x7faa0c00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 shutdown_connections 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa9f406c8f0 0x7fa9f406edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14151e10 0x7faa1414cae0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa140aca30 0x7faa141dee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 --2- 172.21.15.135:0/1963636879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14074620 0x7faa141de900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 >> 172.21.15.135:0/1963636879 conn(0x7faa140ca2f0 msgr2=0x7faa141539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 shutdown_connections 2024-04-26T21:59:58.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.258+0000 7faa1a3f0640 1 -- 172.21.15.135:0/1963636879 wait complete. 2024-04-26T21:59:58.270 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a standby_count_wanted 0 2024-04-26T21:59:58.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe6727bf640 1 Processor -- start 2024-04-26T21:59:58.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe6727bf640 1 -- start start 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe6727bf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 0x7fe66c0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe6727bf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0c8a30 con 0x7fe66c158180 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe6727bf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0c8d10 con 0x7fe66c15a280 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe6727bf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c072be0 con 0x7fe66c13ffb0 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe670d5c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe66c158180 0x7fe66c161610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55824/0 (socket says 172.21.15.135:55824) 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.369+0000 7fe670d5c640 1 -- 172.21.15.135:0/1648842617 learned_addr learned my addr 172.21.15.135:0/1648842617 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66bfff640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 0x7fe66c0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520560639 0 0) 0x7fe66c0c8a30 con 0x7fe66c158180 2024-04-26T21:59:58.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe654003520 con 0x7fe66c158180 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3830628016 0 0) 0x7fe654003520 con 0x7fe66c158180 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 msgr2=0x7fe66c0c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 0x7fe66c0c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 >> v1:172.21.15.139:6789/0 conn(0x7fe66c15a280 legacy=0x7fe66c072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0c8ff0 con 0x7fe66c158180 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fe6640019f0 con 0x7fe66c158180 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 >> v1:172.21.15.135:6789/0 conn(0x7fe66c158180 legacy=0x7fe66c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 0x7fe6540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe654003f60 0x7fe654004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe65400bb50 con 0x7fe654004d40 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe65400bcd0 con 0x7fe654003f60 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe65400bed0 con 0x7fe66c13ffb0 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66bfff640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 0x7fe6540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe66b7fe640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe654003f60 0x7fe654004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe670d5c640 1 -- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 msgr2=0x7fe6540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c13ffb0 0x7fe6540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe670d5c640 1 -- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe654003f60 msgr2=0x7fe654004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe654003f60 0x7fe654004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.370+0000 7fe670d5c640 1 -- 172.21.15.135:0/1648842617 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe654003520 con 0x7fe654004d40 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.371+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a860 tx=0x7fe654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc00f4a57584ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.371+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400a480 con 0x7fe654004d40 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.371+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66402f070 con 0x7fe654004d40 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.371+0000 7fe66affd640 1 -- 172.21.15.135:0/1648842617 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66402bce0 con 0x7fe654004d40 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe6727bf640 1 -- 172.21.15.135:0/1648842617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe654004d40 msgr2=0x7fe6540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1648842617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a860 tx=0x7fe654005830 comp rx=0 tx=0).stop 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe670d5c640 1 -- 172.21.15.135:0/1648842617 reap_dead start 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe6727bf640 1 -- 172.21.15.135:0/1648842617 shutdown_connections 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe6727bf640 1 -- 172.21.15.135:0/1648842617 >> 172.21.15.135:0/1648842617 conn(0x7fe66c06dc10 msgr2=0x7fe66c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe6727bf640 1 -- 172.21.15.135:0/1648842617 shutdown_connections 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.372+0000 7fe6727bf640 1 -- 172.21.15.135:0/1648842617 wait complete. 2024-04-26T21:59:58.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.373+0000 7fe6727bf640 1 Processor -- start 2024-04-26T21:59:58.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.373+0000 7fe6727bf640 1 -- start start 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c155b50 0x7fe66c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c135900 0x7fe66c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 0x7fe66c1ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c073700 con 0x7fe66c1ec880 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0c9640 con 0x7fe66c155b50 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c073880 con 0x7fe66c135900 2024-04-26T21:59:58.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 0x7fe66c1ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 0x7fe66c1ecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41238/0 (socket says 172.21.15.135:41238) 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 -- 172.21.15.135:0/1383855162 learned_addr learned my addr 172.21.15.135:0/1383855162 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe66b7fe640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c135900 0x7fe66c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe66bfff640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c155b50 0x7fe66c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 -- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c135900 msgr2=0x7fe66c154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c135900 0x7fe66c154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 -- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c155b50 msgr2=0x7fe66c153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c155b50 0x7fe66c153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 -- 172.21.15.135:0/1383855162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c157a60 con 0x7fe66c1ec880 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe66bfff640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c155b50 0x7fe66c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe670d5c640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 0x7fe66c1ecd30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a980 tx=0x7fe66402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400a480 con 0x7fe66c1ec880 2024-04-26T21:59:58.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664034470 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c157780 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664034820 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.374+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1570a0 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.375+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c1572a0 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.376+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6640349c0 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.376+0000 7fe668ff9640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe65806c920 0x7fe65806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.376+0000 7fe66bfff640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe65806c920 0x7fe65806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.376+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5689+0+0 (secure 0 0 0) 0x7fe6640aba90 con 0x7fe66c1ec880 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.377+0000 7fe66bfff640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe65806c920 0x7fe65806ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe660009bc0 tx=0x7fe660008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.380+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe66c1572a0 con 0x7fe66c1ec880 2024-04-26T21:59:58.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.592+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe66c076650 con 0x7fe66c1ec880 2024-04-26T21:59:58.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.755+0000 7fe668ff9640 1 -- 172.21.15.135:0/1383855162 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 113+0+0 (secure 0 0 0) 0x7fe664046060 con 0x7fe66c1ec880 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.757+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe65806c920 msgr2=0x7fe65806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe65806c920 0x7fe65806ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe660009bc0 tx=0x7fe660008040 comp rx=0 tx=0).stop 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 msgr2=0x7fe66c1ecd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 0x7fe66c1ecd30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a980 tx=0x7fe66402d070 comp rx=0 tx=0).stop 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 shutdown_connections 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe65806c920 0x7fe65806ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1ec880 0x7fe66c1ecd30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe66c135900 0x7fe66c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 --2- 172.21.15.135:0/1383855162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe66c155b50 0x7fe66c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 >> 172.21.15.135:0/1383855162 conn(0x7fe66c06dc10 msgr2=0x7fe66c159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 shutdown_connections 2024-04-26T21:59:58.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.758+0000 7fe6727bf640 1 -- 172.21.15.135:0/1383855162 wait complete. 2024-04-26T21:59:58.770 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-04-26T21:59:58.770 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a allow_standby_replay false 2024-04-26T21:59:58.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.868+0000 7f3dcee47640 1 Processor -- start 2024-04-26T21:59:58.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.868+0000 7f3dcee47640 1 -- start start 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcee47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc81554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcee47640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3dc806f920 0x7f3dc80abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcee47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc8070670 con 0x7f3dc8073aa0 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcee47640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc80ab8f0 con 0x7f3dc806f920 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcee47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc80700b0 con 0x7f3dc81523e0 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3dc806f920 0x7f3dc80abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3dc806f920 0x7f3dc80abd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46242/0 (socket says 172.21.15.135:46242) 2024-04-26T21:59:58.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 -- 172.21.15.135:0/2875267127 learned_addr learned my addr 172.21.15.135:0/2875267127 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dc7fff640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc81554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 -- 172.21.15.135:0/2875267127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 msgr2=0x7f3dc81554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc81554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 -- 172.21.15.135:0/2875267127 >> v1:172.21.15.139:6789/0 conn(0x7f3dc8073aa0 legacy=0x7f3dc8151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dcd3e4640 1 -- 172.21.15.135:0/2875267127 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc8070390 con 0x7f3dc806f920 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.869+0000 7f3dc7fff640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc81554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.870+0000 7f3dcd3e4640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc806f920 0x7f3dc80abd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009e50 tx=0x7f3dc002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade96e3bb8e50f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.870+0000 7f3dc77fe640 1 -- 172.21.15.135:0/2875267127 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0036070 con 0x7f3dc806f920 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.870+0000 7f3dc77fe640 1 -- 172.21.15.135:0/2875267127 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0031600 con 0x7f3dc806f920 2024-04-26T21:59:58.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.870+0000 7f3dc77fe640 1 -- 172.21.15.135:0/2875267127 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00319d0 con 0x7f3dc806f920 2024-04-26T21:59:58.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 -- 172.21.15.135:0/2875267127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc806f920 msgr2=0x7f3dc80abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc806f920 0x7f3dc80abd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009e50 tx=0x7f3dc002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 -- 172.21.15.135:0/2875267127 shutdown_connections 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc806f920 0x7f3dc80abd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 --2- 172.21.15.135:0/2875267127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc81554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 -- 172.21.15.135:0/2875267127 >> 172.21.15.135:0/2875267127 conn(0x7f3dc80c6ed0 msgr2=0x7f3dc80c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 -- 172.21.15.135:0/2875267127 shutdown_connections 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.871+0000 7f3dcee47640 1 -- 172.21.15.135:0/2875267127 wait complete. 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.872+0000 7f3dcee47640 1 Processor -- start 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.872+0000 7f3dcee47640 1 -- start start 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcee47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc806f920 0x7f3dc8148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcee47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 0x7f3dc80ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcee47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc8152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcee47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8073e80 con 0x7f3dc8073aa0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcee47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc806fdd0 con 0x7f3dc806f920 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcee47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc8074000 con 0x7f3dc81523e0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 0x7f3dc80ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 0x7f3dc80ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46254/0 (socket says 172.21.15.135:46254) 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 -- 172.21.15.135:0/1346056838 learned_addr learned my addr 172.21.15.135:0/1346056838 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dccbe3640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc806f920 0x7f3dc8148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dcd3e4640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc8152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 -- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 msgr2=0x7f3dc8152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc8152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 -- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc806f920 msgr2=0x7f3dc8148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc806f920 0x7f3dc8148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 -- 172.21.15.135:0/1346056838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc8153ef0 con 0x7f3dc8073aa0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc7fff640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 0x7f3dc80ade90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc010e30 tx=0x7f3dbc00e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc013890 con 0x7f3dc8073aa0 2024-04-26T21:59:58.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.873+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc013a30 con 0x7f3dc8073aa0 2024-04-26T21:59:58.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.874+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc8146be0 con 0x7f3dc8073aa0 2024-04-26T21:59:58.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.874+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc013e00 con 0x7f3dc8073aa0 2024-04-26T21:59:58.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.874+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc8151580 con 0x7f3dc8073aa0 2024-04-26T21:59:58.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.874+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc80507a0 con 0x7f3dc8073aa0 2024-04-26T21:59:58.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.875+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dbc01c400 con 0x7f3dc8073aa0 2024-04-26T21:59:58.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.875+0000 7f3dc57fa640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3db406c8f0 0x7f3db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.875+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f3dbc0906c0 con 0x7f3dc8073aa0 2024-04-26T21:59:58.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.875+0000 7f3dccbe3640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3db406c8f0 0x7f3db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.876+0000 7f3dccbe3640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3db406c8f0 0x7f3db406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3db8004f60 tx=0x7f3db8007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:58.880+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3dbc018030 con 0x7f3dc8073aa0 2024-04-26T21:59:59.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.094+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3dc8149b10 con 0x7f3dc8073aa0 2024-04-26T21:59:59.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.762+0000 7f3dc57fa640 1 -- 172.21.15.135:0/1346056838 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 117+0+0 (secure 0 0 0) 0x7f3dbc00ee20 con 0x7f3dc8073aa0 2024-04-26T21:59:59.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3db406c8f0 msgr2=0x7f3db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3db406c8f0 0x7f3db406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3db8004f60 tx=0x7f3db8007e30 comp rx=0 tx=0).stop 2024-04-26T21:59:59.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 msgr2=0x7f3dc80ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 0x7f3dc80ade90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc010e30 tx=0x7f3dbc00e650 comp rx=0 tx=0).stop 2024-04-26T21:59:59.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 shutdown_connections 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3db406c8f0 0x7f3db406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dc81523e0 0x7f3dc8152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc8073aa0 0x7f3dc80ade90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 --2- 172.21.15.135:0/1346056838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dc806f920 0x7f3dc8148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 >> 172.21.15.135:0/1346056838 conn(0x7f3dc80c6ed0 msgr2=0x7f3dc8151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 shutdown_connections 2024-04-26T21:59:59.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.765+0000 7f3dcee47640 1 -- 172.21.15.135:0/1346056838 wait complete. 2024-04-26T21:59:59.779 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a session_timeout 300 2024-04-26T21:59:59.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.883+0000 7f3ca8ce3640 1 Processor -- start 2024-04-26T21:59:59.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.883+0000 7f3ca8ce3640 1 -- start start 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.883+0000 7f3ca8ce3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3ca40c7000 0x7f3ca40ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.883+0000 7f3ca8ce3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca4156de0 con 0x7f3ca41543f0 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.883+0000 7f3ca8ce3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca4157b60 con 0x7f3ca40c9830 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.883+0000 7f3ca8ce3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca40cad40 con 0x7f3ca40c7000 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca2d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3ca41543f0 0x7f3ca4072930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38930/0 (socket says 172.21.15.135:38930) 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca2d9d640 1 -- 172.21.15.135:0/2683394209 learned_addr learned my addr 172.21.15.135:0/2683394209 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca259c640 1 --2- 172.21.15.135:0/2683394209 >> v2:172.21.15.139:3300/0 conn(0x7f3ca40c7000 0x7f3ca40ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 939269178 0 0) 0x7f3ca4156de0 con 0x7f3ca41543f0 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c8c003520 con 0x7f3ca41543f0 2024-04-26T21:59:59.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2038119642 0 0) 0x7f3c8c003520 con 0x7f3ca41543f0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 >> v2:172.21.15.139:3300/0 conn(0x7f3ca40c7000 msgr2=0x7f3ca40ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 --2- 172.21.15.135:0/2683394209 >> v2:172.21.15.139:3300/0 conn(0x7f3ca40c7000 0x7f3ca40ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 >> v1:172.21.15.139:6789/0 conn(0x7f3ca40c9830 legacy=0x7f3ca4153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca4157080 con 0x7f3ca41543f0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f3c94001690 con 0x7f3ca41543f0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 >> v1:172.21.15.135:6789/0 conn(0x7f3ca41543f0 legacy=0x7f3ca4072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3c8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c004800 0x7f3c8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0051c0 0x7f3c8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca40cad40 con 0x7f3c8c0039b0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c00c060 con 0x7f3c8c004800 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c00c260 con 0x7f3c8c0051c0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca259c640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3c8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca1d9b640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c004800 0x7f3c8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.884+0000 7f3ca2d9d640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0051c0 0x7f3c8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca259c640 1 -- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0051c0 msgr2=0x7f3c8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca259c640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0051c0 0x7f3c8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca259c640 1 -- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c004800 msgr2=0x7f3c8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca259c640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c004800 0x7f3c8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca259c640 1 -- 172.21.15.135:0/2683394209 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c003520 con 0x7f3c8c0039b0 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca2d9d640 1 -- 172.21.15.135:0/2683394209 reap_dead start 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca259c640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3c8c003dd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009fc0 tx=0x7f3c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337e144066cf9443 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c009c10 con 0x7f3c8c0039b0 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c036070 con 0x7f3c8c0039b0 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.885+0000 7f3ca159a640 1 -- 172.21.15.135:0/2683394209 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c031880 con 0x7f3c8c0039b0 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/2683394209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 msgr2=0x7f3c8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3c8c003dd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009fc0 tx=0x7f3c8c005830 comp rx=0 tx=0).stop 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/2683394209 shutdown_connections 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/2683394209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3c8c003dd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/2683394209 >> 172.21.15.135:0/2683394209 conn(0x7f3ca40adf20 msgr2=0x7f3ca40ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:59.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/2683394209 shutdown_connections 2024-04-26T21:59:59.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.886+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/2683394209 wait complete. 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.887+0000 7f3ca8ce3640 1 Processor -- start 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.887+0000 7f3ca8ce3640 1 -- start start 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca8ce3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3ca415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca8ce3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ca414db60 0x7f3ca4159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca8ce3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca4159840 0x7f3ca40c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca8ce3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca40c73e0 con 0x7f3c8c0039b0 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca8ce3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4157640 con 0x7f3ca4159840 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca8ce3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca40c7560 con 0x7f3ca414db60 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3ca415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3ca415a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46276/0 (socket says 172.21.15.135:46276) 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 -- 172.21.15.135:0/1194157225 learned_addr learned my addr 172.21.15.135:0/1194157225 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca1d9b640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ca414db60 0x7f3ca4159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca2d9d640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca4159840 0x7f3ca40c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 -- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ca414db60 msgr2=0x7f3ca4159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ca414db60 0x7f3ca4159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 -- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca4159840 msgr2=0x7f3ca40c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca4159840 0x7f3ca40c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 -- 172.21.15.135:0/1194157225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca414d910 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.888+0000 7f3ca259c640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3ca415a570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c02aee0 tx=0x7f3c9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.889+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c009c10 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.889+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c003990 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.889+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca414d630 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.889+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c003d60 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.889+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca40c3860 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.890+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca40766c0 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.890+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c9c005d10 con 0x7f3c8c0039b0 2024-04-26T21:59:59.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.890+0000 7f3c92ffd640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3c8406c8f0 0x7f3c8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:59.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.890+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f3c9c0acdd0 con 0x7f3c8c0039b0 2024-04-26T21:59:59.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.890+0000 7f3ca1d9b640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3c8406c8f0 0x7f3c8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:59.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.891+0000 7f3ca1d9b640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3c8406c8f0 0x7f3c8406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:59.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T21:59:59.895+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c9c043380 con 0x7f3c8c0039b0 2024-04-26T22:00:00.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.108+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"} v 0) -- 0x7f3ca40507a0 con 0x7f3c8c0039b0 2024-04-26T22:00:00.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.771+0000 7f3c92ffd640 1 -- 172.21.15.135:0/1194157225 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"}]=0 v8) ==== 110+0+0 (secure 0 0 0) 0x7f3c9c07b220 con 0x7f3c8c0039b0 2024-04-26T22:00:00.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3c8406c8f0 msgr2=0x7f3c8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3c8406c8f0 0x7f3c8406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c98009040 comp rx=0 tx=0).stop 2024-04-26T22:00:00.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 msgr2=0x7f3ca415a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3ca415a570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c02aee0 tx=0x7f3c9c009750 comp rx=0 tx=0).stop 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 shutdown_connections 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3c8406c8f0 0x7f3c8406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ca4159840 0x7f3ca40c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ca414db60 0x7f3ca4159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 --2- 172.21.15.135:0/1194157225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c8c0039b0 0x7f3ca415a570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 >> 172.21.15.135:0/1194157225 conn(0x7f3ca40adf20 msgr2=0x7f3ca40c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 shutdown_connections 2024-04-26T22:00:00.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.773+0000 7f3ca8ce3640 1 -- 172.21.15.135:0/1194157225 wait complete. 2024-04-26T22:00:00.785 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'a' 2024-04-26T22:00:00.785 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.887+0000 7fbeb014d640 1 Processor -- start 2024-04-26T22:00:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.887+0000 7fbeb014d640 1 -- start start 2024-04-26T22:00:00.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbeb014d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea8155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbeb014d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbea80ab620 0x7fbea80aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbeb014d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea80737a0 con 0x7fbea80c63d0 2024-04-26T22:00:00.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbeb014d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbea8073a40 con 0x7fbea80ab620 2024-04-26T22:00:00.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbeb014d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbea8073d20 con 0x7fbea80acdc0 2024-04-26T22:00:00.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbead6e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbeae6ea640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbea80ab620 0x7fbea80aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbead6e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea8155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:50002/0 (socket says 172.21.15.135:50002) 2024-04-26T22:00:00.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.888+0000 7fbead6e8640 1 -- 172.21.15.135:0/3635956567 learned_addr learned my addr 172.21.15.135:0/3635956567 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:00.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.889+0000 7fbead6e8640 1 -- 172.21.15.135:0/3635956567 >> v2:172.21.15.139:3300/0 conn(0x7fbea80ab620 msgr2=0x7fbea80aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.889+0000 7fbead6e8640 1 --2- 172.21.15.135:0/3635956567 >> v2:172.21.15.139:3300/0 conn(0x7fbea80ab620 0x7fbea80aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.889+0000 7fbead6e8640 1 -- 172.21.15.135:0/3635956567 >> v1:172.21.15.139:6789/0 conn(0x7fbea80c63d0 legacy=0x7fbea80c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.889+0000 7fbead6e8640 1 -- 172.21.15.135:0/3635956567 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea8074000 con 0x7fbea80acdc0 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.889+0000 7fbeae6ea640 1 --2- 172.21.15.135:0/3635956567 >> v2:172.21.15.139:3300/0 conn(0x7fbea80ab620 0x7fbea80aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.889+0000 7fbead6e8640 1 --2- 172.21.15.135:0/3635956567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea8155420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009870 tx=0x7fbe9802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01cf0a6378543d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeacee7640 1 -- 172.21.15.135:0/3635956567 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98037070 con 0x7fbea80acdc0 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeacee7640 1 -- 172.21.15.135:0/3635956567 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe98032440 con 0x7fbea80acdc0 2024-04-26T22:00:00.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeacee7640 1 -- 172.21.15.135:0/3635956567 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98032810 con 0x7fbea80acdc0 2024-04-26T22:00:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 -- 172.21.15.135:0/3635956567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 msgr2=0x7fbea8155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 --2- 172.21.15.135:0/3635956567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea8155420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009870 tx=0x7fbe9802a270 comp rx=0 tx=0).stop 2024-04-26T22:00:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 -- 172.21.15.135:0/3635956567 shutdown_connections 2024-04-26T22:00:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 --2- 172.21.15.135:0/3635956567 >> v2:172.21.15.139:3300/0 conn(0x7fbea80ab620 0x7fbea80aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 --2- 172.21.15.135:0/3635956567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea8155420 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 -- 172.21.15.135:0/3635956567 >> 172.21.15.135:0/3635956567 conn(0x7fbea80c6e60 msgr2=0x7fbea80c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:00.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 -- 172.21.15.135:0/3635956567 shutdown_connections 2024-04-26T22:00:00.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.890+0000 7fbeb014d640 1 -- 172.21.15.135:0/3635956567 wait complete. 2024-04-26T22:00:00.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.891+0000 7fbeb014d640 1 Processor -- start 2024-04-26T22:00:00.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.891+0000 7fbeb014d640 1 -- start start 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeb014d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 0x7fbea81eb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeb014d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea81580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeb014d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbea80c63d0 0x7fbea81edc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeb014d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbea80c9ed0 con 0x7fbea80ab620 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeb014d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8074d60 con 0x7fbea80c63d0 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeb014d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbea80ca050 con 0x7fbea80acdc0 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeadee9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 0x7fbea81eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeadee9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 0x7fbea81eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46288/0 (socket says 172.21.15.135:46288) 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeadee9640 1 -- 172.21.15.135:0/2160249717 learned_addr learned my addr 172.21.15.135:0/2160249717 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbeae6ea640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbea80c63d0 0x7fbea81edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.892+0000 7fbead6e8640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea81580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeadee9640 1 -- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 msgr2=0x7fbea81580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeadee9640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea81580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeadee9640 1 -- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbea80c63d0 msgr2=0x7fbea81edc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeadee9640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbea80c63d0 0x7fbea81edc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeadee9640 1 -- 172.21.15.135:0/2160249717 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea814f1a0 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeae6ea640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbea80c63d0 0x7fbea81edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeadee9640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 0x7fbea81eb910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbea400c730 tx=0x7fbea400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4016020 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea4006d50 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea814f480 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003bb0 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.893+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81578d0 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.894+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea80507a0 con 0x7fbea80ab620 2024-04-26T22:00:00.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.894+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbea400a850 con 0x7fbea80ab620 2024-04-26T22:00:00.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.895+0000 7fbe9e7fc640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:00.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.895+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fbea408b7d0 con 0x7fbea80ab620 2024-04-26T22:00:00.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.895+0000 7fbead6e8640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:00.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.895+0000 7fbead6e8640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbe98034000 tx=0x7fbe9802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:00.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:00.899+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbea4020020 con 0x7fbea80ab620 2024-04-26T22:00:01.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.115+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbea80af910 con 0x7fbea80ab620 2024-04-26T22:00:01.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.116+0000 7fbe9e7fc640 1 -- 172.21.15.135:0/2160249717 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fbea4019310 con 0x7fbea80ab620 2024-04-26T22:00:01.116 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:01.116 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-04-26T22:00:01.116 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 8 2024-04-26T22:00:01.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbe8c06c8f0 msgr2=0x7fbe8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbe98034000 tx=0x7fbe9802e000 comp rx=0 tx=0).stop 2024-04-26T22:00:01.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 msgr2=0x7fbea81eb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 0x7fbea81eb910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbea400c730 tx=0x7fbea400cbf0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 shutdown_connections 2024-04-26T22:00:01.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbea80c63d0 0x7fbea81edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbea80acdc0 0x7fbea81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 --2- 172.21.15.135:0/2160249717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbea80ab620 0x7fbea81eb910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 >> 172.21.15.135:0/2160249717 conn(0x7fbea80c6e60 msgr2=0x7fbea80c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 shutdown_connections 2024-04-26T22:00:01.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.118+0000 7fbeb014d640 1 -- 172.21.15.135:0/2160249717 wait complete. 2024-04-26T22:00:01.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:01.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.225+0000 7fe6f93b7640 1 Processor -- start 2024-04-26T22:00:01.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.225+0000 7fe6f93b7640 1 -- start start 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.226+0000 7fe6f93b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 0x7fe6f4074170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.226+0000 7fe6f93b7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f4070a30 con 0x7fe6f4154060 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.226+0000 7fe6f93b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f4074df0 con 0x7fe6f4156a60 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.226+0000 7fe6f93b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f4070ca0 con 0x7fe6f40c47d0 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.226+0000 7fe6f2ffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe6f4154060 0x7fe6f40ca740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38934/0 (socket says 172.21.15.135:38934) 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.226+0000 7fe6f2ffd640 1 -- 172.21.15.135:0/3005460545 learned_addr learned my addr 172.21.15.135:0/3005460545 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f27fc640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 0x7fe6f4074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870687315 0 0) 0x7fe6f4070a30 con 0x7fe6f4154060 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6dc003520 con 0x7fe6f4154060 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587276199 0 0) 0x7fe6f4074df0 con 0x7fe6f4156a60 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6f4070a30 con 0x7fe6f4156a60 2024-04-26T22:00:01.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 991923141 0 0) 0x7fe6dc003520 con 0x7fe6f4154060 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 msgr2=0x7fe6f4074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 0x7fe6f4074170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 >> v1:172.21.15.139:6789/0 conn(0x7fe6f4156a60 legacy=0x7fe6f4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f4070f80 con 0x7fe6f4154060 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fe6e40019f0 con 0x7fe6f4154060 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 >> v1:172.21.15.135:6789/0 conn(0x7fe6f4154060 legacy=0x7fe6f40ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 0x7fe6dc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc004140 0x7fe6dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc004d40 0x7fe6dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4070ca0 con 0x7fe6dc004d40 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc00bb00 con 0x7fe6dc004140 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc00bd00 con 0x7fe6f40c47d0 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc004d40 0x7fe6dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f27fc640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc004140 0x7fe6dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f2ffd640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 0x7fe6dc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f37fe640 1 -- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 msgr2=0x7fe6dc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40c47d0 0x7fe6dc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f37fe640 1 -- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc004140 msgr2=0x7fe6dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc004140 0x7fe6dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.227+0000 7fe6f37fe640 1 -- 172.21.15.135:0/3005460545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc003520 con 0x7fe6dc004d40 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.228+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc004d40 0x7fe6dc0051c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0018d0 tx=0x7fe6dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cc383e57d30643 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.228+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0014f0 con 0x7fe6dc004d40 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.228+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ec0052d0 con 0x7fe6dc004d40 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.228+0000 7fe6f1ffb640 1 -- 172.21.15.135:0/3005460545 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec02aca0 con 0x7fe6dc004d40 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f93b7640 1 -- 172.21.15.135:0/3005460545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc004d40 msgr2=0x7fe6dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/3005460545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc004d40 0x7fe6dc0051c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0018d0 tx=0x7fe6dc005830 comp rx=0 tx=0).stop 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f37fe640 1 -- 172.21.15.135:0/3005460545 reap_dead start 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f93b7640 1 -- 172.21.15.135:0/3005460545 shutdown_connections 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f93b7640 1 -- 172.21.15.135:0/3005460545 >> 172.21.15.135:0/3005460545 conn(0x7fe6f406e700 msgr2=0x7fe6f406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f93b7640 1 -- 172.21.15.135:0/3005460545 shutdown_connections 2024-04-26T22:00:01.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.229+0000 7fe6f93b7640 1 -- 172.21.15.135:0/3005460545 wait complete. 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.230+0000 7fe6f93b7640 1 Processor -- start 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.230+0000 7fe6f93b7640 1 -- start start 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f406dca0 0x7fe6f4157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f4150e20 0x7fe6f41f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 0x7fe6f4158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40747a0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4154440 con 0x7fe6f406dca0 2024-04-26T22:00:01.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40cae50 con 0x7fe6f4150e20 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 0x7fe6f4158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 0x7fe6f4158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46306/0 (socket says 172.21.15.135:46306) 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 -- 172.21.15.135:0/2490969161 learned_addr learned my addr 172.21.15.135:0/2490969161 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f27fc640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f4150e20 0x7fe6f41f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f2ffd640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f406dca0 0x7fe6f4157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 -- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f4150e20 msgr2=0x7fe6f41f0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f4150e20 0x7fe6f41f0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 -- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f406dca0 msgr2=0x7fe6f4157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f406dca0 0x7fe6f4157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 -- 172.21.15.135:0/2490969161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40ca0d0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f37fe640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 0x7fe6f4158770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec005040 tx=0x7fe6ec0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0014f0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ec041070 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec005480 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f415a7f0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.231+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f41ec9e0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.232+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f406f530 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.233+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6ec03a070 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.233+0000 7fe6db7fe640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6e006c920 0x7fe6e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.233+0000 7fe6f2ffd640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6e006c920 0x7fe6e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.233+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fe6ec0ab1b0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.234+0000 7fe6f2ffd640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6e006c920 0x7fe6e006ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4004bc0 tx=0x7fe6e4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.238+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6ec079560 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.414+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6f40507a0 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.415+0000 7fe6db7fe640 1 -- 172.21.15.135:0/2490969161 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16633 (secure 0 0 0) 0x7fe6ec078e60 con 0x7fe6f41ed1e0 2024-04-26T22:00:01.415 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:01.415 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":20,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:58.630451+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:01.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6e006c920 msgr2=0x7fe6e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6e006c920 0x7fe6e006ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4004bc0 tx=0x7fe6e4006040 comp rx=0 tx=0).stop 2024-04-26T22:00:01.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 msgr2=0x7fe6f4158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 0x7fe6f4158770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec005040 tx=0x7fe6ec0012d0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 shutdown_connections 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6e006c920 0x7fe6e006ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6f41ed1e0 0x7fe6f4158770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f4150e20 0x7fe6f41f0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 --2- 172.21.15.135:0/2490969161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f406dca0 0x7fe6f4157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 >> 172.21.15.135:0/2490969161 conn(0x7fe6f406e700 msgr2=0x7fe6f4074920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 shutdown_connections 2024-04-26T22:00:01.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.417+0000 7fe6f93b7640 1 -- 172.21.15.135:0/2490969161 wait complete. 2024-04-26T22:00:01.429 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:00:01.429 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:01.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.526+0000 7f3d056da640 1 Processor -- start 2024-04-26T22:00:01.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.526+0000 7f3d056da640 1 -- start start 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3d056da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d00154160 0x7f3d00156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3d056da640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3d000ca5a0 0x7f3d0015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3d056da640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d000c8df0 con 0x7f3d000c6e00 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3d056da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3d0015a6e0 con 0x7f3d00154160 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3d056da640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3d000c9100 con 0x7f3d000ca5a0 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d00154160 0x7f3d00156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d00154160 0x7f3d00156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46318/0 (socket says 172.21.15.135:46318) 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 -- 172.21.15.135:0/3774060264 learned_addr learned my addr 172.21.15.135:0/3774060264 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 -- 172.21.15.135:0/3774060264 >> v2:172.21.15.139:3300/0 conn(0x7f3d000ca5a0 msgr2=0x7f3d0015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfe7fc640 1 --2- 172.21.15.135:0/3774060264 >> v2:172.21.15.139:3300/0 conn(0x7f3d000ca5a0 0x7f3d0015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 --2- 172.21.15.135:0/3774060264 >> v2:172.21.15.139:3300/0 conn(0x7f3d000ca5a0 0x7f3d0015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 -- 172.21.15.135:0/3774060264 >> v1:172.21.15.139:6789/0 conn(0x7f3d000c6e00 legacy=0x7f3d000c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.527+0000 7f3cfeffd640 1 -- 172.21.15.135:0/3774060264 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d00154540 con 0x7f3d00154160 2024-04-26T22:00:01.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3cfeffd640 1 --2- 172.21.15.135:0/3774060264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d00154160 0x7f3d00156bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d000c9280 tx=0x7f3cf002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb18388359ef63cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3cfdffb640 1 -- 172.21.15.135:0/3774060264 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0036070 con 0x7f3d00154160 2024-04-26T22:00:01.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3cfdffb640 1 -- 172.21.15.135:0/3774060264 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf0031600 con 0x7f3d00154160 2024-04-26T22:00:01.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3cfdffb640 1 -- 172.21.15.135:0/3774060264 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00319d0 con 0x7f3d00154160 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3d056da640 1 -- 172.21.15.135:0/3774060264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d00154160 msgr2=0x7f3d00156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3d056da640 1 --2- 172.21.15.135:0/3774060264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d00154160 0x7f3d00156bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d000c9280 tx=0x7f3cf002a6d0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3d056da640 1 -- 172.21.15.135:0/3774060264 shutdown_connections 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3d056da640 1 --2- 172.21.15.135:0/3774060264 >> v2:172.21.15.139:3300/0 conn(0x7f3d000ca5a0 0x7f3d0015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3d056da640 1 --2- 172.21.15.135:0/3774060264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d00154160 0x7f3d00156bb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.528+0000 7f3d056da640 1 -- 172.21.15.135:0/3774060264 >> 172.21.15.135:0/3774060264 conn(0x7f3d000ae010 msgr2=0x7f3d000ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.529+0000 7f3d056da640 1 -- 172.21.15.135:0/3774060264 shutdown_connections 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.529+0000 7f3d056da640 1 -- 172.21.15.135:0/3774060264 wait complete. 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 Processor -- start 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 -- start start 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d000c6e00 0x7f3d0014a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 0x7f3d0014ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d00154160 0x7f3d00159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0015aed0 con 0x7f3d000ca5a0 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0015abf0 con 0x7f3d000c6e00 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.530+0000 7f3d056da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d000c9ca0 con 0x7f3d00154160 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 0x7f3d0014ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 0x7f3d0014ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46326/0 (socket says 172.21.15.135:46326) 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 -- 172.21.15.135:0/969661072 learned_addr learned my addr 172.21.15.135:0/969661072 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfeffd640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d000c6e00 0x7f3d0014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 -- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d00154160 msgr2=0x7f3d00159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cff7fe640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d00154160 0x7f3d00159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d00154160 0x7f3d00159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 -- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d000c6e00 msgr2=0x7f3d0014a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d000c6e00 0x7f3d0014a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 -- 172.21.15.135:0/969661072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0014d540 con 0x7f3d000ca5a0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3cfe7fc640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 0x7f3d0014ad00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c820 tx=0x7f3cf400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4015070 con 0x7f3d000ca5a0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf4004240 con 0x7f3d000ca5a0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d001f0bc0 con 0x7f3d000ca5a0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400ce10 con 0x7f3d000ca5a0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.531+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d001f00e0 con 0x7f3d000ca5a0 2024-04-26T22:00:01.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.532+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d00076650 con 0x7f3d000ca5a0 2024-04-26T22:00:01.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.533+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cf4007830 con 0x7f3d000ca5a0 2024-04-26T22:00:01.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.533+0000 7f3ce77fe640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3cec06c8f0 0x7f3cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.533+0000 7f3cfeffd640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3cec06c8f0 0x7f3cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.533+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7f3cf408b990 con 0x7f3d000ca5a0 2024-04-26T22:00:01.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.533+0000 7f3cfeffd640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3cec06c8f0 0x7f3cec06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0033000 tx=0x7f3cf0009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.537+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cf4059e60 con 0x7f3d000ca5a0 2024-04-26T22:00:01.618 INFO:tasks.ceph.mds.a.smithi135.stderr:2024-04-26T22:00:01.619+0000 7ff9bd210640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:00:01.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.712+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d000507a0 con 0x7f3cec06c8f0 2024-04-26T22:00:01.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.718+0000 7f3ce77fe640 1 -- 172.21.15.135:0/969661072 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f3d000507a0 con 0x7f3cec06c8f0 2024-04-26T22:00:01.721 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:01.722 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3cec06c8f0 msgr2=0x7f3cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3cec06c8f0 0x7f3cec06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0033000 tx=0x7f3cf0009000 comp rx=0 tx=0).stop 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 msgr2=0x7f3d0014ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 0x7f3d0014ad00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c820 tx=0x7f3cf400cce0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 shutdown_connections 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3cec06c8f0 0x7f3cec06edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d00154160 0x7f3d00159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d000ca5a0 0x7f3d0014ad00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 --2- 172.21.15.135:0/969661072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d000c6e00 0x7f3d0014a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 >> 172.21.15.135:0/969661072 conn(0x7f3d000ae010 msgr2=0x7f3d0014fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 shutdown_connections 2024-04-26T22:00:01.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.721+0000 7f3d056da640 1 -- 172.21.15.135:0/969661072 wait complete. 2024-04-26T22:00:01.732 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T22:00:01.069764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2854912,"data_stored":1733563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.446871"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016402+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T21:59:59.016402+0000","last_peered":"2024-04-26T21:59:59.016402+0000","last_clean":"2024-04-26T21:59:59.016402+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T21:59:59.016402+0000","last_undegraded":"2024-04-26T21:59:59.016402+0000","last_fullsized":"2024-04-26T21:59:59.016402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.634003+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T21:59:58.634003+0000","last_peered":"2024-04-26T21:59:58.634003+0000","last_clean":"2024-04-26T21:59:58.634003+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T21:59:58.634003+0000","last_undegraded":"2024-04-26T21:59:58.634003+0000","last_fullsized":"2024-04-26T21:59:58.634003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656155+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T21:59:57.656155+0000","last_peered":"2024-04-26T21:59:57.656155+0000","last_clean":"2024-04-26T21:59:57.656155+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T21:59:57.656155+0000","last_undegraded":"2024-04-26T21:59:57.656155+0000","last_fullsized":"2024-04-26T21:59:57.656155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315533+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T21:59:59.315533+0000","last_peered":"2024-04-26T21:59:59.315533+0000","last_clean":"2024-04-26T21:59:59.315533+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T21:59:59.315533+0000","last_undegraded":"2024-04-26T21:59:59.315533+0000","last_fullsized":"2024-04-26T21:59:59.315533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016022+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T21:59:59.016022+0000","last_peered":"2024-04-26T21:59:59.016022+0000","last_clean":"2024-04-26T21:59:59.016022+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T21:59:59.016022+0000","last_undegraded":"2024-04-26T21:59:59.016022+0000","last_fullsized":"2024-04-26T21:59:59.016022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315482+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T21:59:59.315482+0000","last_peered":"2024-04-26T21:59:59.315482+0000","last_clean":"2024-04-26T21:59:59.315482+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T21:59:59.315482+0000","last_undegraded":"2024-04-26T21:59:59.315482+0000","last_fullsized":"2024-04-26T21:59:59.315482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015951+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T21:59:59.015951+0000","last_peered":"2024-04-26T21:59:59.015951+0000","last_clean":"2024-04-26T21:59:59.015951+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T21:59:59.015951+0000","last_undegraded":"2024-04-26T21:59:59.015951+0000","last_fullsized":"2024-04-26T21:59:59.015951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707463+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T21:59:59.707463+0000","last_peered":"2024-04-26T21:59:59.707463+0000","last_clean":"2024-04-26T21:59:59.707463+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T21:59:59.707463+0000","last_undegraded":"2024-04-26T21:59:59.707463+0000","last_fullsized":"2024-04-26T21:59:59.707463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016415+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T21:59:59.016415+0000","last_peered":"2024-04-26T21:59:59.016415+0000","last_clean":"2024-04-26T21:59:59.016415+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T21:59:59.016415+0000","last_undegraded":"2024-04-26T21:59:59.016415+0000","last_fullsized":"2024-04-26T21:59:59.016415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638047+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T21:59:58.638047+0000","last_peered":"2024-04-26T21:59:58.638047+0000","last_clean":"2024-04-26T21:59:58.638047+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T21:59:58.638047+0000","last_undegraded":"2024-04-26T21:59:58.638047+0000","last_fullsized":"2024-04-26T21:59:58.638047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637658+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T21:59:58.637658+0000","last_peered":"2024-04-26T21:59:58.637658+0000","last_clean":"2024-04-26T21:59:58.637658+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T21:59:58.637658+0000","last_undegraded":"2024-04-26T21:59:58.637658+0000","last_fullsized":"2024-04-26T21:59:58.637658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638530+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T21:59:58.638530+0000","last_peered":"2024-04-26T21:59:58.638530+0000","last_clean":"2024-04-26T21:59:58.638530+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T21:59:58.638530+0000","last_undegraded":"2024-04-26T21:59:58.638530+0000","last_fullsized":"2024-04-26T21:59:58.638530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707864+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T21:59:59.707864+0000","last_peered":"2024-04-26T21:59:59.707864+0000","last_clean":"2024-04-26T21:59:59.707864+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T21:59:59.707864+0000","last_undegraded":"2024-04-26T21:59:59.707864+0000","last_fullsized":"2024-04-26T21:59:59.707864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654923+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T21:59:57.654923+0000","last_peered":"2024-04-26T21:59:57.654923+0000","last_clean":"2024-04-26T21:59:57.654923+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T21:59:57.654923+0000","last_undegraded":"2024-04-26T21:59:57.654923+0000","last_fullsized":"2024-04-26T21:59:57.654923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:21.311477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633859+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T21:59:58.633859+0000","last_peered":"2024-04-26T21:59:58.633859+0000","last_clean":"2024-04-26T21:59:58.633859+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T21:59:58.633859+0000","last_undegraded":"2024-04-26T21:59:58.633859+0000","last_fullsized":"2024-04-26T21:59:58.633859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315337+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T21:59:59.315337+0000","last_peered":"2024-04-26T21:59:59.315337+0000","last_clean":"2024-04-26T21:59:59.315337+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T21:59:59.315337+0000","last_undegraded":"2024-04-26T21:59:59.315337+0000","last_fullsized":"2024-04-26T21:59:59.315337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.316014+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T21:59:59.316014+0000","last_peered":"2024-04-26T21:59:59.316014+0000","last_clean":"2024-04-26T21:59:59.316014+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T21:59:59.316014+0000","last_undegraded":"2024-04-26T21:59:59.316014+0000","last_fullsized":"2024-04-26T21:59:59.316014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654666+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T21:59:57.654666+0000","last_peered":"2024-04-26T21:59:57.654666+0000","last_clean":"2024-04-26T21:59:57.654666+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T21:59:57.654666+0000","last_undegraded":"2024-04-26T21:59:57.654666+0000","last_fullsized":"2024-04-26T21:59:57.654666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016427+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T21:59:59.016427+0000","last_peered":"2024-04-26T21:59:59.016427+0000","last_clean":"2024-04-26T21:59:59.016427+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T21:59:59.016427+0000","last_undegraded":"2024-04-26T21:59:59.016427+0000","last_fullsized":"2024-04-26T21:59:59.016427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637715+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T21:59:58.637715+0000","last_peered":"2024-04-26T21:59:58.637715+0000","last_clean":"2024-04-26T21:59:58.637715+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T21:59:58.637715+0000","last_undegraded":"2024-04-26T21:59:58.637715+0000","last_fullsized":"2024-04-26T21:59:58.637715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656154+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T21:59:57.656154+0000","last_peered":"2024-04-26T21:59:57.656154+0000","last_clean":"2024-04-26T21:59:57.656154+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T21:59:57.656154+0000","last_undegraded":"2024-04-26T21:59:57.656154+0000","last_fullsized":"2024-04-26T21:59:57.656154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315530+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T21:59:59.315530+0000","last_peered":"2024-04-26T21:59:59.315530+0000","last_clean":"2024-04-26T21:59:59.315530+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T21:59:59.315530+0000","last_undegraded":"2024-04-26T21:59:59.315530+0000","last_fullsized":"2024-04-26T21:59:59.315530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707605+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T21:59:59.707605+0000","last_peered":"2024-04-26T21:59:59.707605+0000","last_clean":"2024-04-26T21:59:59.707605+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T21:59:59.707605+0000","last_undegraded":"2024-04-26T21:59:59.707605+0000","last_fullsized":"2024-04-26T21:59:59.707605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315406+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T21:59:59.315406+0000","last_peered":"2024-04-26T21:59:59.315406+0000","last_clean":"2024-04-26T21:59:59.315406+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T21:59:59.315406+0000","last_undegraded":"2024-04-26T21:59:59.315406+0000","last_fullsized":"2024-04-26T21:59:59.315406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708018+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T21:59:59.708018+0000","last_peered":"2024-04-26T21:59:59.708018+0000","last_clean":"2024-04-26T21:59:59.708018+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T21:59:59.708018+0000","last_undegraded":"2024-04-26T21:59:59.708018+0000","last_fullsized":"2024-04-26T21:59:59.708018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638216+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T21:59:58.638216+0000","last_peered":"2024-04-26T21:59:58.638216+0000","last_clean":"2024-04-26T21:59:58.638216+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T21:59:58.638216+0000","last_undegraded":"2024-04-26T21:59:58.638216+0000","last_fullsized":"2024-04-26T21:59:58.638216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633789+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T21:59:58.633789+0000","last_peered":"2024-04-26T21:59:58.633789+0000","last_clean":"2024-04-26T21:59:58.633789+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T21:59:58.633789+0000","last_undegraded":"2024-04-26T21:59:58.633789+0000","last_fullsized":"2024-04-26T21:59:58.633789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707880+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T21:59:59.707880+0000","last_peered":"2024-04-26T21:59:59.707880+0000","last_clean":"2024-04-26T21:59:59.707880+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T21:59:59.707880+0000","last_undegraded":"2024-04-26T21:59:59.707880+0000","last_fullsized":"2024-04-26T21:59:59.707880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T21:59:57.656134+0000","last_peered":"2024-04-26T21:59:57.656134+0000","last_clean":"2024-04-26T21:59:57.656134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T21:59:57.656134+0000","last_undegraded":"2024-04-26T21:59:57.656134+0000","last_fullsized":"2024-04-26T21:59:57.656134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315660+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T21:59:59.315660+0000","last_peered":"2024-04-26T21:59:59.315660+0000","last_clean":"2024-04-26T21:59:59.315660+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T21:59:59.315660+0000","last_undegraded":"2024-04-26T21:59:59.315660+0000","last_fullsized":"2024-04-26T21:59:59.315660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.659987+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T21:59:57.659987+0000","last_peered":"2024-04-26T21:59:57.659987+0000","last_clean":"2024-04-26T21:59:57.659987+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T21:59:57.659987+0000","last_undegraded":"2024-04-26T21:59:57.659987+0000","last_fullsized":"2024-04-26T21:59:57.659987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637363+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T21:59:58.637363+0000","last_peered":"2024-04-26T21:59:58.637363+0000","last_clean":"2024-04-26T21:59:58.637363+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T21:59:58.637363+0000","last_undegraded":"2024-04-26T21:59:58.637363+0000","last_fullsized":"2024-04-26T21:59:58.637363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637591+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T21:59:58.637591+0000","last_peered":"2024-04-26T21:59:58.637591+0000","last_clean":"2024-04-26T21:59:58.637591+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T21:59:58.637591+0000","last_undegraded":"2024-04-26T21:59:58.637591+0000","last_fullsized":"2024-04-26T21:59:58.637591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707915+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T21:59:59.707915+0000","last_peered":"2024-04-26T21:59:59.707915+0000","last_clean":"2024-04-26T21:59:59.707915+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T21:59:59.707915+0000","last_undegraded":"2024-04-26T21:59:59.707915+0000","last_fullsized":"2024-04-26T21:59:59.707915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656112+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T21:59:57.656112+0000","last_peered":"2024-04-26T21:59:57.656112+0000","last_clean":"2024-04-26T21:59:57.656112+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T21:59:57.656112+0000","last_undegraded":"2024-04-26T21:59:57.656112+0000","last_fullsized":"2024-04-26T21:59:57.656112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:58.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638288+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T21:59:58.638288+0000","last_peered":"2024-04-26T21:59:58.638288+0000","last_clean":"2024-04-26T21:59:58.638288+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T21:59:58.638288+0000","last_undegraded":"2024-04-26T21:59:58.638288+0000","last_fullsized":"2024-04-26T21:59:58.638288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707700+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T21:59:59.707700+0000","last_peered":"2024-04-26T21:59:59.707700+0000","last_clean":"2024-04-26T21:59:59.707700+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T21:59:59.707700+0000","last_undegraded":"2024-04-26T21:59:59.707700+0000","last_fullsized":"2024-04-26T21:59:59.707700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707594+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T21:59:59.707594+0000","last_peered":"2024-04-26T21:59:59.707594+0000","last_clean":"2024-04-26T21:59:59.707594+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T21:59:59.707594+0000","last_undegraded":"2024-04-26T21:59:59.707594+0000","last_fullsized":"2024-04-26T21:59:59.707594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315502+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T21:59:59.315502+0000","last_peered":"2024-04-26T21:59:59.315502+0000","last_clean":"2024-04-26T21:59:59.315502+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T21:59:59.315502+0000","last_undegraded":"2024-04-26T21:59:59.315502+0000","last_fullsized":"2024-04-26T21:59:59.315502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633698+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T21:59:58.633698+0000","last_peered":"2024-04-26T21:59:58.633698+0000","last_clean":"2024-04-26T21:59:58.633698+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T21:59:58.633698+0000","last_undegraded":"2024-04-26T21:59:58.633698+0000","last_fullsized":"2024-04-26T21:59:58.633698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637863+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T21:59:58.637863+0000","last_peered":"2024-04-26T21:59:58.637863+0000","last_clean":"2024-04-26T21:59:58.637863+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T21:59:58.637863+0000","last_undegraded":"2024-04-26T21:59:58.637863+0000","last_fullsized":"2024-04-26T21:59:58.637863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315907+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T21:59:59.315907+0000","last_peered":"2024-04-26T21:59:59.315907+0000","last_clean":"2024-04-26T21:59:59.315907+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T21:59:59.315907+0000","last_undegraded":"2024-04-26T21:59:59.315907+0000","last_fullsized":"2024-04-26T21:59:59.315907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315932+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T21:59:59.315932+0000","last_peered":"2024-04-26T21:59:59.315932+0000","last_clean":"2024-04-26T21:59:59.315932+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T21:59:59.315932+0000","last_undegraded":"2024-04-26T21:59:59.315932+0000","last_fullsized":"2024-04-26T21:59:59.315932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.655796+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T21:59:57.655796+0000","last_peered":"2024-04-26T21:59:57.655796+0000","last_clean":"2024-04-26T21:59:57.655796+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T21:59:57.655796+0000","last_undegraded":"2024-04-26T21:59:57.655796+0000","last_fullsized":"2024-04-26T21:59:57.655796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633771+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T21:59:58.633771+0000","last_peered":"2024-04-26T21:59:58.633771+0000","last_clean":"2024-04-26T21:59:58.633771+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T21:59:58.633771+0000","last_undegraded":"2024-04-26T21:59:58.633771+0000","last_fullsized":"2024-04-26T21:59:58.633771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638642+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T21:59:58.638642+0000","last_peered":"2024-04-26T21:59:58.638642+0000","last_clean":"2024-04-26T21:59:58.638642+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T21:59:58.638642+0000","last_undegraded":"2024-04-26T21:59:58.638642+0000","last_fullsized":"2024-04-26T21:59:58.638642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637822+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T21:59:58.637822+0000","last_peered":"2024-04-26T21:59:58.637822+0000","last_clean":"2024-04-26T21:59:58.637822+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T21:59:58.637822+0000","last_undegraded":"2024-04-26T21:59:58.637822+0000","last_fullsized":"2024-04-26T21:59:58.637822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016229+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T21:59:59.016229+0000","last_peered":"2024-04-26T21:59:59.016229+0000","last_clean":"2024-04-26T21:59:59.016229+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T21:59:59.016229+0000","last_undegraded":"2024-04-26T21:59:59.016229+0000","last_fullsized":"2024-04-26T21:59:59.016229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.662840+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T21:59:57.662840+0000","last_peered":"2024-04-26T21:59:57.662840+0000","last_clean":"2024-04-26T21:59:57.662840+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T21:59:57.662840+0000","last_undegraded":"2024-04-26T21:59:57.662840+0000","last_fullsized":"2024-04-26T21:59:57.662840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654943+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T21:59:57.654943+0000","last_peered":"2024-04-26T21:59:57.654943+0000","last_clean":"2024-04-26T21:59:57.654943+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T21:59:57.654943+0000","last_undegraded":"2024-04-26T21:59:57.654943+0000","last_fullsized":"2024-04-26T21:59:57.654943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315652+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T21:59:59.315652+0000","last_peered":"2024-04-26T21:59:59.315652+0000","last_clean":"2024-04-26T21:59:59.315652+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T21:59:59.315652+0000","last_undegraded":"2024-04-26T21:59:59.315652+0000","last_fullsized":"2024-04-26T21:59:59.315652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016055+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T21:59:59.016055+0000","last_peered":"2024-04-26T21:59:59.016055+0000","last_clean":"2024-04-26T21:59:59.016055+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T21:59:59.016055+0000","last_undegraded":"2024-04-26T21:59:59.016055+0000","last_fullsized":"2024-04-26T21:59:59.016055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637503+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T21:59:58.637503+0000","last_peered":"2024-04-26T21:59:58.637503+0000","last_clean":"2024-04-26T21:59:58.637503+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T21:59:58.637503+0000","last_undegraded":"2024-04-26T21:59:58.637503+0000","last_fullsized":"2024-04-26T21:59:58.637503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637408+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T21:59:58.637408+0000","last_peered":"2024-04-26T21:59:58.637408+0000","last_clean":"2024-04-26T21:59:58.637408+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T21:59:58.637408+0000","last_undegraded":"2024-04-26T21:59:58.637408+0000","last_fullsized":"2024-04-26T21:59:58.637408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.662460+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T21:59:57.662460+0000","last_peered":"2024-04-26T21:59:57.662460+0000","last_clean":"2024-04-26T21:59:57.662460+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T21:59:57.662460+0000","last_undegraded":"2024-04-26T21:59:57.662460+0000","last_fullsized":"2024-04-26T21:59:57.662460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015997+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T21:59:59.015997+0000","last_peered":"2024-04-26T21:59:59.015997+0000","last_clean":"2024-04-26T21:59:59.015997+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T21:59:59.015997+0000","last_undegraded":"2024-04-26T21:59:59.015997+0000","last_fullsized":"2024-04-26T21:59:59.015997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638686+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T21:59:58.638686+0000","last_peered":"2024-04-26T21:59:58.638686+0000","last_clean":"2024-04-26T21:59:58.638686+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T21:59:58.638686+0000","last_undegraded":"2024-04-26T21:59:58.638686+0000","last_fullsized":"2024-04-26T21:59:58.638686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016362+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T21:59:59.016362+0000","last_peered":"2024-04-26T21:59:59.016362+0000","last_clean":"2024-04-26T21:59:59.016362+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T21:59:59.016362+0000","last_undegraded":"2024-04-26T21:59:59.016362+0000","last_fullsized":"2024-04-26T21:59:59.016362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.661744+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T21:59:57.661744+0000","last_peered":"2024-04-26T21:59:57.661744+0000","last_clean":"2024-04-26T21:59:57.661744+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T21:59:57.661744+0000","last_undegraded":"2024-04-26T21:59:57.661744+0000","last_fullsized":"2024-04-26T21:59:57.661744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016578+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T21:59:59.016578+0000","last_peered":"2024-04-26T21:59:59.016578+0000","last_clean":"2024-04-26T21:59:59.016578+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T21:59:59.016578+0000","last_undegraded":"2024-04-26T21:59:59.016578+0000","last_fullsized":"2024-04-26T21:59:59.016578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633871+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T21:59:58.633871+0000","last_peered":"2024-04-26T21:59:58.633871+0000","last_clean":"2024-04-26T21:59:58.633871+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T21:59:58.633871+0000","last_undegraded":"2024-04-26T21:59:58.633871+0000","last_fullsized":"2024-04-26T21:59:58.633871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315722+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T21:59:59.315722+0000","last_peered":"2024-04-26T21:59:59.315722+0000","last_clean":"2024-04-26T21:59:59.315722+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T21:59:59.315722+0000","last_undegraded":"2024-04-26T21:59:59.315722+0000","last_fullsized":"2024-04-26T21:59:59.315722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.643052+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T21:59:57.643052+0000","last_peered":"2024-04-26T21:59:57.643052+0000","last_clean":"2024-04-26T21:59:57.643052+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T21:59:57.643052+0000","last_undegraded":"2024-04-26T21:59:57.643052+0000","last_fullsized":"2024-04-26T21:59:57.643052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016208+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T21:59:59.016208+0000","last_peered":"2024-04-26T21:59:59.016208+0000","last_clean":"2024-04-26T21:59:59.016208+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T21:59:59.016208+0000","last_undegraded":"2024-04-26T21:59:59.016208+0000","last_fullsized":"2024-04-26T21:59:59.016208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015860+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T21:59:59.015860+0000","last_peered":"2024-04-26T21:59:59.015860+0000","last_clean":"2024-04-26T21:59:59.015860+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T21:59:59.015860+0000","last_undegraded":"2024-04-26T21:59:59.015860+0000","last_fullsized":"2024-04-26T21:59:59.015860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315421+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T21:59:59.315421+0000","last_peered":"2024-04-26T21:59:59.315421+0000","last_clean":"2024-04-26T21:59:59.315421+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T21:59:59.315421+0000","last_undegraded":"2024-04-26T21:59:59.315421+0000","last_fullsized":"2024-04-26T21:59:59.315421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707695+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T21:59:59.707695+0000","last_peered":"2024-04-26T21:59:59.707695+0000","last_clean":"2024-04-26T21:59:59.707695+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T21:59:59.707695+0000","last_undegraded":"2024-04-26T21:59:59.707695+0000","last_fullsized":"2024-04-26T21:59:59.707695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015763+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T21:59:59.015763+0000","last_peered":"2024-04-26T21:59:59.015763+0000","last_clean":"2024-04-26T21:59:59.015763+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T21:59:59.015763+0000","last_undegraded":"2024-04-26T21:59:59.015763+0000","last_fullsized":"2024-04-26T21:59:59.015763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656036+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T21:59:57.656036+0000","last_peered":"2024-04-26T21:59:57.656036+0000","last_clean":"2024-04-26T21:59:57.656036+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T21:59:57.656036+0000","last_undegraded":"2024-04-26T21:59:57.656036+0000","last_fullsized":"2024-04-26T21:59:57.656036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:11.186506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.642317+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T21:59:57.642317+0000","last_peered":"2024-04-26T21:59:57.642317+0000","last_clean":"2024-04-26T21:59:57.642317+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T21:59:57.642317+0000","last_undegraded":"2024-04-26T21:59:57.642317+0000","last_fullsized":"2024-04-26T21:59:57.642317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633705+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T21:59:58.633705+0000","last_peered":"2024-04-26T21:59:58.633705+0000","last_clean":"2024-04-26T21:59:58.633705+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T21:59:58.633705+0000","last_undegraded":"2024-04-26T21:59:58.633705+0000","last_fullsized":"2024-04-26T21:59:58.633705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315803+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T21:59:59.315803+0000","last_peered":"2024-04-26T21:59:59.315803+0000","last_clean":"2024-04-26T21:59:59.315803+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T21:59:59.315803+0000","last_undegraded":"2024-04-26T21:59:59.315803+0000","last_fullsized":"2024-04-26T21:59:59.315803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637923+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T21:59:58.637923+0000","last_peered":"2024-04-26T21:59:58.637923+0000","last_clean":"2024-04-26T21:59:58.637923+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T21:59:58.637923+0000","last_undegraded":"2024-04-26T21:59:58.637923+0000","last_fullsized":"2024-04-26T21:59:58.637923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633980+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T21:59:58.633980+0000","last_peered":"2024-04-26T21:59:58.633980+0000","last_clean":"2024-04-26T21:59:58.633980+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T21:59:58.633980+0000","last_undegraded":"2024-04-26T21:59:58.633980+0000","last_fullsized":"2024-04-26T21:59:58.633980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315934+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T21:59:59.315934+0000","last_peered":"2024-04-26T21:59:59.315934+0000","last_clean":"2024-04-26T21:59:59.315934+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T21:59:59.315934+0000","last_undegraded":"2024-04-26T21:59:59.315934+0000","last_fullsized":"2024-04-26T21:59:59.315934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016420+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T21:59:59.016420+0000","last_peered":"2024-04-26T21:59:59.016420+0000","last_clean":"2024-04-26T21:59:59.016420+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T21:59:59.016420+0000","last_undegraded":"2024-04-26T21:59:59.016420+0000","last_fullsized":"2024-04-26T21:59:59.016420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633894+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T21:59:58.633894+0000","last_peered":"2024-04-26T21:59:58.633894+0000","last_clean":"2024-04-26T21:59:58.633894+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T21:59:58.633894+0000","last_undegraded":"2024-04-26T21:59:58.633894+0000","last_fullsized":"2024-04-26T21:59:58.633894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.649584+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T21:59:58.649584+0000","last_peered":"2024-04-26T21:59:58.649584+0000","last_clean":"2024-04-26T21:59:58.649584+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T21:59:58.649584+0000","last_undegraded":"2024-04-26T21:59:58.649584+0000","last_fullsized":"2024-04-26T21:59:58.649584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708073+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T21:59:59.708073+0000","last_peered":"2024-04-26T21:59:59.708073+0000","last_clean":"2024-04-26T21:59:59.708073+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T21:59:59.708073+0000","last_undegraded":"2024-04-26T21:59:59.708073+0000","last_fullsized":"2024-04-26T21:59:59.708073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637980+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T21:59:58.637980+0000","last_peered":"2024-04-26T21:59:58.637980+0000","last_clean":"2024-04-26T21:59:58.637980+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T21:59:58.637980+0000","last_undegraded":"2024-04-26T21:59:58.637980+0000","last_fullsized":"2024-04-26T21:59:58.637980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654831+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T21:59:57.654831+0000","last_peered":"2024-04-26T21:59:57.654831+0000","last_clean":"2024-04-26T21:59:57.654831+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T21:59:57.654831+0000","last_undegraded":"2024-04-26T21:59:57.654831+0000","last_fullsized":"2024-04-26T21:59:57.654831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:14.481004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015759+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T21:59:59.015759+0000","last_peered":"2024-04-26T21:59:59.015759+0000","last_clean":"2024-04-26T21:59:59.015759+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T21:59:59.015759+0000","last_undegraded":"2024-04-26T21:59:59.015759+0000","last_fullsized":"2024-04-26T21:59:59.015759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638723+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T21:59:58.638723+0000","last_peered":"2024-04-26T21:59:58.638723+0000","last_clean":"2024-04-26T21:59:58.638723+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T21:59:58.638723+0000","last_undegraded":"2024-04-26T21:59:58.638723+0000","last_fullsized":"2024-04-26T21:59:58.638723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633795+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T21:59:58.633795+0000","last_peered":"2024-04-26T21:59:58.633795+0000","last_clean":"2024-04-26T21:59:58.633795+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T21:59:58.633795+0000","last_undegraded":"2024-04-26T21:59:58.633795+0000","last_fullsized":"2024-04-26T21:59:58.633795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707799+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T21:59:59.707799+0000","last_peered":"2024-04-26T21:59:59.707799+0000","last_clean":"2024-04-26T21:59:59.707799+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T21:59:59.707799+0000","last_undegraded":"2024-04-26T21:59:59.707799+0000","last_fullsized":"2024-04-26T21:59:59.707799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633805+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T21:59:58.633805+0000","last_peered":"2024-04-26T21:59:58.633805+0000","last_clean":"2024-04-26T21:59:58.633805+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T21:59:58.633805+0000","last_undegraded":"2024-04-26T21:59:58.633805+0000","last_fullsized":"2024-04-26T21:59:58.633805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707766+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T21:59:59.707766+0000","last_peered":"2024-04-26T21:59:59.707766+0000","last_clean":"2024-04-26T21:59:59.707766+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T21:59:59.707766+0000","last_undegraded":"2024-04-26T21:59:59.707766+0000","last_fullsized":"2024-04-26T21:59:59.707766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016157+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T21:59:59.016157+0000","last_peered":"2024-04-26T21:59:59.016157+0000","last_clean":"2024-04-26T21:59:59.016157+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T21:59:59.016157+0000","last_undegraded":"2024-04-26T21:59:59.016157+0000","last_fullsized":"2024-04-26T21:59:59.016157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638174+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T21:59:58.638174+0000","last_peered":"2024-04-26T21:59:58.638174+0000","last_clean":"2024-04-26T21:59:58.638174+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T21:59:58.638174+0000","last_undegraded":"2024-04-26T21:59:58.638174+0000","last_fullsized":"2024-04-26T21:59:58.638174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315965+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T21:59:59.315965+0000","last_peered":"2024-04-26T21:59:59.315965+0000","last_clean":"2024-04-26T21:59:59.315965+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T21:59:59.315965+0000","last_undegraded":"2024-04-26T21:59:59.315965+0000","last_fullsized":"2024-04-26T21:59:59.315965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016148+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T21:59:59.016148+0000","last_peered":"2024-04-26T21:59:59.016148+0000","last_clean":"2024-04-26T21:59:59.016148+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T21:59:59.016148+0000","last_undegraded":"2024-04-26T21:59:59.016148+0000","last_fullsized":"2024-04-26T21:59:59.016148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315569+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T21:59:59.315569+0000","last_peered":"2024-04-26T21:59:59.315569+0000","last_clean":"2024-04-26T21:59:59.315569+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T21:59:59.315569+0000","last_undegraded":"2024-04-26T21:59:59.315569+0000","last_fullsized":"2024-04-26T21:59:59.315569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016343+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T21:59:59.016343+0000","last_peered":"2024-04-26T21:59:59.016343+0000","last_clean":"2024-04-26T21:59:59.016343+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T21:59:59.016343+0000","last_undegraded":"2024-04-26T21:59:59.016343+0000","last_fullsized":"2024-04-26T21:59:59.016343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.655832+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T21:59:57.655832+0000","last_peered":"2024-04-26T21:59:57.655832+0000","last_clean":"2024-04-26T21:59:57.655832+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T21:59:57.655832+0000","last_undegraded":"2024-04-26T21:59:57.655832+0000","last_fullsized":"2024-04-26T21:59:57.655832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708025+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T21:59:59.708025+0000","last_peered":"2024-04-26T21:59:59.708025+0000","last_clean":"2024-04-26T21:59:59.708025+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T21:59:59.708025+0000","last_undegraded":"2024-04-26T21:59:59.708025+0000","last_fullsized":"2024-04-26T21:59:59.708025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708004+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T21:59:59.708004+0000","last_peered":"2024-04-26T21:59:59.708004+0000","last_clean":"2024-04-26T21:59:59.708004+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T21:59:59.708004+0000","last_undegraded":"2024-04-26T21:59:59.708004+0000","last_fullsized":"2024-04-26T21:59:59.708004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.664743+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T21:59:57.664743+0000","last_peered":"2024-04-26T21:59:57.664743+0000","last_clean":"2024-04-26T21:59:57.664743+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T21:59:57.664743+0000","last_undegraded":"2024-04-26T21:59:57.664743+0000","last_fullsized":"2024-04-26T21:59:57.664743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:01.736 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:01.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.869+0000 7fbd12c0f640 1 Processor -- start 2024-04-26T22:00:01.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.869+0000 7fbd12c0f640 1 -- start start 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd12c0f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbd0c0740e0 0x7fbd0c157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd12c0f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd0c15a530 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd12c0f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd0c15a810 con 0x7fbd0c0c6e20 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd12c0f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbd0c072930 con 0x7fbd0c0740e0 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd0bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbd0c0740e0 0x7fbd0c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd111ac640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbd0c0ca3a0 0x7fbd0c161450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38936/0 (socket says 172.21.15.135:38936) 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd0bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbd0c0740e0 0x7fbd0c157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46336/0 (socket says 172.21.15.135:46336) 2024-04-26T22:00:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd111ac640 1 -- 172.21.15.135:0/3672926701 learned_addr learned my addr 172.21.15.135:0/3672926701 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:01.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd0bfff640 1 -- 172.21.15.135:0/3672926701 >> v1:172.21.15.139:6789/0 conn(0x7fbd0c0c6e20 legacy=0x7fbd0c156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd0bfff640 1 -- 172.21.15.135:0/3672926701 >> v1:172.21.15.135:6789/0 conn(0x7fbd0c0ca3a0 legacy=0x7fbd0c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd0bfff640 1 -- 172.21.15.135:0/3672926701 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c15aaf0 con 0x7fbd0c0740e0 2024-04-26T22:00:01.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.870+0000 7fbd0b7fe640 1 -- 172.21.15.135:0/3672926701 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527507786 0 0) 0x7fbd0c15a530 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd0bfff640 1 --2- 172.21.15.135:0/3672926701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0740e0 0x7fbd0c157630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80097a0 tx=0x7fbcf802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbee6b9bf42ae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd0b7fe640 1 -- 172.21.15.135:0/3672926701 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8037070 con 0x7fbd0c0740e0 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd0b7fe640 1 -- 172.21.15.135:0/3672926701 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf802ae50 con 0x7fbd0c0740e0 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd0b7fe640 1 -- 172.21.15.135:0/3672926701 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8032660 con 0x7fbd0c0740e0 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 -- 172.21.15.135:0/3672926701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0740e0 msgr2=0x7fbd0c157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/3672926701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0740e0 0x7fbd0c157630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80097a0 tx=0x7fbcf802a140 comp rx=0 tx=0).stop 2024-04-26T22:00:01.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 -- 172.21.15.135:0/3672926701 shutdown_connections 2024-04-26T22:00:01.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/3672926701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0740e0 0x7fbd0c157630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 -- 172.21.15.135:0/3672926701 >> 172.21.15.135:0/3672926701 conn(0x7fbd0c0ae0b0 msgr2=0x7fbd0c0ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 -- 172.21.15.135:0/3672926701 shutdown_connections 2024-04-26T22:00:01.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.871+0000 7fbd12c0f640 1 -- 172.21.15.135:0/3672926701 wait complete. 2024-04-26T22:00:01.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.872+0000 7fbd12c0f640 1 Processor -- start 2024-04-26T22:00:01.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 -- start start 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0c0740e0 0x7fbd0c143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd0c0c6e20 0x7fbd0c1ec780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 0x7fbd0c1eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c154540 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1543c0 con 0x7fbd0c0c6e20 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd12c0f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0c15a0c0 con 0x7fbd0c0740e0 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd111ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 0x7fbd0c1eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd111ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 0x7fbd0c1eccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46342/0 (socket says 172.21.15.135:46342) 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.873+0000 7fbd111ac640 1 -- 172.21.15.135:0/2187444493 learned_addr learned my addr 172.21.15.135:0/2187444493 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd111ac640 1 -- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0c0740e0 msgr2=0x7fbd0c143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd0bfff640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd0c0c6e20 0x7fbd0c1ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd109ab640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0c0740e0 0x7fbd0c143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd111ac640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0c0740e0 0x7fbd0c143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd111ac640 1 -- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd0c0c6e20 msgr2=0x7fbd0c1ec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd111ac640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd0c0c6e20 0x7fbd0c1ec780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd111ac640 1 -- 172.21.15.135:0/2187444493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c0c14c0 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd111ac640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 0x7fbd0c1eccc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c7a0 tx=0x7fbd0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04018070 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0400a640 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c0c17a0 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400aab0 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.874+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c143780 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.875+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c0766c0 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.875+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd0401b070 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.876+0000 7fbd097fa640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbcf006c8f0 0x7fbcf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.876+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fbd0408b100 con 0x7fbd0c0ca3a0 2024-04-26T22:00:01.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.876+0000 7fbd109ab640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbcf006c8f0 0x7fbcf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.876+0000 7fbd109ab640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbcf006c8f0 0x7fbcf006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000a750 tx=0x7fbd00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:01.880+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd040595b0 con 0x7fbd0c0ca3a0 2024-04-26T22:00:02.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.055+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd0c0507a0 con 0x7fbcf006c8f0 2024-04-26T22:00:02.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.061+0000 7fbd097fa640 1 -- 172.21.15.135:0/2187444493 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fbd0c0507a0 con 0x7fbcf006c8f0 2024-04-26T22:00:02.064 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:02.065 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:02.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbcf006c8f0 msgr2=0x7fbcf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbcf006c8f0 0x7fbcf006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000a750 tx=0x7fbd00008040 comp rx=0 tx=0).stop 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 msgr2=0x7fbd0c1eccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 0x7fbd0c1eccc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c7a0 tx=0x7fbd0400cc60 comp rx=0 tx=0).stop 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 shutdown_connections 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbcf006c8f0 0x7fbcf006edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd0c0ca3a0 0x7fbd0c1eccc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd0c0c6e20 0x7fbd0c1ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 --2- 172.21.15.135:0/2187444493 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0c0740e0 0x7fbd0c143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 >> 172.21.15.135:0/2187444493 conn(0x7fbd0c0ae0b0 msgr2=0x7fbd0c0ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 shutdown_connections 2024-04-26T22:00:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.064+0000 7fbd12c0f640 1 -- 172.21.15.135:0/2187444493 wait complete. 2024-04-26T22:00:02.075 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T22:00:01.069764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2854912,"data_stored":1733563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.446871"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016402+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T21:59:59.016402+0000","last_peered":"2024-04-26T21:59:59.016402+0000","last_clean":"2024-04-26T21:59:59.016402+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T21:59:59.016402+0000","last_undegraded":"2024-04-26T21:59:59.016402+0000","last_fullsized":"2024-04-26T21:59:59.016402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.634003+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T21:59:58.634003+0000","last_peered":"2024-04-26T21:59:58.634003+0000","last_clean":"2024-04-26T21:59:58.634003+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T21:59:58.634003+0000","last_undegraded":"2024-04-26T21:59:58.634003+0000","last_fullsized":"2024-04-26T21:59:58.634003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656155+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T21:59:57.656155+0000","last_peered":"2024-04-26T21:59:57.656155+0000","last_clean":"2024-04-26T21:59:57.656155+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T21:59:57.656155+0000","last_undegraded":"2024-04-26T21:59:57.656155+0000","last_fullsized":"2024-04-26T21:59:57.656155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315533+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T21:59:59.315533+0000","last_peered":"2024-04-26T21:59:59.315533+0000","last_clean":"2024-04-26T21:59:59.315533+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T21:59:59.315533+0000","last_undegraded":"2024-04-26T21:59:59.315533+0000","last_fullsized":"2024-04-26T21:59:59.315533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016022+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T21:59:59.016022+0000","last_peered":"2024-04-26T21:59:59.016022+0000","last_clean":"2024-04-26T21:59:59.016022+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T21:59:59.016022+0000","last_undegraded":"2024-04-26T21:59:59.016022+0000","last_fullsized":"2024-04-26T21:59:59.016022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315482+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T21:59:59.315482+0000","last_peered":"2024-04-26T21:59:59.315482+0000","last_clean":"2024-04-26T21:59:59.315482+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T21:59:59.315482+0000","last_undegraded":"2024-04-26T21:59:59.315482+0000","last_fullsized":"2024-04-26T21:59:59.315482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015951+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T21:59:59.015951+0000","last_peered":"2024-04-26T21:59:59.015951+0000","last_clean":"2024-04-26T21:59:59.015951+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T21:59:59.015951+0000","last_undegraded":"2024-04-26T21:59:59.015951+0000","last_fullsized":"2024-04-26T21:59:59.015951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707463+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T21:59:59.707463+0000","last_peered":"2024-04-26T21:59:59.707463+0000","last_clean":"2024-04-26T21:59:59.707463+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T21:59:59.707463+0000","last_undegraded":"2024-04-26T21:59:59.707463+0000","last_fullsized":"2024-04-26T21:59:59.707463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016415+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T21:59:59.016415+0000","last_peered":"2024-04-26T21:59:59.016415+0000","last_clean":"2024-04-26T21:59:59.016415+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T21:59:59.016415+0000","last_undegraded":"2024-04-26T21:59:59.016415+0000","last_fullsized":"2024-04-26T21:59:59.016415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638047+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T21:59:58.638047+0000","last_peered":"2024-04-26T21:59:58.638047+0000","last_clean":"2024-04-26T21:59:58.638047+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T21:59:58.638047+0000","last_undegraded":"2024-04-26T21:59:58.638047+0000","last_fullsized":"2024-04-26T21:59:58.638047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637658+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T21:59:58.637658+0000","last_peered":"2024-04-26T21:59:58.637658+0000","last_clean":"2024-04-26T21:59:58.637658+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T21:59:58.637658+0000","last_undegraded":"2024-04-26T21:59:58.637658+0000","last_fullsized":"2024-04-26T21:59:58.637658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638530+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T21:59:58.638530+0000","last_peered":"2024-04-26T21:59:58.638530+0000","last_clean":"2024-04-26T21:59:58.638530+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T21:59:58.638530+0000","last_undegraded":"2024-04-26T21:59:58.638530+0000","last_fullsized":"2024-04-26T21:59:58.638530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707864+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T21:59:59.707864+0000","last_peered":"2024-04-26T21:59:59.707864+0000","last_clean":"2024-04-26T21:59:59.707864+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T21:59:59.707864+0000","last_undegraded":"2024-04-26T21:59:59.707864+0000","last_fullsized":"2024-04-26T21:59:59.707864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654923+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T21:59:57.654923+0000","last_peered":"2024-04-26T21:59:57.654923+0000","last_clean":"2024-04-26T21:59:57.654923+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T21:59:57.654923+0000","last_undegraded":"2024-04-26T21:59:57.654923+0000","last_fullsized":"2024-04-26T21:59:57.654923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:21.311477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633859+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T21:59:58.633859+0000","last_peered":"2024-04-26T21:59:58.633859+0000","last_clean":"2024-04-26T21:59:58.633859+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T21:59:58.633859+0000","last_undegraded":"2024-04-26T21:59:58.633859+0000","last_fullsized":"2024-04-26T21:59:58.633859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315337+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T21:59:59.315337+0000","last_peered":"2024-04-26T21:59:59.315337+0000","last_clean":"2024-04-26T21:59:59.315337+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T21:59:59.315337+0000","last_undegraded":"2024-04-26T21:59:59.315337+0000","last_fullsized":"2024-04-26T21:59:59.315337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.316014+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T21:59:59.316014+0000","last_peered":"2024-04-26T21:59:59.316014+0000","last_clean":"2024-04-26T21:59:59.316014+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T21:59:59.316014+0000","last_undegraded":"2024-04-26T21:59:59.316014+0000","last_fullsized":"2024-04-26T21:59:59.316014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654666+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T21:59:57.654666+0000","last_peered":"2024-04-26T21:59:57.654666+0000","last_clean":"2024-04-26T21:59:57.654666+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T21:59:57.654666+0000","last_undegraded":"2024-04-26T21:59:57.654666+0000","last_fullsized":"2024-04-26T21:59:57.654666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016427+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T21:59:59.016427+0000","last_peered":"2024-04-26T21:59:59.016427+0000","last_clean":"2024-04-26T21:59:59.016427+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T21:59:59.016427+0000","last_undegraded":"2024-04-26T21:59:59.016427+0000","last_fullsized":"2024-04-26T21:59:59.016427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637715+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T21:59:58.637715+0000","last_peered":"2024-04-26T21:59:58.637715+0000","last_clean":"2024-04-26T21:59:58.637715+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T21:59:58.637715+0000","last_undegraded":"2024-04-26T21:59:58.637715+0000","last_fullsized":"2024-04-26T21:59:58.637715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656154+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T21:59:57.656154+0000","last_peered":"2024-04-26T21:59:57.656154+0000","last_clean":"2024-04-26T21:59:57.656154+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T21:59:57.656154+0000","last_undegraded":"2024-04-26T21:59:57.656154+0000","last_fullsized":"2024-04-26T21:59:57.656154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315530+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T21:59:59.315530+0000","last_peered":"2024-04-26T21:59:59.315530+0000","last_clean":"2024-04-26T21:59:59.315530+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T21:59:59.315530+0000","last_undegraded":"2024-04-26T21:59:59.315530+0000","last_fullsized":"2024-04-26T21:59:59.315530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707605+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T21:59:59.707605+0000","last_peered":"2024-04-26T21:59:59.707605+0000","last_clean":"2024-04-26T21:59:59.707605+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T21:59:59.707605+0000","last_undegraded":"2024-04-26T21:59:59.707605+0000","last_fullsized":"2024-04-26T21:59:59.707605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315406+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T21:59:59.315406+0000","last_peered":"2024-04-26T21:59:59.315406+0000","last_clean":"2024-04-26T21:59:59.315406+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T21:59:59.315406+0000","last_undegraded":"2024-04-26T21:59:59.315406+0000","last_fullsized":"2024-04-26T21:59:59.315406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708018+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T21:59:59.708018+0000","last_peered":"2024-04-26T21:59:59.708018+0000","last_clean":"2024-04-26T21:59:59.708018+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T21:59:59.708018+0000","last_undegraded":"2024-04-26T21:59:59.708018+0000","last_fullsized":"2024-04-26T21:59:59.708018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638216+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T21:59:58.638216+0000","last_peered":"2024-04-26T21:59:58.638216+0000","last_clean":"2024-04-26T21:59:58.638216+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T21:59:58.638216+0000","last_undegraded":"2024-04-26T21:59:58.638216+0000","last_fullsized":"2024-04-26T21:59:58.638216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633789+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T21:59:58.633789+0000","last_peered":"2024-04-26T21:59:58.633789+0000","last_clean":"2024-04-26T21:59:58.633789+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T21:59:58.633789+0000","last_undegraded":"2024-04-26T21:59:58.633789+0000","last_fullsized":"2024-04-26T21:59:58.633789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707880+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T21:59:59.707880+0000","last_peered":"2024-04-26T21:59:59.707880+0000","last_clean":"2024-04-26T21:59:59.707880+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T21:59:59.707880+0000","last_undegraded":"2024-04-26T21:59:59.707880+0000","last_fullsized":"2024-04-26T21:59:59.707880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T21:59:57.656134+0000","last_peered":"2024-04-26T21:59:57.656134+0000","last_clean":"2024-04-26T21:59:57.656134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T21:59:57.656134+0000","last_undegraded":"2024-04-26T21:59:57.656134+0000","last_fullsized":"2024-04-26T21:59:57.656134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315660+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T21:59:59.315660+0000","last_peered":"2024-04-26T21:59:59.315660+0000","last_clean":"2024-04-26T21:59:59.315660+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T21:59:59.315660+0000","last_undegraded":"2024-04-26T21:59:59.315660+0000","last_fullsized":"2024-04-26T21:59:59.315660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.659987+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T21:59:57.659987+0000","last_peered":"2024-04-26T21:59:57.659987+0000","last_clean":"2024-04-26T21:59:57.659987+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T21:59:57.659987+0000","last_undegraded":"2024-04-26T21:59:57.659987+0000","last_fullsized":"2024-04-26T21:59:57.659987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637363+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T21:59:58.637363+0000","last_peered":"2024-04-26T21:59:58.637363+0000","last_clean":"2024-04-26T21:59:58.637363+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T21:59:58.637363+0000","last_undegraded":"2024-04-26T21:59:58.637363+0000","last_fullsized":"2024-04-26T21:59:58.637363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637591+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T21:59:58.637591+0000","last_peered":"2024-04-26T21:59:58.637591+0000","last_clean":"2024-04-26T21:59:58.637591+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T21:59:58.637591+0000","last_undegraded":"2024-04-26T21:59:58.637591+0000","last_fullsized":"2024-04-26T21:59:58.637591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707915+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T21:59:59.707915+0000","last_peered":"2024-04-26T21:59:59.707915+0000","last_clean":"2024-04-26T21:59:59.707915+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T21:59:59.707915+0000","last_undegraded":"2024-04-26T21:59:59.707915+0000","last_fullsized":"2024-04-26T21:59:59.707915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656112+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T21:59:57.656112+0000","last_peered":"2024-04-26T21:59:57.656112+0000","last_clean":"2024-04-26T21:59:57.656112+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T21:59:57.656112+0000","last_undegraded":"2024-04-26T21:59:57.656112+0000","last_fullsized":"2024-04-26T21:59:57.656112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:58.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638288+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T21:59:58.638288+0000","last_peered":"2024-04-26T21:59:58.638288+0000","last_clean":"2024-04-26T21:59:58.638288+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T21:59:58.638288+0000","last_undegraded":"2024-04-26T21:59:58.638288+0000","last_fullsized":"2024-04-26T21:59:58.638288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707700+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T21:59:59.707700+0000","last_peered":"2024-04-26T21:59:59.707700+0000","last_clean":"2024-04-26T21:59:59.707700+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T21:59:59.707700+0000","last_undegraded":"2024-04-26T21:59:59.707700+0000","last_fullsized":"2024-04-26T21:59:59.707700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707594+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T21:59:59.707594+0000","last_peered":"2024-04-26T21:59:59.707594+0000","last_clean":"2024-04-26T21:59:59.707594+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T21:59:59.707594+0000","last_undegraded":"2024-04-26T21:59:59.707594+0000","last_fullsized":"2024-04-26T21:59:59.707594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315502+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T21:59:59.315502+0000","last_peered":"2024-04-26T21:59:59.315502+0000","last_clean":"2024-04-26T21:59:59.315502+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T21:59:59.315502+0000","last_undegraded":"2024-04-26T21:59:59.315502+0000","last_fullsized":"2024-04-26T21:59:59.315502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633698+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T21:59:58.633698+0000","last_peered":"2024-04-26T21:59:58.633698+0000","last_clean":"2024-04-26T21:59:58.633698+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T21:59:58.633698+0000","last_undegraded":"2024-04-26T21:59:58.633698+0000","last_fullsized":"2024-04-26T21:59:58.633698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637863+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T21:59:58.637863+0000","last_peered":"2024-04-26T21:59:58.637863+0000","last_clean":"2024-04-26T21:59:58.637863+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T21:59:58.637863+0000","last_undegraded":"2024-04-26T21:59:58.637863+0000","last_fullsized":"2024-04-26T21:59:58.637863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315907+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T21:59:59.315907+0000","last_peered":"2024-04-26T21:59:59.315907+0000","last_clean":"2024-04-26T21:59:59.315907+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T21:59:59.315907+0000","last_undegraded":"2024-04-26T21:59:59.315907+0000","last_fullsized":"2024-04-26T21:59:59.315907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315932+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T21:59:59.315932+0000","last_peered":"2024-04-26T21:59:59.315932+0000","last_clean":"2024-04-26T21:59:59.315932+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T21:59:59.315932+0000","last_undegraded":"2024-04-26T21:59:59.315932+0000","last_fullsized":"2024-04-26T21:59:59.315932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.655796+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T21:59:57.655796+0000","last_peered":"2024-04-26T21:59:57.655796+0000","last_clean":"2024-04-26T21:59:57.655796+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T21:59:57.655796+0000","last_undegraded":"2024-04-26T21:59:57.655796+0000","last_fullsized":"2024-04-26T21:59:57.655796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633771+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T21:59:58.633771+0000","last_peered":"2024-04-26T21:59:58.633771+0000","last_clean":"2024-04-26T21:59:58.633771+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T21:59:58.633771+0000","last_undegraded":"2024-04-26T21:59:58.633771+0000","last_fullsized":"2024-04-26T21:59:58.633771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638642+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T21:59:58.638642+0000","last_peered":"2024-04-26T21:59:58.638642+0000","last_clean":"2024-04-26T21:59:58.638642+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T21:59:58.638642+0000","last_undegraded":"2024-04-26T21:59:58.638642+0000","last_fullsized":"2024-04-26T21:59:58.638642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637822+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T21:59:58.637822+0000","last_peered":"2024-04-26T21:59:58.637822+0000","last_clean":"2024-04-26T21:59:58.637822+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T21:59:58.637822+0000","last_undegraded":"2024-04-26T21:59:58.637822+0000","last_fullsized":"2024-04-26T21:59:58.637822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016229+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T21:59:59.016229+0000","last_peered":"2024-04-26T21:59:59.016229+0000","last_clean":"2024-04-26T21:59:59.016229+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T21:59:59.016229+0000","last_undegraded":"2024-04-26T21:59:59.016229+0000","last_fullsized":"2024-04-26T21:59:59.016229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.662840+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T21:59:57.662840+0000","last_peered":"2024-04-26T21:59:57.662840+0000","last_clean":"2024-04-26T21:59:57.662840+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T21:59:57.662840+0000","last_undegraded":"2024-04-26T21:59:57.662840+0000","last_fullsized":"2024-04-26T21:59:57.662840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654943+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T21:59:57.654943+0000","last_peered":"2024-04-26T21:59:57.654943+0000","last_clean":"2024-04-26T21:59:57.654943+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T21:59:57.654943+0000","last_undegraded":"2024-04-26T21:59:57.654943+0000","last_fullsized":"2024-04-26T21:59:57.654943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315652+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T21:59:59.315652+0000","last_peered":"2024-04-26T21:59:59.315652+0000","last_clean":"2024-04-26T21:59:59.315652+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T21:59:59.315652+0000","last_undegraded":"2024-04-26T21:59:59.315652+0000","last_fullsized":"2024-04-26T21:59:59.315652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016055+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T21:59:59.016055+0000","last_peered":"2024-04-26T21:59:59.016055+0000","last_clean":"2024-04-26T21:59:59.016055+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T21:59:59.016055+0000","last_undegraded":"2024-04-26T21:59:59.016055+0000","last_fullsized":"2024-04-26T21:59:59.016055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637503+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T21:59:58.637503+0000","last_peered":"2024-04-26T21:59:58.637503+0000","last_clean":"2024-04-26T21:59:58.637503+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T21:59:58.637503+0000","last_undegraded":"2024-04-26T21:59:58.637503+0000","last_fullsized":"2024-04-26T21:59:58.637503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637408+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T21:59:58.637408+0000","last_peered":"2024-04-26T21:59:58.637408+0000","last_clean":"2024-04-26T21:59:58.637408+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T21:59:58.637408+0000","last_undegraded":"2024-04-26T21:59:58.637408+0000","last_fullsized":"2024-04-26T21:59:58.637408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.662460+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T21:59:57.662460+0000","last_peered":"2024-04-26T21:59:57.662460+0000","last_clean":"2024-04-26T21:59:57.662460+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T21:59:57.662460+0000","last_undegraded":"2024-04-26T21:59:57.662460+0000","last_fullsized":"2024-04-26T21:59:57.662460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015997+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T21:59:59.015997+0000","last_peered":"2024-04-26T21:59:59.015997+0000","last_clean":"2024-04-26T21:59:59.015997+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T21:59:59.015997+0000","last_undegraded":"2024-04-26T21:59:59.015997+0000","last_fullsized":"2024-04-26T21:59:59.015997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638686+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T21:59:58.638686+0000","last_peered":"2024-04-26T21:59:58.638686+0000","last_clean":"2024-04-26T21:59:58.638686+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T21:59:58.638686+0000","last_undegraded":"2024-04-26T21:59:58.638686+0000","last_fullsized":"2024-04-26T21:59:58.638686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016362+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T21:59:59.016362+0000","last_peered":"2024-04-26T21:59:59.016362+0000","last_clean":"2024-04-26T21:59:59.016362+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T21:59:59.016362+0000","last_undegraded":"2024-04-26T21:59:59.016362+0000","last_fullsized":"2024-04-26T21:59:59.016362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.661744+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T21:59:57.661744+0000","last_peered":"2024-04-26T21:59:57.661744+0000","last_clean":"2024-04-26T21:59:57.661744+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T21:59:57.661744+0000","last_undegraded":"2024-04-26T21:59:57.661744+0000","last_fullsized":"2024-04-26T21:59:57.661744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016578+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T21:59:59.016578+0000","last_peered":"2024-04-26T21:59:59.016578+0000","last_clean":"2024-04-26T21:59:59.016578+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T21:59:59.016578+0000","last_undegraded":"2024-04-26T21:59:59.016578+0000","last_fullsized":"2024-04-26T21:59:59.016578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633871+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T21:59:58.633871+0000","last_peered":"2024-04-26T21:59:58.633871+0000","last_clean":"2024-04-26T21:59:58.633871+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T21:59:58.633871+0000","last_undegraded":"2024-04-26T21:59:58.633871+0000","last_fullsized":"2024-04-26T21:59:58.633871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315722+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T21:59:59.315722+0000","last_peered":"2024-04-26T21:59:59.315722+0000","last_clean":"2024-04-26T21:59:59.315722+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T21:59:59.315722+0000","last_undegraded":"2024-04-26T21:59:59.315722+0000","last_fullsized":"2024-04-26T21:59:59.315722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.643052+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T21:59:57.643052+0000","last_peered":"2024-04-26T21:59:57.643052+0000","last_clean":"2024-04-26T21:59:57.643052+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T21:59:57.643052+0000","last_undegraded":"2024-04-26T21:59:57.643052+0000","last_fullsized":"2024-04-26T21:59:57.643052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016208+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T21:59:59.016208+0000","last_peered":"2024-04-26T21:59:59.016208+0000","last_clean":"2024-04-26T21:59:59.016208+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T21:59:59.016208+0000","last_undegraded":"2024-04-26T21:59:59.016208+0000","last_fullsized":"2024-04-26T21:59:59.016208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015860+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T21:59:59.015860+0000","last_peered":"2024-04-26T21:59:59.015860+0000","last_clean":"2024-04-26T21:59:59.015860+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T21:59:59.015860+0000","last_undegraded":"2024-04-26T21:59:59.015860+0000","last_fullsized":"2024-04-26T21:59:59.015860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315421+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T21:59:59.315421+0000","last_peered":"2024-04-26T21:59:59.315421+0000","last_clean":"2024-04-26T21:59:59.315421+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T21:59:59.315421+0000","last_undegraded":"2024-04-26T21:59:59.315421+0000","last_fullsized":"2024-04-26T21:59:59.315421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707695+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T21:59:59.707695+0000","last_peered":"2024-04-26T21:59:59.707695+0000","last_clean":"2024-04-26T21:59:59.707695+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T21:59:59.707695+0000","last_undegraded":"2024-04-26T21:59:59.707695+0000","last_fullsized":"2024-04-26T21:59:59.707695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015763+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T21:59:59.015763+0000","last_peered":"2024-04-26T21:59:59.015763+0000","last_clean":"2024-04-26T21:59:59.015763+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T21:59:59.015763+0000","last_undegraded":"2024-04-26T21:59:59.015763+0000","last_fullsized":"2024-04-26T21:59:59.015763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.656036+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T21:59:57.656036+0000","last_peered":"2024-04-26T21:59:57.656036+0000","last_clean":"2024-04-26T21:59:57.656036+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T21:59:57.656036+0000","last_undegraded":"2024-04-26T21:59:57.656036+0000","last_fullsized":"2024-04-26T21:59:57.656036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:11.186506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.642317+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T21:59:57.642317+0000","last_peered":"2024-04-26T21:59:57.642317+0000","last_clean":"2024-04-26T21:59:57.642317+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T21:59:57.642317+0000","last_undegraded":"2024-04-26T21:59:57.642317+0000","last_fullsized":"2024-04-26T21:59:57.642317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633705+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T21:59:58.633705+0000","last_peered":"2024-04-26T21:59:58.633705+0000","last_clean":"2024-04-26T21:59:58.633705+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T21:59:58.633705+0000","last_undegraded":"2024-04-26T21:59:58.633705+0000","last_fullsized":"2024-04-26T21:59:58.633705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315803+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T21:59:59.315803+0000","last_peered":"2024-04-26T21:59:59.315803+0000","last_clean":"2024-04-26T21:59:59.315803+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T21:59:59.315803+0000","last_undegraded":"2024-04-26T21:59:59.315803+0000","last_fullsized":"2024-04-26T21:59:59.315803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637923+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T21:59:58.637923+0000","last_peered":"2024-04-26T21:59:58.637923+0000","last_clean":"2024-04-26T21:59:58.637923+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T21:59:58.637923+0000","last_undegraded":"2024-04-26T21:59:58.637923+0000","last_fullsized":"2024-04-26T21:59:58.637923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633980+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T21:59:58.633980+0000","last_peered":"2024-04-26T21:59:58.633980+0000","last_clean":"2024-04-26T21:59:58.633980+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T21:59:58.633980+0000","last_undegraded":"2024-04-26T21:59:58.633980+0000","last_fullsized":"2024-04-26T21:59:58.633980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315934+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T21:59:59.315934+0000","last_peered":"2024-04-26T21:59:59.315934+0000","last_clean":"2024-04-26T21:59:59.315934+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T21:59:59.315934+0000","last_undegraded":"2024-04-26T21:59:59.315934+0000","last_fullsized":"2024-04-26T21:59:59.315934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016420+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T21:59:59.016420+0000","last_peered":"2024-04-26T21:59:59.016420+0000","last_clean":"2024-04-26T21:59:59.016420+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T21:59:59.016420+0000","last_undegraded":"2024-04-26T21:59:59.016420+0000","last_fullsized":"2024-04-26T21:59:59.016420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633894+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T21:59:58.633894+0000","last_peered":"2024-04-26T21:59:58.633894+0000","last_clean":"2024-04-26T21:59:58.633894+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T21:59:58.633894+0000","last_undegraded":"2024-04-26T21:59:58.633894+0000","last_fullsized":"2024-04-26T21:59:58.633894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.649584+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T21:59:58.649584+0000","last_peered":"2024-04-26T21:59:58.649584+0000","last_clean":"2024-04-26T21:59:58.649584+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T21:59:58.649584+0000","last_undegraded":"2024-04-26T21:59:58.649584+0000","last_fullsized":"2024-04-26T21:59:58.649584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708073+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T21:59:59.708073+0000","last_peered":"2024-04-26T21:59:59.708073+0000","last_clean":"2024-04-26T21:59:59.708073+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T21:59:59.708073+0000","last_undegraded":"2024-04-26T21:59:59.708073+0000","last_fullsized":"2024-04-26T21:59:59.708073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.637980+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T21:59:58.637980+0000","last_peered":"2024-04-26T21:59:58.637980+0000","last_clean":"2024-04-26T21:59:58.637980+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T21:59:58.637980+0000","last_undegraded":"2024-04-26T21:59:58.637980+0000","last_fullsized":"2024-04-26T21:59:58.637980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.654831+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T21:59:57.654831+0000","last_peered":"2024-04-26T21:59:57.654831+0000","last_clean":"2024-04-26T21:59:57.654831+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T21:59:57.654831+0000","last_undegraded":"2024-04-26T21:59:57.654831+0000","last_fullsized":"2024-04-26T21:59:57.654831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:14.481004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.015759+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T21:59:59.015759+0000","last_peered":"2024-04-26T21:59:59.015759+0000","last_clean":"2024-04-26T21:59:59.015759+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T21:59:59.015759+0000","last_undegraded":"2024-04-26T21:59:59.015759+0000","last_fullsized":"2024-04-26T21:59:59.015759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638723+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T21:59:58.638723+0000","last_peered":"2024-04-26T21:59:58.638723+0000","last_clean":"2024-04-26T21:59:58.638723+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T21:59:58.638723+0000","last_undegraded":"2024-04-26T21:59:58.638723+0000","last_fullsized":"2024-04-26T21:59:58.638723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633795+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T21:59:58.633795+0000","last_peered":"2024-04-26T21:59:58.633795+0000","last_clean":"2024-04-26T21:59:58.633795+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T21:59:58.633795+0000","last_undegraded":"2024-04-26T21:59:58.633795+0000","last_fullsized":"2024-04-26T21:59:58.633795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707799+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T21:59:59.707799+0000","last_peered":"2024-04-26T21:59:59.707799+0000","last_clean":"2024-04-26T21:59:59.707799+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T21:59:59.707799+0000","last_undegraded":"2024-04-26T21:59:59.707799+0000","last_fullsized":"2024-04-26T21:59:59.707799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.633805+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T21:59:58.633805+0000","last_peered":"2024-04-26T21:59:58.633805+0000","last_clean":"2024-04-26T21:59:58.633805+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T21:59:58.633805+0000","last_undegraded":"2024-04-26T21:59:58.633805+0000","last_fullsized":"2024-04-26T21:59:58.633805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.707766+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T21:59:59.707766+0000","last_peered":"2024-04-26T21:59:59.707766+0000","last_clean":"2024-04-26T21:59:59.707766+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T21:59:59.707766+0000","last_undegraded":"2024-04-26T21:59:59.707766+0000","last_fullsized":"2024-04-26T21:59:59.707766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016157+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T21:59:59.016157+0000","last_peered":"2024-04-26T21:59:59.016157+0000","last_clean":"2024-04-26T21:59:59.016157+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T21:59:59.016157+0000","last_undegraded":"2024-04-26T21:59:59.016157+0000","last_fullsized":"2024-04-26T21:59:59.016157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:58.638174+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T21:59:58.638174+0000","last_peered":"2024-04-26T21:59:58.638174+0000","last_clean":"2024-04-26T21:59:58.638174+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T21:59:58.638174+0000","last_undegraded":"2024-04-26T21:59:58.638174+0000","last_fullsized":"2024-04-26T21:59:58.638174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315965+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T21:59:59.315965+0000","last_peered":"2024-04-26T21:59:59.315965+0000","last_clean":"2024-04-26T21:59:59.315965+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T21:59:59.315965+0000","last_undegraded":"2024-04-26T21:59:59.315965+0000","last_fullsized":"2024-04-26T21:59:59.315965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016148+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T21:59:59.016148+0000","last_peered":"2024-04-26T21:59:59.016148+0000","last_clean":"2024-04-26T21:59:59.016148+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T21:59:59.016148+0000","last_undegraded":"2024-04-26T21:59:59.016148+0000","last_fullsized":"2024-04-26T21:59:59.016148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.315569+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T21:59:59.315569+0000","last_peered":"2024-04-26T21:59:59.315569+0000","last_clean":"2024-04-26T21:59:59.315569+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T21:59:59.315569+0000","last_undegraded":"2024-04-26T21:59:59.315569+0000","last_fullsized":"2024-04-26T21:59:59.315569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.016343+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T21:59:59.016343+0000","last_peered":"2024-04-26T21:59:59.016343+0000","last_clean":"2024-04-26T21:59:59.016343+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T21:59:59.016343+0000","last_undegraded":"2024-04-26T21:59:59.016343+0000","last_fullsized":"2024-04-26T21:59:59.016343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.655832+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T21:59:57.655832+0000","last_peered":"2024-04-26T21:59:57.655832+0000","last_clean":"2024-04-26T21:59:57.655832+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T21:59:57.655832+0000","last_undegraded":"2024-04-26T21:59:57.655832+0000","last_fullsized":"2024-04-26T21:59:57.655832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708025+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T21:59:59.708025+0000","last_peered":"2024-04-26T21:59:59.708025+0000","last_clean":"2024-04-26T21:59:59.708025+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T21:59:59.708025+0000","last_undegraded":"2024-04-26T21:59:59.708025+0000","last_fullsized":"2024-04-26T21:59:59.708025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T21:59:59.708004+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T21:59:59.708004+0000","last_peered":"2024-04-26T21:59:59.708004+0000","last_clean":"2024-04-26T21:59:59.708004+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T21:59:59.708004+0000","last_undegraded":"2024-04-26T21:59:59.708004+0000","last_fullsized":"2024-04-26T21:59:59.708004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-26T21:59:57.664743+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T21:59:57.664743+0000","last_peered":"2024-04-26T21:59:57.664743+0000","last_clean":"2024-04-26T21:59:57.664743+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T21:59:57.664743+0000","last_undegraded":"2024-04-26T21:59:57.664743+0000","last_fullsized":"2024-04-26T21:59:57.664743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:02.079 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:00:02.079 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:02.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.214+0000 7ff5046c8640 1 Processor -- start 2024-04-26T22:00:02.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.214+0000 7ff5046c8640 1 -- start start 2024-04-26T22:00:02.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff5046c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff4fc13fe20 0x7ff4fc0c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff5046c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff5046c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4fc15a230 con 0x7ff4fc06d860 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff5046c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff4fc0c39b0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff5046c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc15b120 con 0x7ff4fc0c3c70 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff502464640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff4fc13fe20 0x7ff4fc0c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff502464640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff4fc13fe20 0x7ff4fc0c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46350/0 (socket says 172.21.15.135:46350) 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff502464640 1 -- 172.21.15.135:0/487528217 learned_addr learned my addr 172.21.15.135:0/487528217 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.215+0000 7ff501c63640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff502464640 1 -- 172.21.15.135:0/487528217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 msgr2=0x7ff4fc06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff502464640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff502464640 1 -- 172.21.15.135:0/487528217 >> v1:172.21.15.139:6789/0 conn(0x7ff4fc06d860 legacy=0x7ff4fc157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff501462640 1 -- 172.21.15.135:0/487528217 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2054451280 0 0) 0x7ff4fc15a230 con 0x7ff4fc06d860 2024-04-26T22:00:02.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff502464640 1 -- 172.21.15.135:0/487528217 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc15a510 con 0x7ff4fc13fe20 2024-04-26T22:00:02.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff501c63640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:02.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff502464640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc0c5430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009870 tx=0x7ff4ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762ed1c11ee02642 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.216+0000 7ff501462640 1 -- 172.21.15.135:0/487528217 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec037070 con 0x7ff4fc13fe20 2024-04-26T22:00:02.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff501462640 1 -- 172.21.15.135:0/487528217 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ec032440 con 0x7ff4fc13fe20 2024-04-26T22:00:02.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff501462640 1 -- 172.21.15.135:0/487528217 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec032810 con 0x7ff4fc13fe20 2024-04-26T22:00:02.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 -- 172.21.15.135:0/487528217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 msgr2=0x7ff4fc0c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc0c5430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009870 tx=0x7ff4ec02a270 comp rx=0 tx=0).stop 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 -- 172.21.15.135:0/487528217 shutdown_connections 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 --2- 172.21.15.135:0/487528217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc0c5430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 -- 172.21.15.135:0/487528217 >> 172.21.15.135:0/487528217 conn(0x7ff4fc0c5d70 msgr2=0x7ff4fc0c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 -- 172.21.15.135:0/487528217 shutdown_connections 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.217+0000 7ff5046c8640 1 -- 172.21.15.135:0/487528217 wait complete. 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.218+0000 7ff5046c8640 1 Processor -- start 2024-04-26T22:00:02.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.218+0000 7ff5046c8640 1 -- start start 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff5046c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4fc06d860 0x7ff4fc1f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff5046c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc0c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff5046c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc13be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff5046c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc15adb0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff5046c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc15aad0 con 0x7ff4fc06d860 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff5046c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc140200 con 0x7ff4fc0c3c70 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc13be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc13be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46360/0 (socket says 172.21.15.135:46360) 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 -- 172.21.15.135:0/3666674713 learned_addr learned my addr 172.21.15.135:0/3666674713 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502464640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4fc06d860 0x7ff4fc1f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff501c63640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc0c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 -- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 msgr2=0x7ff4fc0c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc0c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 -- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4fc06d860 msgr2=0x7ff4fc1f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4fc06d860 0x7ff4fc1f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 -- 172.21.15.135:0/3666674713 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc074880 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.219+0000 7ff502c65640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc13be00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400c760 tx=0x7ff4f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.220+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4003da0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.220+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f4005070 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.220+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc074b60 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.220+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4005460 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.220+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc1f02e0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.221+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc069bc0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.221+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4f4005600 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.221+0000 7ff4f2ffd640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff4e006c8f0 0x7ff4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.222+0000 7ff502464640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff4e006c8f0 0x7ff4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.222+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7ff4f408b980 con 0x7ff4fc13fe20 2024-04-26T22:00:02.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.222+0000 7ff502464640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff4e006c8f0 0x7ff4e006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec034000 tx=0x7ff4ec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.226+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4fc069bc0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.408+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4fc0c84e0 con 0x7ff4fc13fe20 2024-04-26T22:00:02.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.409+0000 7ff4f2ffd640 1 -- 172.21.15.135:0/3666674713 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16633 (secure 0 0 0) 0x7ff4f4059770 con 0x7ff4fc13fe20 2024-04-26T22:00:02.409 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:02.409 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":20,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T21:59:58.630451+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff4e006c8f0 msgr2=0x7ff4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff4e006c8f0 0x7ff4e006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec034000 tx=0x7ff4ec02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 msgr2=0x7ff4fc13be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc13be00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400c760 tx=0x7ff4f400cc20 comp rx=0 tx=0).stop 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 shutdown_connections 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff4e006c8f0 0x7ff4e006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4fc13fe20 0x7ff4fc13be00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4fc0c3c70 0x7ff4fc0c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 --2- 172.21.15.135:0/3666674713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4fc06d860 0x7ff4fc1f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 >> 172.21.15.135:0/3666674713 conn(0x7ff4fc0c5d70 msgr2=0x7ff4fc156380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 shutdown_connections 2024-04-26T22:00:02.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.411+0000 7ff5046c8640 1 -- 172.21.15.135:0/3666674713 wait complete. 2024-04-26T22:00:02.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:02.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:02.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.525+0000 7fc428fc1640 1 Processor -- start 2024-04-26T22:00:02.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.525+0000 7fc428fc1640 1 -- start start 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc428fc1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4240c3060 0x7fc4240720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc428fc1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4240c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc428fc1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc424154ba0 con 0x7fc4240c40e0 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc428fc1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc424154e80 con 0x7fc4240c3060 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc428fc1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc424155160 con 0x7fc4240c46e0 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc421d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4240c40e0 0x7fc424151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:35956/0 (socket says 172.21.15.135:35956) 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc422d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4240c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc421d9b640 1 -- 172.21.15.135:0/3212849707 learned_addr learned my addr 172.21.15.135:0/3212849707 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.526+0000 7fc42259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4240c3060 0x7fc4240720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.527+0000 7fc42259c640 1 -- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 msgr2=0x7fc4240c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.527+0000 7fc42259c640 1 --2- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4240c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.527+0000 7fc42259c640 1 -- 172.21.15.135:0/3212849707 >> v1:172.21.15.139:6789/0 conn(0x7fc4240c40e0 legacy=0x7fc424151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.527+0000 7fc42259c640 1 -- 172.21.15.135:0/3212849707 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc424155440 con 0x7fc4240c3060 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.527+0000 7fc42259c640 1 --2- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc4240720a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0097a0 tx=0x7fc41c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6844210580990f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc42159a640 1 -- 172.21.15.135:0/3212849707 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c037070 con 0x7fc4240c3060 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc42159a640 1 -- 172.21.15.135:0/3212849707 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41c02ae50 con 0x7fc4240c3060 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc42159a640 1 -- 172.21.15.135:0/3212849707 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c032660 con 0x7fc4240c3060 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 -- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 msgr2=0x7fc4240720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc4240720a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0097a0 tx=0x7fc41c02a100 comp rx=0 tx=0).stop 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 -- 172.21.15.135:0/3212849707 shutdown_connections 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4240c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3212849707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc4240720a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 -- 172.21.15.135:0/3212849707 >> 172.21.15.135:0/3212849707 conn(0x7fc42406c310 msgr2=0x7fc42406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 -- 172.21.15.135:0/3212849707 shutdown_connections 2024-04-26T22:00:02.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.528+0000 7fc428fc1640 1 -- 172.21.15.135:0/3212849707 wait complete. 2024-04-26T22:00:02.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.529+0000 7fc428fc1640 1 Processor -- start 2024-04-26T22:00:02.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 -- start start 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc424158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 0x7fc424156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4241ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240c3440 con 0x7fc4240c40e0 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4240c3aa0 con 0x7fc4240c3060 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc428fc1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4240c35c0 con 0x7fc4240c46e0 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc421d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 0x7fc424156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc421d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 0x7fc424156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46370/0 (socket says 172.21.15.135:46370) 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.530+0000 7fc421d9b640 1 -- 172.21.15.135:0/3195065629 learned_addr learned my addr 172.21.15.135:0/3195065629 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc422d9d640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4241ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc42259c640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc424158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc421d9b640 1 -- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 msgr2=0x7fc4241ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc421d9b640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4241ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc421d9b640 1 -- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 msgr2=0x7fc424158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc421d9b640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc424158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc421d9b640 1 -- 172.21.15.135:0/3195065629 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4240aae90 con 0x7fc4240c40e0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc42259c640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc424158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc421d9b640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 0x7fc424156e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc41800cb10 tx=0x7fc41800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x7fc4240c40e0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41800ac70 con 0x7fc4240c40e0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc424155ea0 con 0x7fc4240c40e0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418012620 con 0x7fc4240c40e0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.531+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc424156460 con 0x7fc4240c40e0 2024-04-26T22:00:02.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.532+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4240c52c0 con 0x7fc4240c40e0 2024-04-26T22:00:02.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.532+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc41801a070 con 0x7fc4240c40e0 2024-04-26T22:00:02.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.533+0000 7fc412ffd640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc40406c8f0 0x7fc40406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.533+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fc41808b930 con 0x7fc4240c40e0 2024-04-26T22:00:02.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.533+0000 7fc42259c640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc40406c8f0 0x7fc40406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.533+0000 7fc42259c640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc40406c8f0 0x7fc40406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc41c034000 tx=0x7fc41c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.537+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4240c52c0 con 0x7fc4240c40e0 2024-04-26T22:00:02.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.718+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4240c8040 con 0x7fc4240c40e0 2024-04-26T22:00:02.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.719+0000 7fc412ffd640 1 -- 172.21.15.135:0/3195065629 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc418059d70 con 0x7fc4240c40e0 2024-04-26T22:00:02.719 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:02.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc40406c8f0 msgr2=0x7fc40406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc40406c8f0 0x7fc40406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc41c034000 tx=0x7fc41c02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 msgr2=0x7fc424156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 0x7fc424156e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc41800cb10 tx=0x7fc41800a060 comp rx=0 tx=0).stop 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 shutdown_connections 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc40406c8f0 0x7fc40406edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4240c46e0 0x7fc4241ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240c40e0 0x7fc424156e40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 --2- 172.21.15.135:0/3195065629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4240c3060 0x7fc424158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 >> 172.21.15.135:0/3195065629 conn(0x7fc42406c310 msgr2=0x7fc42406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 shutdown_connections 2024-04-26T22:00:02.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.721+0000 7fc428fc1640 1 -- 172.21.15.135:0/3195065629 wait complete. 2024-04-26T22:00:02.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:02.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.835+0000 7fc7f79f7640 1 Processor -- start 2024-04-26T22:00:02.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.835+0000 7fc7f79f7640 1 -- start start 2024-04-26T22:00:02.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f79f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc7f0154830 0x7fc7f00bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f79f7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc7f00c5710 0x7fc7f00c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f79f7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f00c0190 con 0x7fc7f00c4730 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f79f7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f0074c70 con 0x7fc7f00c5710 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f79f7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f00bff40 con 0x7fc7f0154830 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc7f00c5710 0x7fc7f00c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc7f00c5710 0x7fc7f00c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46382/0 (socket says 172.21.15.135:46382) 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 -- 172.21.15.135:0/3445577836 learned_addr learned my addr 172.21.15.135:0/3445577836 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5793640 1 --2- 172.21.15.135:0/3445577836 >> v2:172.21.15.139:3300/0 conn(0x7fc7f0154830 0x7fc7f00bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 -- 172.21.15.135:0/3445577836 >> v2:172.21.15.139:3300/0 conn(0x7fc7f0154830 msgr2=0x7fc7f00bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 --2- 172.21.15.135:0/3445577836 >> v2:172.21.15.139:3300/0 conn(0x7fc7f0154830 0x7fc7f00bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 -- 172.21.15.135:0/3445577836 >> v1:172.21.15.139:6789/0 conn(0x7fc7f00c4730 legacy=0x7fc7f006ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.836+0000 7fc7f5f94640 1 -- 172.21.15.135:0/3445577836 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f006f1d0 con 0x7fc7f00c5710 2024-04-26T22:00:02.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.837+0000 7fc7f5f94640 1 --2- 172.21.15.135:0/3445577836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c5710 0x7fc7f00c4c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00c9fb0 tx=0x7fc7e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cad6182430d2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.837+0000 7fc7e7fff640 1 -- 172.21.15.135:0/3445577836 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8036070 con 0x7fc7f00c5710 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7e7fff640 1 -- 172.21.15.135:0/3445577836 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e8031600 con 0x7fc7f00c5710 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7e7fff640 1 -- 172.21.15.135:0/3445577836 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80319d0 con 0x7fc7f00c5710 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 -- 172.21.15.135:0/3445577836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c5710 msgr2=0x7fc7f00c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/3445577836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c5710 0x7fc7f00c4c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00c9fb0 tx=0x7fc7e802a6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 -- 172.21.15.135:0/3445577836 shutdown_connections 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/3445577836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c5710 0x7fc7f00c4c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/3445577836 >> v2:172.21.15.139:3300/0 conn(0x7fc7f0154830 0x7fc7f00bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 -- 172.21.15.135:0/3445577836 >> 172.21.15.135:0/3445577836 conn(0x7fc7f00c91e0 msgr2=0x7fc7f00c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 -- 172.21.15.135:0/3445577836 shutdown_connections 2024-04-26T22:00:02.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.838+0000 7fc7f79f7640 1 -- 172.21.15.135:0/3445577836 wait complete. 2024-04-26T22:00:02.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.839+0000 7fc7f79f7640 1 Processor -- start 2024-04-26T22:00:02.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.839+0000 7fc7f79f7640 1 -- start start 2024-04-26T22:00:02.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f79f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 0x7fc7f01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f79f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f00c5710 0x7fc7f01f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f79f7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7f0154830 0x7fc7f00ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f79f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f00706a0 con 0x7fc7f00c4730 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f79f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f006f820 con 0x7fc7f00c5710 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f79f7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f00c51c0 con 0x7fc7f0154830 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 0x7fc7f01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 0x7fc7f01583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46392/0 (socket says 172.21.15.135:46392) 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 -- 172.21.15.135:0/1810868416 learned_addr learned my addr 172.21.15.135:0/1810868416 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 -- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7f0154830 msgr2=0x7fc7f00ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7f0154830 0x7fc7f00ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f4f92640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f00c5710 0x7fc7f01f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 -- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f00c5710 msgr2=0x7fc7f01f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f00c5710 0x7fc7f01f0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 -- 172.21.15.135:0/1810868416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f0158930 con 0x7fc7f00c4730 2024-04-26T22:00:02.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7f5793640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 0x7fc7f01583f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000e870 tx=0x7fc7e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.840+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0017070 con 0x7fc7f00c4730 2024-04-26T22:00:02.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.841+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e000c940 con 0x7fc7f00c4730 2024-04-26T22:00:02.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.841+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f01f01a0 con 0x7fc7f00c4730 2024-04-26T22:00:02.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.841+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000cd30 con 0x7fc7f00c4730 2024-04-26T22:00:02.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.841+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f0074870 con 0x7fc7f00c4730 2024-04-26T22:00:02.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.842+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f0074a70 con 0x7fc7f00c4730 2024-04-26T22:00:02.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.842+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7e001a070 con 0x7fc7f00c4730 2024-04-26T22:00:02.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.843+0000 7fc7e5ffb640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7d406c8f0 0x7fc7d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.843+0000 7fc7f4f92640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7d406c8f0 0x7fc7d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.843+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fc7e008b970 con 0x7fc7f00c4730 2024-04-26T22:00:02.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.843+0000 7fc7f4f92640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7d406c8f0 0x7fc7d406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7f01f1140 tx=0x7fc7ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:02.847+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7e0059e40 con 0x7fc7f00c4730 2024-04-26T22:00:03.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.044+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7fc7f0068dc0 con 0x7fc7f00c4730 2024-04-26T22:00:03.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.045+0000 7fc7e5ffb640 1 -- 172.21.15.135:0/1810868416 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v20) ==== 99+0+11 (secure 0 0 0) 0x7fc7e0059740 con 0x7fc7f00c4730 2024-04-26T22:00:03.045 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7d406c8f0 msgr2=0x7fc7d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7d406c8f0 0x7fc7d406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7f01f1140 tx=0x7fc7ec009040 comp rx=0 tx=0).stop 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 msgr2=0x7fc7f01583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 0x7fc7f01583f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000e870 tx=0x7fc7e000ed30 comp rx=0 tx=0).stop 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 shutdown_connections 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7d406c8f0 0x7fc7d406edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7f0154830 0x7fc7f00ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f00c5710 0x7fc7f01f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 --2- 172.21.15.135:0/1810868416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7f00c4730 0x7fc7f01583f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 >> 172.21.15.135:0/1810868416 conn(0x7fc7f00c91e0 msgr2=0x7fc7f006c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 shutdown_connections 2024-04-26T22:00:03.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.047+0000 7fc7f79f7640 1 -- 172.21.15.135:0/1810868416 wait complete. 2024-04-26T22:00:03.057 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:03.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.157+0000 7fdaaa7a4640 1 Processor -- start 2024-04-26T22:00:03.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.158+0000 7fdaaa7a4640 1 -- start start 2024-04-26T22:00:03.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.158+0000 7fdaaa7a4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdaa4151e10 0x7fdaa40ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.158+0000 7fdaaa7a4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa4074620 0x7fdaa41582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.158+0000 7fdaaa7a4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa4154920 con 0x7fdaa40aca30 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.158+0000 7fdaaa7a4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa4074e90 con 0x7fdaa4151e10 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.158+0000 7fdaaa7a4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa40c6b60 con 0x7fdaa4074620 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa4074620 0x7fdaa41582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdaa4151e10 0x7fdaa40ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa8d41640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdaa40aca30 0x7fdaa4158830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:35978/0 (socket says 172.21.15.135:35978) 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa8d41640 1 -- 172.21.15.135:0/2966255341 learned_addr learned my addr 172.21.15.135:0/2966255341 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa2ffd640 1 -- 172.21.15.135:0/2966255341 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748772329 0 0) 0x7fdaa4154920 con 0x7fdaa40aca30 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa2ffd640 1 -- 172.21.15.135:0/2966255341 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda8c003520 con 0x7fdaa40aca30 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa3fff640 1 -- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa4074620 msgr2=0x7fdaa41582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa3fff640 1 --2- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa4074620 0x7fdaa41582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa3fff640 1 -- 172.21.15.135:0/2966255341 >> v1:172.21.15.139:6789/0 conn(0x7fdaa40aca30 legacy=0x7fdaa4158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa3fff640 1 -- 172.21.15.135:0/2966255341 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40c62c0 con 0x7fdaa4151e10 2024-04-26T22:00:03.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.159+0000 7fdaa37fe640 1 --2- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa4074620 0x7fdaa41582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:03.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaa3fff640 1 --2- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa40ac4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda940097a0 tx=0x7fda9402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d99d7b207dc042 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaa2ffd640 1 -- 172.21.15.135:0/2966255341 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94037070 con 0x7fdaa4151e10 2024-04-26T22:00:03.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaa2ffd640 1 -- 172.21.15.135:0/2966255341 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda94032440 con 0x7fdaa4151e10 2024-04-26T22:00:03.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaa2ffd640 1 -- 172.21.15.135:0/2966255341 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda940327f0 con 0x7fdaa4151e10 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 msgr2=0x7fdaa40ac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa40ac4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda940097a0 tx=0x7fda9402a180 comp rx=0 tx=0).stop 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/2966255341 shutdown_connections 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa4074620 0x7fdaa41582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/2966255341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa40ac4f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/2966255341 >> 172.21.15.135:0/2966255341 conn(0x7fdaa40ca2f0 msgr2=0x7fdaa40ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/2966255341 shutdown_connections 2024-04-26T22:00:03.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.160+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/2966255341 wait complete. 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.161+0000 7fdaaa7a4640 1 Processor -- start 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 -- start start 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 0x7fdaa414c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa40aca30 0x7fdaa41ed930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa413fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa40c5f50 con 0x7fdaa4074620 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa40c6880 con 0x7fdaa4151e10 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaaa7a4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa4071f60 con 0x7fdaa40aca30 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 0x7fdaa414c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 0x7fdaa414c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46408/0 (socket says 172.21.15.135:46408) 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaa3fff640 1 -- 172.21.15.135:0/724527943 learned_addr learned my addr 172.21.15.135:0/724527943 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaa37fe640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa40aca30 0x7fdaa41ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.162+0000 7fdaa8d41640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa413fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa3fff640 1 -- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa40aca30 msgr2=0x7fdaa41ed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa3fff640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa40aca30 0x7fdaa41ed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa3fff640 1 -- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 msgr2=0x7fdaa413fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa3fff640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa413fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa3fff640 1 -- 172.21.15.135:0/724527943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa4154780 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa3fff640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 0x7fdaa414c150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda94035040 tx=0x7fda940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94037040 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda94042070 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa40ad3e0 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9403b930 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.163+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa40ad6c0 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.164+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa40766c0 con 0x7fdaa4074620 2024-04-26T22:00:03.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.164+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda9403bad0 con 0x7fdaa4074620 2024-04-26T22:00:03.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.165+0000 7fdaa0ff9640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fda9006c8f0 0x7fda9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.165+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fda940ac220 con 0x7fdaa4074620 2024-04-26T22:00:03.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.165+0000 7fdaa37fe640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fda9006c8f0 0x7fda9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.165+0000 7fdaa37fe640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fda9006c8f0 0x7fda9006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda98009560 tx=0x7fda98009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.169+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda9407a0a0 con 0x7fdaa4074620 2024-04-26T22:00:03.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.362+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fdaa40507a0 con 0x7fdaa4074620 2024-04-26T22:00:03.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.362+0000 7fdaa0ff9640 1 -- 172.21.15.135:0/724527943 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v20) ==== 95+0+11 (secure 0 0 0) 0x7fda9407e3a0 con 0x7fdaa4074620 2024-04-26T22:00:03.362 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:03.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fda9006c8f0 msgr2=0x7fda9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fda9006c8f0 0x7fda9006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda98009560 tx=0x7fda98009190 comp rx=0 tx=0).stop 2024-04-26T22:00:03.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 msgr2=0x7fdaa414c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 0x7fdaa414c150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda94035040 tx=0x7fda940046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 shutdown_connections 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fda9006c8f0 0x7fda9006edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaa4151e10 0x7fdaa413fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaa40aca30 0x7fdaa41ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 --2- 172.21.15.135:0/724527943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4074620 0x7fdaa414c150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 >> 172.21.15.135:0/724527943 conn(0x7fdaa40ca2f0 msgr2=0x7fdaa40ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 shutdown_connections 2024-04-26T22:00:03.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.364+0000 7fdaaa7a4640 1 -- 172.21.15.135:0/724527943 wait complete. 2024-04-26T22:00:03.375 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.472+0000 7ff35a389640 1 Processor -- start 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.472+0000 7ff35a389640 1 -- start start 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff35a389640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3540c9830 0x7ff354153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff35a389640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff354156de0 con 0x7ff3540c7000 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff35a389640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff354157b60 con 0x7ff354154220 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff35a389640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff3540cad40 con 0x7ff3540c9830 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff353fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff3540c7000 0x7ff3540ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38952/0 (socket says 172.21.15.135:38952) 2024-04-26T22:00:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff353fff640 1 -- 172.21.15.135:0/557458658 learned_addr learned my addr 172.21.15.135:0/557458658 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff3537fe640 1 --2- 172.21.15.135:0/557458658 >> v2:172.21.15.139:3300/0 conn(0x7ff3540c9830 0x7ff354153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff352ffd640 1 -- 172.21.15.135:0/557458658 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450489420 0 0) 0x7ff354156de0 con 0x7ff3540c7000 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff352ffd640 1 -- 172.21.15.135:0/557458658 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff33c003520 con 0x7ff3540c7000 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff352ffd640 1 -- 172.21.15.135:0/557458658 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 156549843 0 0) 0x7ff33c003520 con 0x7ff3540c7000 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff3537fe640 1 -- 172.21.15.135:0/557458658 >> v1:172.21.15.139:6789/0 conn(0x7ff354154220 legacy=0x7ff354072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff3537fe640 1 -- 172.21.15.135:0/557458658 >> v1:172.21.15.135:6789/0 conn(0x7ff3540c7000 legacy=0x7ff3540ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.473+0000 7ff3537fe640 1 -- 172.21.15.135:0/557458658 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff354157080 con 0x7ff3540c9830 2024-04-26T22:00:03.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.474+0000 7ff3537fe640 1 --2- 172.21.15.135:0/557458658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff354153ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff3480097a0 tx=0x7ff34802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14567b69a226ff82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.474+0000 7ff352ffd640 1 -- 172.21.15.135:0/557458658 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348037070 con 0x7ff3540c9830 2024-04-26T22:00:03.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.474+0000 7ff352ffd640 1 -- 172.21.15.135:0/557458658 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34802ac50 con 0x7ff3540c9830 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.474+0000 7ff352ffd640 1 -- 172.21.15.135:0/557458658 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344001670 con 0x7ff3540c9830 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 -- 172.21.15.135:0/557458658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 msgr2=0x7ff354153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 --2- 172.21.15.135:0/557458658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff354153ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff3480097a0 tx=0x7ff34802a1c0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 -- 172.21.15.135:0/557458658 shutdown_connections 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 --2- 172.21.15.135:0/557458658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff354153ce0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 -- 172.21.15.135:0/557458658 >> 172.21.15.135:0/557458658 conn(0x7ff3540adf20 msgr2=0x7ff3540ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 -- 172.21.15.135:0/557458658 shutdown_connections 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.475+0000 7ff35a389640 1 -- 172.21.15.135:0/557458658 wait complete. 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.476+0000 7ff35a389640 1 Processor -- start 2024-04-26T22:00:03.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.476+0000 7ff35a389640 1 -- start start 2024-04-26T22:00:03.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3540c7000 0x7ff354155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff3541562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 0x7ff35414a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540c73e0 con 0x7ff354154220 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff354157640 con 0x7ff3540c9830 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3540c7560 con 0x7ff3540c7000 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 0x7ff35414a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 0x7ff35414a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46412/0 (socket says 172.21.15.135:46412) 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 -- 172.21.15.135:0/237241595 learned_addr learned my addr 172.21.15.135:0/237241595 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff353fff640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3540c7000 0x7ff354155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff3537fe640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff3541562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 -- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3540c7000 msgr2=0x7ff354155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3540c7000 0x7ff354155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 -- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 msgr2=0x7ff3541562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff3541562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 -- 172.21.15.135:0/237241595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3540c34e0 con 0x7ff354154220 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff358926640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 0x7ff35414a8c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c6b0 tx=0x7ff34c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c003b70 con 0x7ff354154220 2024-04-26T22:00:03.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34c003d30 con 0x7ff354154220 2024-04-26T22:00:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3541ed030 con 0x7ff354154220 2024-04-26T22:00:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c005310 con 0x7ff354154220 2024-04-26T22:00:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.477+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35414e720 con 0x7ff354154220 2024-04-26T22:00:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.478+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3540c8470 con 0x7ff354154220 2024-04-26T22:00:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.479+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff34c0054b0 con 0x7ff354154220 2024-04-26T22:00:03.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.479+0000 7ff350ff9640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff34006c8f0 0x7ff34006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.479+0000 7ff353fff640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff34006c8f0 0x7ff34006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.479+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7ff34c08b7f0 con 0x7ff354154220 2024-04-26T22:00:03.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.480+0000 7ff353fff640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff34006c8f0 0x7ff34006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff344005880 tx=0x7ff344000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.483 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.484+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3540c8470 con 0x7ff354154220 2024-04-26T22:00:03.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.661+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff3540766c0 con 0x7ff354154220 2024-04-26T22:00:03.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.661+0000 7ff350ff9640 1 -- 172.21.15.135:0/237241595 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) ==== 84+0+197 (secure 0 0 0) 0x7ff34c059670 con 0x7ff354154220 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout:[ 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: "poolnum": 1, 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: "poolname": ".mgr" 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: "poolnum": 2, 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: "poolname": "a_metadata" 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-26T22:00:03.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-26T22:00:03.663 INFO:teuthology.orchestra.run.smithi135.stdout: "poolnum": 3, 2024-04-26T22:00:03.663 INFO:teuthology.orchestra.run.smithi135.stdout: "poolname": "a_data" 2024-04-26T22:00:03.663 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-26T22:00:03.663 INFO:teuthology.orchestra.run.smithi135.stdout:] 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff34006c8f0 msgr2=0x7ff34006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff34006c8f0 0x7ff34006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff344005880 tx=0x7ff344000bc0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 msgr2=0x7ff35414a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 0x7ff35414a8c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c6b0 tx=0x7ff34c00cb70 comp rx=0 tx=0).stop 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 shutdown_connections 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff34006c8f0 0x7ff34006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff354154220 0x7ff35414a8c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3540c9830 0x7ff3541562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 --2- 172.21.15.135:0/237241595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3540c7000 0x7ff354155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 >> 172.21.15.135:0/237241595 conn(0x7ff3540adf20 msgr2=0x7ff3540ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 shutdown_connections 2024-04-26T22:00:03.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.664+0000 7ff35a389640 1 -- 172.21.15.135:0/237241595 wait complete. 2024-04-26T22:00:03.674 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'b' 2024-04-26T22:00:03.674 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_metadata --pg_num_min 64 2024-04-26T22:00:03.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.775+0000 7fb3979e7640 1 Processor -- start 2024-04-26T22:00:03.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb3979e7640 1 -- start start 2024-04-26T22:00:03.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb3979e7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb3900c3e90 0x7fb39015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb3979e7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3900c49d0 con 0x7fb3900c77c0 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb3979e7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3900ab5f0 con 0x7fb390073c90 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb3979e7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb3900c7ba0 con 0x7fb3900c3e90 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb395783640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb3900c77c0 0x7fb39015a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38966/0 (socket says 172.21.15.135:38966) 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb395783640 1 -- 172.21.15.135:0/2989592290 learned_addr learned my addr 172.21.15.135:0/2989592290 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.776+0000 7fb394f82640 1 --2- 172.21.15.135:0/2989592290 >> v2:172.21.15.135:3300/0 conn(0x7fb3900c3e90 0x7fb39015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.777+0000 7fb394f82640 1 -- 172.21.15.135:0/2989592290 >> v1:172.21.15.139:6789/0 conn(0x7fb390073c90 legacy=0x7fb3900740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.777+0000 7fb394f82640 1 -- 172.21.15.135:0/2989592290 >> v1:172.21.15.135:6789/0 conn(0x7fb3900c77c0 legacy=0x7fb39015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.777+0000 7fb394f82640 1 -- 172.21.15.135:0/2989592290 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3900ab8d0 con 0x7fb3900c3e90 2024-04-26T22:00:03.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.777+0000 7fb394f82640 1 --2- 172.21.15.135:0/2989592290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c3e90 0x7fb39015a7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3800097a0 tx=0x7fb38002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f92362431fa1f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.780+0000 7fb387fff640 1 -- 172.21.15.135:0/2989592290 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380037070 con 0x7fb3900c3e90 2024-04-26T22:00:03.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.780+0000 7fb387fff640 1 -- 172.21.15.135:0/2989592290 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38002adc0 con 0x7fb3900c3e90 2024-04-26T22:00:03.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.780+0000 7fb387fff640 1 -- 172.21.15.135:0/2989592290 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380032660 con 0x7fb3900c3e90 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 -- 172.21.15.135:0/2989592290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c3e90 msgr2=0x7fb39015a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2989592290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c3e90 0x7fb39015a7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3800097a0 tx=0x7fb38002a0b0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 -- 172.21.15.135:0/2989592290 shutdown_connections 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2989592290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c3e90 0x7fb39015a7b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 -- 172.21.15.135:0/2989592290 >> 172.21.15.135:0/2989592290 conn(0x7fb390075d50 msgr2=0x7fb39006e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 -- 172.21.15.135:0/2989592290 shutdown_connections 2024-04-26T22:00:03.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.781+0000 7fb3979e7640 1 -- 172.21.15.135:0/2989592290 wait complete. 2024-04-26T22:00:03.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 Processor -- start 2024-04-26T22:00:03.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 -- start start 2024-04-26T22:00:03.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb390073c90 0x7fb390154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3900c3e90 0x7fb39014f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 0x7fb390155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb390071230 con 0x7fb3900c77c0 2024-04-26T22:00:03.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb390071510 con 0x7fb390073c90 2024-04-26T22:00:03.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.782+0000 7fb3979e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb39015acf0 con 0x7fb3900c3e90 2024-04-26T22:00:03.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 0x7fb390155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 0x7fb390155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46428/0 (socket says 172.21.15.135:46428) 2024-04-26T22:00:03.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 -- 172.21.15.135:0/2064126907 learned_addr learned my addr 172.21.15.135:0/2064126907 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:03.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb394f82640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3900c3e90 0x7fb39014f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395783640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb390073c90 0x7fb390154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 -- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3900c3e90 msgr2=0x7fb39014f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3900c3e90 0x7fb39014f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 -- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb390073c90 msgr2=0x7fb390154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb390073c90 0x7fb390154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 -- 172.21.15.135:0/2064126907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39014fbd0 con 0x7fb3900c77c0 2024-04-26T22:00:03.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.783+0000 7fb395f84640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 0x7fb390155330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b490 tx=0x7fb38c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.785+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c015020 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.785+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c011850 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.785+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c011c40 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.785+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3901574a0 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.785+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3900ae5d0 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.786+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3900aca90 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.786+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb38c018020 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.787+0000 7fb385ffb640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb37406c8f0 0x7fb37406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.787+0000 7fb395783640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb37406c8f0 0x7fb37406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.787+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5689+0+0 (secure 0 0 0) 0x7fb38c08b8b0 con 0x7fb3900c77c0 2024-04-26T22:00:03.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.787+0000 7fb395783640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb37406c8f0 0x7fb37406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3880012e0 tx=0x7fb388001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.791+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb38c01f020 con 0x7fb3900c77c0 2024-04-26T22:00:03.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:03.966+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64} v 0) -- 0x7fb3900aeab0 con 0x7fb3900c77c0 2024-04-26T22:00:04.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.790+0000 7fb385ffb640 1 -- 172.21.15.135:0/2064126907 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64}]=0 pool 'b_metadata' created v21) ==== 128+0+0 (secure 0 0 0) 0x7fb38c0596a0 con 0x7fb3900c77c0 2024-04-26T22:00:04.790 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'b_metadata' created 2024-04-26T22:00:04.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb37406c8f0 msgr2=0x7fb37406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb37406c8f0 0x7fb37406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3880012e0 tx=0x7fb388001a30 comp rx=0 tx=0).stop 2024-04-26T22:00:04.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 msgr2=0x7fb390155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 0x7fb390155330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b490 tx=0x7fb38c00b950 comp rx=0 tx=0).stop 2024-04-26T22:00:04.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 shutdown_connections 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb37406c8f0 0x7fb37406edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3900c77c0 0x7fb390155330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3900c3e90 0x7fb39014f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 --2- 172.21.15.135:0/2064126907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb390073c90 0x7fb390154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 >> 172.21.15.135:0/2064126907 conn(0x7fb390075d50 msgr2=0x7fb3900c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 shutdown_connections 2024-04-26T22:00:04.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.795+0000 7fb3979e7640 1 -- 172.21.15.135:0/2064126907 wait complete. 2024-04-26T22:00:04.811 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T22:00:04.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.919+0000 7ff2f6046640 1 Processor -- start 2024-04-26T22:00:04.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.919+0000 7ff2f6046640 1 -- start start 2024-04-26T22:00:04.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2f6046640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2f013ddd0 0x7ff2f0154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2f6046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 0x7ff2f0071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2f6046640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f00707e0 con 0x7ff2f0071cc0 2024-04-26T22:00:04.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2f6046640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff2f0155450 con 0x7ff2f013ddd0 2024-04-26T22:00:04.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2f6046640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f0155730 con 0x7ff2f0070f70 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2eeffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff2f0071cc0 0x7ff2f0073700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38974/0 (socket says 172.21.15.135:38974) 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.920+0000 7ff2eeffd640 1 -- 172.21.15.135:0/3373568032 learned_addr learned my addr 172.21.15.135:0/3373568032 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217354005 0 0) 0x7ff2f00707e0 con 0x7ff2f0071cc0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ef7fe640 1 --2- 172.21.15.135:0/3373568032 >> v2:172.21.15.139:3300/0 conn(0x7ff2f013ddd0 0x7ff2f0154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2effff640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 0x7ff2f0071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2d8003520 con 0x7ff2f0071cc0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 519545864 0 0) 0x7ff2d8003520 con 0x7ff2f0071cc0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 msgr2=0x7ff2f0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 0x7ff2f0071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 >> v2:172.21.15.139:3300/0 conn(0x7ff2f013ddd0 msgr2=0x7ff2f0154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 --2- 172.21.15.135:0/3373568032 >> v2:172.21.15.139:3300/0 conn(0x7ff2f013ddd0 0x7ff2f0154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f0155a10 con 0x7ff2f0071cc0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7ff2e4001670 con 0x7ff2f0071cc0 2024-04-26T22:00:04.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 >> v1:172.21.15.135:6789/0 conn(0x7ff2f0071cc0 legacy=0x7ff2f0073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 0x7ff2d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d8004d40 0x7ff2d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0155450 con 0x7ff2d8003f30 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0155730 con 0x7ff2d8004d40 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d800ba10 con 0x7ff2f0070f70 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2ef7fe640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 0x7ff2d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2effff640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d8004d40 0x7ff2d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2eeffd640 1 -- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 msgr2=0x7ff2d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f0070f70 0x7ff2d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2eeffd640 1 -- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d8004d40 msgr2=0x7ff2d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d8004d40 0x7ff2d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.921+0000 7ff2eeffd640 1 -- 172.21.15.135:0/3373568032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8003520 con 0x7ff2d8003f30 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.922+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2d8004800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c210 tx=0x7ff2d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59137dd2b93fa7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.922+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4007d50 con 0x7ff2d8003f30 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.922+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e4005ad0 con 0x7ff2d8003f30 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.922+0000 7ff2ee7fc640 1 -- 172.21.15.135:0/3373568032 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4007070 con 0x7ff2d8003f30 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2f6046640 1 -- 172.21.15.135:0/3373568032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 msgr2=0x7ff2d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2f6046640 1 --2- 172.21.15.135:0/3373568032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2d8004800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c210 tx=0x7ff2d8005830 comp rx=0 tx=0).stop 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2effff640 1 -- 172.21.15.135:0/3373568032 reap_dead start 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2f6046640 1 -- 172.21.15.135:0/3373568032 shutdown_connections 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2f6046640 1 -- 172.21.15.135:0/3373568032 >> 172.21.15.135:0/3373568032 conn(0x7ff2f00ca2f0 msgr2=0x7ff2f00ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2f6046640 1 -- 172.21.15.135:0/3373568032 shutdown_connections 2024-04-26T22:00:04.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.923+0000 7ff2f6046640 1 -- 172.21.15.135:0/3373568032 wait complete. 2024-04-26T22:00:04.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.924+0000 7ff2f6046640 1 Processor -- start 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.924+0000 7ff2f6046640 1 -- start start 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2f0153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2f006dd30 0x7ff2f013abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f013b110 0x7ff2f01ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f013e330 con 0x7ff2d8003f30 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f013e1b0 con 0x7ff2f006dd30 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f0155040 con 0x7ff2f013b110 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2f0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2f0153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46442/0 (socket says 172.21.15.135:46442) 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 -- 172.21.15.135:0/499488022 learned_addr learned my addr 172.21.15.135:0/499488022 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2f006dd30 0x7ff2f013abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2effff640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f013b110 0x7ff2f01ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 -- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f013b110 msgr2=0x7ff2f01ec1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f013b110 0x7ff2f01ec1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 -- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2f006dd30 msgr2=0x7ff2f013abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2f006dd30 0x7ff2f013abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 -- 172.21.15.135:0/499488022 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f013b5c0 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2effff640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f013b110 0x7ff2f01ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2ef7fe640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2f0153380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011960 tx=0x7ff2e0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000fa90 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e000fc30 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f006d050 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.925+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0014440 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.926+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f0157b90 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.927+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f00507a0 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.927+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2e0012040 con 0x7ff2d8003f30 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.927+0000 7ff2d7fff640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff2dc06c920 0x7ff2dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.927+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff2dc06c920 0x7ff2dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.928+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6054+0+0 (secure 0 0 0) 0x7ff2e008c120 con 0x7ff2d8003f30 2024-04-26T22:00:04.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.928+0000 7ff2eeffd640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff2dc06c920 0x7ff2dc06ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4033000 tx=0x7ff2e4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:04.932+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2e0019030 con 0x7ff2d8003f30 2024-04-26T22:00:05.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.109+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff2f0074110 con 0x7ff2d8003f30 2024-04-26T22:00:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.798+0000 7ff2d7fff640 1 -- 172.21.15.135:0/499488022 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'b_data' created v22) ==== 160+0+0 (secure 0 0 0) 0x7ff2e005a480 con 0x7ff2d8003f30 2024-04-26T22:00:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'b_data' created 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff2dc06c920 msgr2=0x7ff2dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff2dc06c920 0x7ff2dc06ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4033000 tx=0x7ff2e4007000 comp rx=0 tx=0).stop 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 msgr2=0x7ff2f0153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2f0153380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011960 tx=0x7ff2e0011e20 comp rx=0 tx=0).stop 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 shutdown_connections 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff2dc06c920 0x7ff2dc06ede0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2f013b110 0x7ff2f01ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2f006dd30 0x7ff2f013abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 --2- 172.21.15.135:0/499488022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2d8003f30 0x7ff2f0153380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 >> 172.21.15.135:0/499488022 conn(0x7ff2f00ca2f0 msgr2=0x7ff2f013d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 shutdown_connections 2024-04-26T22:00:05.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.801+0000 7ff2f6046640 1 -- 172.21.15.135:0/499488022 wait complete. 2024-04-26T22:00:05.823 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new b b_metadata b_data 2024-04-26T22:00:05.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.922+0000 7fa860a46640 1 Processor -- start 2024-04-26T22:00:05.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.922+0000 7fa860a46640 1 -- start start 2024-04-26T22:00:05.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa860a46640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa85c13ffb0 0x7fa85c0c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa860a46640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa85c074c10 0x7fa85c0c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa860a46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c0c45f0 0x7fa85c073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa860a46640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c070880 con 0x7fa85c074c10 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa860a46640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c1579d0 con 0x7fa85c13ffb0 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa860a46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c0ab680 con 0x7fa85c0c45f0 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa859d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa85c074c10 0x7fa85c0c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa859d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa85c074c10 0x7fa85c0c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46450/0 (socket says 172.21.15.135:46450) 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.923+0000 7fa859d9b640 1 -- 172.21.15.135:0/2591418338 learned_addr learned my addr 172.21.15.135:0/2591418338 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa85a59c640 1 --2- 172.21.15.135:0/2591418338 >> v2:172.21.15.139:3300/0 conn(0x7fa85c13ffb0 0x7fa85c0c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa85ad9d640 1 --2- 172.21.15.135:0/2591418338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c0c45f0 0x7fa85c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa859d9b640 1 -- 172.21.15.135:0/2591418338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c0c45f0 msgr2=0x7fa85c073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa859d9b640 1 --2- 172.21.15.135:0/2591418338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c0c45f0 0x7fa85c073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa859d9b640 1 -- 172.21.15.135:0/2591418338 >> v2:172.21.15.139:3300/0 conn(0x7fa85c13ffb0 msgr2=0x7fa85c0c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa859d9b640 1 --2- 172.21.15.135:0/2591418338 >> v2:172.21.15.139:3300/0 conn(0x7fa85c13ffb0 0x7fa85c0c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa859d9b640 1 -- 172.21.15.135:0/2591418338 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c0ab960 con 0x7fa85c074c10 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa859d9b640 1 --2- 172.21.15.135:0/2591418338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c074c10 0x7fa85c0c40b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0cb1e0 tx=0x7fa85002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e683031daf31c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.924+0000 7fa85959a640 1 -- 172.21.15.135:0/2591418338 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850037070 con 0x7fa85c074c10 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa85959a640 1 -- 172.21.15.135:0/2591418338 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa850032440 con 0x7fa85c074c10 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa85959a640 1 -- 172.21.15.135:0/2591418338 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850032810 con 0x7fa85c074c10 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 -- 172.21.15.135:0/2591418338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c074c10 msgr2=0x7fa85c0c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 --2- 172.21.15.135:0/2591418338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c074c10 0x7fa85c0c40b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0cb1e0 tx=0x7fa85002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 -- 172.21.15.135:0/2591418338 shutdown_connections 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 --2- 172.21.15.135:0/2591418338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c074c10 0x7fa85c0c40b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 --2- 172.21.15.135:0/2591418338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c0c45f0 0x7fa85c073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 --2- 172.21.15.135:0/2591418338 >> v2:172.21.15.139:3300/0 conn(0x7fa85c13ffb0 0x7fa85c0c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 -- 172.21.15.135:0/2591418338 >> 172.21.15.135:0/2591418338 conn(0x7fa85c0ca2f0 msgr2=0x7fa85c0ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 -- 172.21.15.135:0/2591418338 shutdown_connections 2024-04-26T22:00:05.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.925+0000 7fa860a46640 1 -- 172.21.15.135:0/2591418338 wait complete. 2024-04-26T22:00:05.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.926+0000 7fa860a46640 1 Processor -- start 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.926+0000 7fa860a46640 1 -- start start 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa860a46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa85c074c10 0x7fa85c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa860a46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 0x7fa85c1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa860a46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c13ffb0 0x7fa85c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa860a46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c073c40 con 0x7fa85c0c45f0 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa860a46640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c157660 con 0x7fa85c074c10 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa860a46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c073dc0 con 0x7fa85c13ffb0 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 0x7fa85c1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 0x7fa85c1512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46458/0 (socket says 172.21.15.135:46458) 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 -- 172.21.15.135:0/681167375 learned_addr learned my addr 172.21.15.135:0/681167375 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa85ad9d640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c13ffb0 0x7fa85c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa85a59c640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa85c074c10 0x7fa85c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 -- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c13ffb0 msgr2=0x7fa85c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c13ffb0 0x7fa85c06db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 -- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa85c074c10 msgr2=0x7fa85c153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa85c074c10 0x7fa85c153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.927+0000 7fa859d9b640 1 -- 172.21.15.135:0/681167375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c0c7d20 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.928+0000 7fa859d9b640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 0x7fa85c1512a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa85002a220 tx=0x7fa8500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.928+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850041020 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.928+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500306c0 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.928+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850037040 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.928+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1ef960 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.928+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c159340 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.929+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c076650 con 0x7fa85c0c45f0 2024-04-26T22:00:05.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.929+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa850044020 con 0x7fa85c0c45f0 2024-04-26T22:00:05.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.929+0000 7fa84affd640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.930+0000 7fa85a59c640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.930+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6431+0+0 (secure 0 0 0) 0x7fa8500ac660 con 0x7fa85c0c45f0 2024-04-26T22:00:05.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.930+0000 7fa85a59c640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa83c06c8f0 0x7fa83c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa84c006fd0 tx=0x7fa84c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:05.934+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa85004b030 con 0x7fa85c0c45f0 2024-04-26T22:00:06.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.147+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"} v 0) -- 0x7fa85c0507a0 con 0x7fa85c0c45f0 2024-04-26T22:00:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.800+0000 7fa84affd640 1 -- 172.21.15.135:0/681167375 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"}]=0 new fs with metadata pool 4 and data pool 5 v9) ==== 157+0+0 (secure 0 0 0) 0x7fa85007a7d0 con 0x7fa85c0c45f0 2024-04-26T22:00:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T22:00:06.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.810+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa83c06c8f0 msgr2=0x7fa83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.810+0000 7fa860a46640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa83c06c8f0 0x7fa83c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa84c006fd0 tx=0x7fa84c008040 comp rx=0 tx=0).stop 2024-04-26T22:00:06.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.810+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 msgr2=0x7fa85c1512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.810+0000 7fa860a46640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 0x7fa85c1512a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa85002a220 tx=0x7fa8500046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 shutdown_connections 2024-04-26T22:00:06.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa85c13ffb0 0x7fa85c06db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85c0c45f0 0x7fa85c1512a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 --2- 172.21.15.135:0/681167375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa85c074c10 0x7fa85c153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 >> 172.21.15.135:0/681167375 conn(0x7fa85c0ca2f0 msgr2=0x7fa85c0c3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.811+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 shutdown_connections 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.812+0000 7fa860a46640 1 -- 172.21.15.135:0/681167375 wait complete. 2024-04-26T22:00:06.838 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:06.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.935+0000 7f94ca847640 1 Processor -- start 2024-04-26T22:00:06.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.935+0000 7f94ca847640 1 -- start start 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94ca847640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f94c40c90c0 0x7f94c415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94ca847640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94c40c6e00 0x7f94c4158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94ca847640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40cbde0 con 0x7f94c40bfbb0 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94ca847640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f94c415c150 con 0x7f94c40c6e00 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94ca847640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f94c40ca380 con 0x7f94c40c90c0 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94c8de4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94c40c6e00 0x7f94c4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94c8de4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94c40c6e00 0x7f94c4158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46468/0 (socket says 172.21.15.135:46468) 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.936+0000 7f94c8de4640 1 -- 172.21.15.135:0/3536767791 learned_addr learned my addr 172.21.15.135:0/3536767791 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:06.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c3fff640 1 --2- 172.21.15.135:0/3536767791 >> v2:172.21.15.139:3300/0 conn(0x7f94c40c90c0 0x7f94c415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c8de4640 1 -- 172.21.15.135:0/3536767791 >> v2:172.21.15.139:3300/0 conn(0x7f94c40c90c0 msgr2=0x7f94c415bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c8de4640 1 --2- 172.21.15.135:0/3536767791 >> v2:172.21.15.139:3300/0 conn(0x7f94c40c90c0 0x7f94c415bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c8de4640 1 -- 172.21.15.135:0/3536767791 >> v1:172.21.15.139:6789/0 conn(0x7f94c40bfbb0 legacy=0x7f94c40cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c8de4640 1 -- 172.21.15.135:0/3536767791 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c40cc0c0 con 0x7f94c40c6e00 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c3fff640 1 --2- 172.21.15.135:0/3536767791 >> v2:172.21.15.139:3300/0 conn(0x7f94c40c90c0 0x7f94c415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c8de4640 1 --2- 172.21.15.135:0/3536767791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c4158510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94c408e0f0 tx=0x7f94bc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a17e8fe273f932 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c2ffd640 1 -- 172.21.15.135:0/3536767791 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc036070 con 0x7f94c40c6e00 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.937+0000 7f94c2ffd640 1 -- 172.21.15.135:0/3536767791 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc031600 con 0x7f94c40c6e00 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94c2ffd640 1 -- 172.21.15.135:0/3536767791 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0319d0 con 0x7f94c40c6e00 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 -- 172.21.15.135:0/3536767791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 msgr2=0x7f94c4158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 --2- 172.21.15.135:0/3536767791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c4158510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94c408e0f0 tx=0x7f94bc02a610 comp rx=0 tx=0).stop 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 -- 172.21.15.135:0/3536767791 shutdown_connections 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 --2- 172.21.15.135:0/3536767791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c4158510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 --2- 172.21.15.135:0/3536767791 >> v2:172.21.15.139:3300/0 conn(0x7f94c40c90c0 0x7f94c415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 -- 172.21.15.135:0/3536767791 >> 172.21.15.135:0/3536767791 conn(0x7f94c40ae010 msgr2=0x7f94c40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 -- 172.21.15.135:0/3536767791 shutdown_connections 2024-04-26T22:00:06.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.938+0000 7f94ca847640 1 -- 172.21.15.135:0/3536767791 wait complete. 2024-04-26T22:00:06.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.939+0000 7f94ca847640 1 Processor -- start 2024-04-26T22:00:06.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.939+0000 7f94ca847640 1 -- start start 2024-04-26T22:00:06.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94ca847640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40bfbb0 0x7f94c41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94ca847640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94ca847640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40c90c0 0x7f94c414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94ca847640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40c94a0 con 0x7f94c40c6e00 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94ca847640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40cb4e0 con 0x7f94c40bfbb0 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94ca847640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f94c40c9620 con 0x7f94c40c90c0 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c414d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46470/0 (socket says 172.21.15.135:46470) 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 -- 172.21.15.135:0/2106155269 learned_addr learned my addr 172.21.15.135:0/2106155269 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c8de4640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40c90c0 0x7f94c414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c3fff640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40bfbb0 0x7f94c41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 -- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40c90c0 msgr2=0x7f94c414dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40c90c0 0x7f94c414dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 -- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40bfbb0 msgr2=0x7f94c41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40bfbb0 0x7f94c41569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 -- 172.21.15.135:0/2106155269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c40c12b0 con 0x7f94c40c6e00 2024-04-26T22:00:06.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.940+0000 7f94c37fe640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c414d5a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c3f0 tx=0x7f94b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.941+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b80052d0 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.941+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b8005490 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.941+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c40c2200 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.941+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8007260 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.941+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c40c27c0 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.942+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c4076650 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.942+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94b8007400 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.942+0000 7f94c0ff9640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f94b006c8f0 0x7f94b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.943+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6521+0+0 (secure 0 0 0) 0x7f94b808b270 con 0x7f94c40c6e00 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.943+0000 7f94c3fff640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f94b006c8f0 0x7f94b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.943+0000 7f94c3fff640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f94b006c8f0 0x7f94b006edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:06.947+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94c40c12b0 con 0x7f94c40c6e00 2024-04-26T22:00:07.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.125+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94c40507a0 con 0x7f94c40c6e00 2024-04-26T22:00:07.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.126+0000 7f94c0ff9640 1 -- 172.21.15.135:0/2106155269 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+20404 (secure 0 0 0) 0x7f94b805f020 con 0x7f94c40c6e00 2024-04-26T22:00:07.126 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:07.126 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:06.148475+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:07.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f94b006c8f0 msgr2=0x7f94b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f94b006c8f0 0x7f94b006edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4008040 comp rx=0 tx=0).stop 2024-04-26T22:00:07.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 msgr2=0x7f94c414d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c414d5a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c3f0 tx=0x7f94b800c8b0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 shutdown_connections 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f94b006c8f0 0x7f94b006edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40c90c0 0x7f94c414dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c40c6e00 0x7f94c414d5a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 --2- 172.21.15.135:0/2106155269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40bfbb0 0x7f94c41569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 >> 172.21.15.135:0/2106155269 conn(0x7f94c40ae010 msgr2=0x7f94c40ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 shutdown_connections 2024-04-26T22:00:07.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.128+0000 7f94ca847640 1 -- 172.21.15.135:0/2106155269 wait complete. 2024-04-26T22:00:07.139 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:07.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.236+0000 7f80c1b78640 1 Processor -- start 2024-04-26T22:00:07.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.236+0000 7f80c1b78640 1 -- start start 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80c1b78640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80bc0c4250 0x7f80bc0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80c1b78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0c3cd0 0x7f80bc1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80c1b78640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc0ad010 con 0x7f80bc0acc30 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80c1b78640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0c8ba0 con 0x7f80bc0c4250 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80c1b78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc156e50 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80baffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80bc0c4250 0x7f80bc0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80baffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80bc0c4250 0x7f80bc0c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46480/0 (socket says 172.21.15.135:46480) 2024-04-26T22:00:07.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.237+0000 7f80baffd640 1 -- 172.21.15.135:0/4131129460 learned_addr learned my addr 172.21.15.135:0/4131129460 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80bbfff640 1 --2- 172.21.15.135:0/4131129460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0c3cd0 0x7f80bc1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80baffd640 1 -- 172.21.15.135:0/4131129460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0c3cd0 msgr2=0x7f80bc1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80baffd640 1 --2- 172.21.15.135:0/4131129460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0c3cd0 0x7f80bc1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80baffd640 1 -- 172.21.15.135:0/4131129460 >> v1:172.21.15.139:6789/0 conn(0x7f80bc0acc30 legacy=0x7f80bc15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80baffd640 1 -- 172.21.15.135:0/4131129460 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0c8300 con 0x7f80bc0c4250 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80baffd640 1 --2- 172.21.15.135:0/4131129460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c3790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009870 tx=0x7f80b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d730f54dc1b13fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80ba7fc640 1 -- 172.21.15.135:0/4131129460 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0037070 con 0x7f80bc0c4250 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80ba7fc640 1 -- 172.21.15.135:0/4131129460 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b0032440 con 0x7f80bc0c4250 2024-04-26T22:00:07.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.238+0000 7f80ba7fc640 1 -- 172.21.15.135:0/4131129460 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0032810 con 0x7f80bc0c4250 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 -- 172.21.15.135:0/4131129460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c4250 msgr2=0x7f80bc0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 --2- 172.21.15.135:0/4131129460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c3790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009870 tx=0x7f80b002a180 comp rx=0 tx=0).stop 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 -- 172.21.15.135:0/4131129460 shutdown_connections 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 --2- 172.21.15.135:0/4131129460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0c3cd0 0x7f80bc1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 --2- 172.21.15.135:0/4131129460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c3790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 -- 172.21.15.135:0/4131129460 >> 172.21.15.135:0/4131129460 conn(0x7f80bc0c6e60 msgr2=0x7f80bc0c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 -- 172.21.15.135:0/4131129460 shutdown_connections 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.239+0000 7f80c1b78640 1 -- 172.21.15.135:0/4131129460 wait complete. 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.240+0000 7f80c1b78640 1 Processor -- start 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.240+0000 7f80c1b78640 1 -- start start 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80c1b78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0acc30 0x7f80bc070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80c1b78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 0x7f80bc1ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80c1b78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80c1b78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0abec0 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80c1b78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0c88c0 con 0x7f80bc0c4250 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80c1b78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0ac040 con 0x7f80bc0acc30 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 0x7f80bc1ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 0x7f80bc1ed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46490/0 (socket says 172.21.15.135:46490) 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 -- 172.21.15.135:0/3404684503 learned_addr learned my addr 172.21.15.135:0/3404684503 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80bb7fe640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0acc30 0x7f80bc070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 -- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0acc30 msgr2=0x7f80bc070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80bbfff640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0acc30 0x7f80bc070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 -- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80bc0c4250 msgr2=0x7f80bc0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 -- 172.21.15.135:0/3404684503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc15a750 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.241+0000 7f80baffd640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 0x7f80bc1ed4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80b00090b0 tx=0x7f80b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.242+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0037040 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.242+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b0042070 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.242+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc1eca20 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.242+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0030bb0 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.242+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc06c720 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.243+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc0507a0 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.243+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80b0047460 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.244+0000 7f80a3fff640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f80a806c8f0 0x7f80a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.244+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6521+0+0 (secure 0 0 0) 0x7f80b00ac9f0 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.244+0000 7f80bb7fe640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f80a806c8f0 0x7f80a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.244+0000 7f80bb7fe640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f80a806c8f0 0x7f80a806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f80ac004ea0 tx=0x7f80ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.248+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80b007ab80 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.422+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80bc0c46f0 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.424+0000 7f80a3fff640 1 -- 172.21.15.135:0/3404684503 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+20404 (secure 0 0 0) 0x7f80b007a480 con 0x7f80bc0c3cd0 2024-04-26T22:00:07.424 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:07.424 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:06.148475+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f80a806c8f0 msgr2=0x7f80a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f80a806c8f0 0x7f80a806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f80ac004ea0 tx=0x7f80ac009040 comp rx=0 tx=0).stop 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 msgr2=0x7f80bc1ed4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 0x7f80bc1ed4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80b00090b0 tx=0x7f80b00046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 shutdown_connections 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f80a806c8f0 0x7f80a806edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80bc0c4250 0x7f80bc0c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80bc0c3cd0 0x7f80bc1ed4e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 --2- 172.21.15.135:0/3404684503 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80bc0acc30 0x7f80bc070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 >> 172.21.15.135:0/3404684503 conn(0x7f80bc0c6e60 msgr2=0x7f80bc0ab470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 shutdown_connections 2024-04-26T22:00:07.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.426+0000 7f80c1b78640 1 -- 172.21.15.135:0/3404684503 wait complete. 2024-04-26T22:00:07.437 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b standby_count_wanted 0 2024-04-26T22:00:07.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 Processor -- start 2024-04-26T22:00:07.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 -- start start 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89040740e0 0x7f8904157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89040ca3a0 0x7f89040c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890415a530 con 0x7f89040c6e20 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8904072930 con 0x7f89040740e0 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.536+0000 7f890acda640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f890415a810 con 0x7f89040ca3a0 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89040740e0 0x7f8904157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89040740e0 0x7f8904157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46500/0 (socket says 172.21.15.135:46500) 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 -- 172.21.15.135:0/981342124 learned_addr learned my addr 172.21.15.135:0/981342124 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8909277640 1 --2- 172.21.15.135:0/981342124 >> v2:172.21.15.139:3300/0 conn(0x7f89040ca3a0 0x7f89040c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 -- 172.21.15.135:0/981342124 >> v2:172.21.15.139:3300/0 conn(0x7f89040ca3a0 msgr2=0x7f89040c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 --2- 172.21.15.135:0/981342124 >> v2:172.21.15.139:3300/0 conn(0x7f89040ca3a0 0x7f89040c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 -- 172.21.15.135:0/981342124 >> v1:172.21.15.139:6789/0 conn(0x7f89040c6e20 legacy=0x7f8904156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 -- 172.21.15.135:0/981342124 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890415aaf0 con 0x7f89040740e0 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8909277640 1 --2- 172.21.15.135:0/981342124 >> v2:172.21.15.139:3300/0 conn(0x7f89040ca3a0 0x7f89040c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.537+0000 7f8903fff640 1 --2- 172.21.15.135:0/981342124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040740e0 0x7f8904157630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89040af150 tx=0x7f88ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4013ee29b8a35cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f89037fe640 1 -- 172.21.15.135:0/981342124 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec037070 con 0x7f89040740e0 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f89037fe640 1 -- 172.21.15.135:0/981342124 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec032440 con 0x7f89040740e0 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f89037fe640 1 -- 172.21.15.135:0/981342124 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec032810 con 0x7f89040740e0 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 -- 172.21.15.135:0/981342124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040740e0 msgr2=0x7f8904157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 --2- 172.21.15.135:0/981342124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040740e0 0x7f8904157630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89040af150 tx=0x7f88ec02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 -- 172.21.15.135:0/981342124 shutdown_connections 2024-04-26T22:00:07.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 --2- 172.21.15.135:0/981342124 >> v2:172.21.15.139:3300/0 conn(0x7f89040ca3a0 0x7f89040c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 --2- 172.21.15.135:0/981342124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040740e0 0x7f8904157630 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 -- 172.21.15.135:0/981342124 >> 172.21.15.135:0/981342124 conn(0x7f89040ae0b0 msgr2=0x7f89040ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:07.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 -- 172.21.15.135:0/981342124 shutdown_connections 2024-04-26T22:00:07.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.538+0000 7f890acda640 1 -- 172.21.15.135:0/981342124 wait complete. 2024-04-26T22:00:07.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.539+0000 7f890acda640 1 Processor -- start 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.539+0000 7f890acda640 1 -- start start 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f890acda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89040740e0 0x7f8904152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f890acda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89040c6e20 0x7f89041edd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f890acda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 0x7f8904146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f890acda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8904154540 con 0x7f89040ca3a0 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f890acda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89041543c0 con 0x7f89040740e0 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f890acda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f890415a0c0 con 0x7f89040c6e20 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 0x7f8904146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 0x7f8904146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46514/0 (socket says 172.21.15.135:46514) 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 -- 172.21.15.135:0/4269477422 learned_addr learned my addr 172.21.15.135:0/4269477422 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8903fff640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89040c6e20 0x7f89041edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8908a76640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89040740e0 0x7f8904152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 -- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89040c6e20 msgr2=0x7f89041edd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89040c6e20 0x7f89041edd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 -- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89040740e0 msgr2=0x7f8904152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89040740e0 0x7f8904152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.540+0000 7f8909277640 1 -- 172.21.15.135:0/4269477422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89041ed6c0 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.541+0000 7f8909277640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 0x7f8904146890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88f400c7c0 tx=0x7f88f400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.541+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4015020 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.541+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f4004230 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.541+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4007620 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.541+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89040c1570 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.541+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041533e0 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.542+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89040766c0 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.542+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88f4018020 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.543+0000 7f89017fa640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f88e406c8f0 0x7f88e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.543+0000 7f8908a76640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f88e406c8f0 0x7f88e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.543+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6521+0+0 (secure 0 0 0) 0x7f88f408bea0 con 0x7f89040ca3a0 2024-04-26T22:00:07.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.543+0000 7f8908a76640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f88e406c8f0 0x7f88e406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005520 tx=0x7f88fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:07.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.547+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88f401f020 con 0x7f89040ca3a0 2024-04-26T22:00:07.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:07.757+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f89040507a0 con 0x7f89040ca3a0 2024-04-26T22:00:08.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.758+0000 7f89017fa640 1 -- 172.21.15.135:0/4269477422 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"}]=0 v11) ==== 113+0+0 (secure 0 0 0) 0x7f88f405a0d0 con 0x7f89040ca3a0 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f88e406c8f0 msgr2=0x7f88e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f88e406c8f0 0x7f88e406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005520 tx=0x7f88fc009040 comp rx=0 tx=0).stop 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 msgr2=0x7f8904146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 0x7f8904146890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88f400c7c0 tx=0x7f88f400cc80 comp rx=0 tx=0).stop 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 shutdown_connections 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f88e406c8f0 0x7f88e406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89040ca3a0 0x7f8904146890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89040c6e20 0x7f89041edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 --2- 172.21.15.135:0/4269477422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89040740e0 0x7f8904152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 >> 172.21.15.135:0/4269477422 conn(0x7f89040ae0b0 msgr2=0x7f8904073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:08.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 shutdown_connections 2024-04-26T22:00:08.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.762+0000 7f890acda640 1 -- 172.21.15.135:0/4269477422 wait complete. 2024-04-26T22:00:08.775 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-04-26T22:00:08.775 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b allow_standby_replay false 2024-04-26T22:00:08.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.882+0000 7ff1bd79e640 1 Processor -- start 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.882+0000 7ff1bd79e640 1 -- start start 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1bd79e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff1b80ada10 0x7ff1b80c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1bd79e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1b8070270 0x7ff1b80c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1bd79e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b80c5050 con 0x7ff1b80c3570 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1bd79e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b80c8600 con 0x7ff1b80ada10 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1bd79e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b806fd40 con 0x7ff1b8070270 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1b7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff1b80ada10 0x7ff1b80c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1b7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff1b80ada10 0x7ff1b80c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40156/0 (socket says 172.21.15.135:40156) 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1b77fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff1b80c3570 0x7ff1b8159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52846/0 (socket says 172.21.15.135:52846) 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1b7fff640 1 -- 172.21.15.135:0/1607533513 learned_addr learned my addr 172.21.15.135:0/1607533513 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:08.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.883+0000 7ff1bcf9d640 1 --2- 172.21.15.135:0/1607533513 >> v2:172.21.15.139:3300/0 conn(0x7ff1b8070270 0x7ff1b80c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b7fff640 1 -- 172.21.15.135:0/1607533513 >> v2:172.21.15.139:3300/0 conn(0x7ff1b8070270 msgr2=0x7ff1b80c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:08.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b7fff640 1 --2- 172.21.15.135:0/1607533513 >> v2:172.21.15.139:3300/0 conn(0x7ff1b8070270 0x7ff1b80c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b7fff640 1 -- 172.21.15.135:0/1607533513 >> v1:172.21.15.135:6789/0 conn(0x7ff1b80c3570 legacy=0x7ff1b8159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b6ffd640 1 -- 172.21.15.135:0/1607533513 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067241421 0 0) 0x7ff1ac001110 con 0x7ff1b80c3570 2024-04-26T22:00:08.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b7fff640 1 -- 172.21.15.135:0/1607533513 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b806ff90 con 0x7ff1b80ada10 2024-04-26T22:00:08.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b7fff640 1 --2- 172.21.15.135:0/1607533513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b80ada10 0x7ff1b80c4b10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80c6e10 tx=0x7ff1a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ad6aa86be259 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:08.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b6ffd640 1 -- 172.21.15.135:0/1607533513 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8037070 con 0x7ff1b80ada10 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b6ffd640 1 -- 172.21.15.135:0/1607533513 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8032440 con 0x7ff1b80ada10 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.884+0000 7ff1b6ffd640 1 -- 172.21.15.135:0/1607533513 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8032810 con 0x7ff1b80ada10 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 -- 172.21.15.135:0/1607533513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b80ada10 msgr2=0x7ff1b80c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/1607533513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b80ada10 0x7ff1b80c4b10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80c6e10 tx=0x7ff1a802a270 comp rx=0 tx=0).stop 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 -- 172.21.15.135:0/1607533513 shutdown_connections 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/1607533513 >> v2:172.21.15.139:3300/0 conn(0x7ff1b8070270 0x7ff1b80c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/1607533513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b80ada10 0x7ff1b80c4b10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 -- 172.21.15.135:0/1607533513 >> 172.21.15.135:0/1607533513 conn(0x7ff1b80aa8d0 msgr2=0x7ff1b80aace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 -- 172.21.15.135:0/1607533513 shutdown_connections 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.885+0000 7ff1bd79e640 1 -- 172.21.15.135:0/1607533513 wait complete. 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.886+0000 7ff1bd79e640 1 Processor -- start 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.886+0000 7ff1bd79e640 1 -- start start 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1bd79e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 0x7ff1b81ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1bd79e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b80ada10 0x7ff1b81ed960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1bd79e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b80c3570 0x7ff1b81edea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1bd79e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b80c7350 con 0x7ff1b8070270 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1bd79e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b80c4680 con 0x7ff1b80c3570 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1bd79e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b80c7630 con 0x7ff1b80ada10 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 0x7ff1b81ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 0x7ff1b81ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40166/0 (socket says 172.21.15.135:40166) 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 -- 172.21.15.135:0/3785718767 learned_addr learned my addr 172.21.15.135:0/3785718767 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 -- 172.21.15.135:0/3785718767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b80ada10 msgr2=0x7ff1b81ed960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b80ada10 0x7ff1b81ed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 -- 172.21.15.135:0/3785718767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b80c3570 msgr2=0x7ff1b81edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b80c3570 0x7ff1b81edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.887+0000 7ff1b7fff640 1 -- 172.21.15.135:0/3785718767 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b8071c40 con 0x7ff1b8070270 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.888+0000 7ff1b7fff640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 0x7ff1b81ef8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff1a802a2b0 tx=0x7ff1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:08.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.888+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8041020 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.888+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a80306c0 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.888+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b8071f20 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.888+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8037040 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.888+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b80ac9d0 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.889+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b80720f0 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.889+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1a8044020 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.889+0000 7ff1b4ff9640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1a406c920 0x7ff1a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.890+0000 7ff1b77fe640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1a406c920 0x7ff1a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.890+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ff1a80ac620 con 0x7ff1b8070270 2024-04-26T22:00:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.890+0000 7ff1b77fe640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1a406c920 0x7ff1a406ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1b81ef5e0 tx=0x7ff1ac001310 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:08.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:08.894+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1a804b030 con 0x7ff1b8070270 2024-04-26T22:00:09.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.108+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff1b80506c0 con 0x7ff1b8070270 2024-04-26T22:00:09.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.830+0000 7ff1b4ff9640 1 -- 172.21.15.135:0/3785718767 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"}]=0 v12) ==== 117+0+0 (secure 0 0 0) 0x7ff1a807a7b0 con 0x7ff1b8070270 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1a406c920 msgr2=0x7ff1a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1a406c920 0x7ff1a406ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1b81ef5e0 tx=0x7ff1ac001310 comp rx=0 tx=0).stop 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 msgr2=0x7ff1b81ef8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 0x7ff1b81ef8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff1a802a2b0 tx=0x7ff1a80046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 shutdown_connections 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1a406c920 0x7ff1a406ede0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b80c3570 0x7ff1b81edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b80ada10 0x7ff1b81ed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 --2- 172.21.15.135:0/3785718767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b8070270 0x7ff1b81ef8f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 >> 172.21.15.135:0/3785718767 conn(0x7ff1b80aa8d0 msgr2=0x7ff1b80aacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:09.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 shutdown_connections 2024-04-26T22:00:09.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.833+0000 7ff1bd79e640 1 -- 172.21.15.135:0/3785718767 wait complete. 2024-04-26T22:00:09.846 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b session_timeout 300 2024-04-26T22:00:09.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 Processor -- start 2024-04-26T22:00:09.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 -- start start 2024-04-26T22:00:09.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c080aecf0 0x7f8c080c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8c080c4820 0x7f8c08073a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c08156f60 con 0x7f8c080c3d40 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8c08157240 con 0x7f8c080aecf0 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0f01d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8c08157520 con 0x7f8c080c4820 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.952+0000 7f8c0cdb9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c080aecf0 0x7f8c080c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c080aecf0 0x7f8c080c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40176/0 (socket says 172.21.15.135:40176) 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0d5ba640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8c080c3d40 0x7f8c08073fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52860/0 (socket says 172.21.15.135:52860) 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 -- 172.21.15.135:0/1916561462 learned_addr learned my addr 172.21.15.135:0/1916561462 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c07fff640 1 --2- 172.21.15.135:0/1916561462 >> v2:172.21.15.139:3300/0 conn(0x7f8c080c4820 0x7f8c08073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 -- 172.21.15.135:0/1916561462 >> v2:172.21.15.139:3300/0 conn(0x7f8c080c4820 msgr2=0x7f8c08073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 --2- 172.21.15.135:0/1916561462 >> v2:172.21.15.139:3300/0 conn(0x7f8c080c4820 0x7f8c08073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 -- 172.21.15.135:0/1916561462 >> v1:172.21.15.135:6789/0 conn(0x7f8c080c3d40 legacy=0x7f8c08073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 -- 172.21.15.135:0/1916561462 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c08157800 con 0x7f8c080aecf0 2024-04-26T22:00:09.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c077fe640 1 -- 172.21.15.135:0/1916561462 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160681250 0 0) 0x7f8c08156f60 con 0x7f8c080c3d40 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c07fff640 1 --2- 172.21.15.135:0/1916561462 >> v2:172.21.15.139:3300/0 conn(0x7f8c080c4820 0x7f8c08073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.953+0000 7f8c0cdb9640 1 --2- 172.21.15.135:0/1916561462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080aecf0 0x7f8c080c3800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8c080ca330 tx=0x7f8bfc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2edcdc8c90dc9c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c077fe640 1 -- 172.21.15.135:0/1916561462 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc037070 con 0x7f8c080aecf0 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c077fe640 1 -- 172.21.15.135:0/1916561462 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc032440 con 0x7f8c080aecf0 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c077fe640 1 -- 172.21.15.135:0/1916561462 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc032810 con 0x7f8c080aecf0 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 -- 172.21.15.135:0/1916561462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080aecf0 msgr2=0x7f8c080c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/1916561462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080aecf0 0x7f8c080c3800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8c080ca330 tx=0x7f8bfc02a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 -- 172.21.15.135:0/1916561462 shutdown_connections 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/1916561462 >> v2:172.21.15.139:3300/0 conn(0x7f8c080c4820 0x7f8c08073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/1916561462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080aecf0 0x7f8c080c3800 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 -- 172.21.15.135:0/1916561462 >> 172.21.15.135:0/1916561462 conn(0x7f8c080c6ed0 msgr2=0x7f8c080c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 -- 172.21.15.135:0/1916561462 shutdown_connections 2024-04-26T22:00:09.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.954+0000 7f8c0f01d640 1 -- 172.21.15.135:0/1916561462 wait complete. 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.955+0000 7f8c0f01d640 1 Processor -- start 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.955+0000 7f8c0f01d640 1 -- start start 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0f01d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c080aecf0 0x7f8c080adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0f01d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 0x7f8c0814e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0f01d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c080c4820 0x7f8c08148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0f01d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08156c10 con 0x7f8c080c3d40 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0f01d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8c080746c0 con 0x7f8c080c4820 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0f01d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8c080af0d0 con 0x7f8c080aecf0 2024-04-26T22:00:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 0x7f8c0814e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 0x7f8c0814e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40190/0 (socket says 172.21.15.135:40190) 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c07fff640 1 -- 172.21.15.135:0/242012311 learned_addr learned my addr 172.21.15.135:0/242012311 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0d5ba640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c080c4820 0x7f8c08148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.956+0000 7f8c0cdb9640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c080aecf0 0x7f8c080adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c07fff640 1 -- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c080aecf0 msgr2=0x7f8c080adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c07fff640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c080aecf0 0x7f8c080adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c07fff640 1 -- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c080c4820 msgr2=0x7f8c08148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c07fff640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c080c4820 0x7f8c08148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c07fff640 1 -- 172.21.15.135:0/242012311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c080c95e0 con 0x7f8c080c3d40 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c07fff640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 0x7f8c0814e920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d7d0 tx=0x7f8bf400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004030 con 0x7f8c080c3d40 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf40041d0 con 0x7f8c080c3d40 2024-04-26T22:00:09.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c080c8d40 con 0x7f8c080c3d40 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40096a0 con 0x7f8c080c3d40 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.957+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c080c9300 con 0x7f8c080c3d40 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.958+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c080507a0 con 0x7f8c080c3d40 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.958+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bf4009840 con 0x7f8c080c3d40 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.959+0000 7f8c057fa640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8bec06c920 0x7f8bec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.959+0000 7f8c0cdb9640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8bec06c920 0x7f8bec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.959+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f8bf408bed0 con 0x7f8c080c3d40 2024-04-26T22:00:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.959+0000 7f8c0cdb9640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8bec06c920 0x7f8bec06ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc034000 tx=0x7f8bfc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:09.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:09.963+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8bf4059a10 con 0x7f8c080c3d40 2024-04-26T22:00:10.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.176+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8c080c4600 con 0x7f8c080c3d40 2024-04-26T22:00:10.609 INFO:tasks.ceph.mds.c.smithi135.stderr:2024-04-26T22:00:10.610+0000 7f7e8a36f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:00:10.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.831+0000 7f8c057fa640 1 -- 172.21.15.135:0/242012311 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"}]=0 v13) ==== 110+0+0 (secure 0 0 0) 0x7f8bf405dd10 con 0x7f8c080c3d40 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8bec06c920 msgr2=0x7f8bec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8bec06c920 0x7f8bec06ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc034000 tx=0x7f8bfc02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 msgr2=0x7f8c0814e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 0x7f8c0814e920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d7d0 tx=0x7f8bf400dc90 comp rx=0 tx=0).stop 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 shutdown_connections 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8bec06c920 0x7f8bec06ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c080c4820 0x7f8c08148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080c3d40 0x7f8c0814e920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 --2- 172.21.15.135:0/242012311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c080aecf0 0x7f8c080adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 >> 172.21.15.135:0/242012311 conn(0x7f8c080c6ed0 msgr2=0x7f8c080c7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 shutdown_connections 2024-04-26T22:00:10.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.834+0000 7f8c0f01d640 1 -- 172.21.15.135:0/242012311 wait complete. 2024-04-26T22:00:10.845 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'b' 2024-04-26T22:00:10.846 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:10.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.945+0000 7fd297e47640 1 Processor -- start 2024-04-26T22:00:10.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.945+0000 7fd297e47640 1 -- start start 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd297e47640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2900aecf0 0x7fd2900c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd297e47640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd290156f60 con 0x7fd2900c4820 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd297e47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd290157240 con 0x7fd2900c3d40 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd297e47640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd290157520 con 0x7fd2900aecf0 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd295be3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2900aecf0 0x7fd2900c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd295be3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2900aecf0 0x7fd2900c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40202/0 (socket says 172.21.15.135:40202) 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd295be3640 1 -- 172.21.15.135:0/1476954891 learned_addr learned my addr 172.21.15.135:0/1476954891 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd2953e2640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd2900c4820 0x7fd290073a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52862/0 (socket says 172.21.15.135:52862) 2024-04-26T22:00:10.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd295be3640 1 -- 172.21.15.135:0/1476954891 >> v1:172.21.15.139:6789/0 conn(0x7fd2900c3d40 legacy=0x7fd290074180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd295be3640 1 -- 172.21.15.135:0/1476954891 >> v1:172.21.15.135:6789/0 conn(0x7fd2900c4820 legacy=0x7fd290073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd295be3640 1 -- 172.21.15.135:0/1476954891 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290157800 con 0x7fd2900aecf0 2024-04-26T22:00:10.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.946+0000 7fd294be1640 1 -- 172.21.15.135:0/1476954891 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891037492 0 0) 0x7fd290156f60 con 0x7fd2900c4820 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.947+0000 7fd295be3640 1 --2- 172.21.15.135:0/1476954891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd2900c3800 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009870 tx=0x7fd28c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688a258469235947 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.947+0000 7fd294be1640 1 -- 172.21.15.135:0/1476954891 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c037070 con 0x7fd2900aecf0 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.947+0000 7fd294be1640 1 -- 172.21.15.135:0/1476954891 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c032440 con 0x7fd2900aecf0 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.947+0000 7fd294be1640 1 -- 172.21.15.135:0/1476954891 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c032810 con 0x7fd2900aecf0 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.947+0000 7fd297e47640 1 -- 172.21.15.135:0/1476954891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 msgr2=0x7fd2900c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.948+0000 7fd297e47640 1 --2- 172.21.15.135:0/1476954891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd2900c3800 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009870 tx=0x7fd28c02a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.948+0000 7fd297e47640 1 -- 172.21.15.135:0/1476954891 shutdown_connections 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.948+0000 7fd297e47640 1 --2- 172.21.15.135:0/1476954891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd2900c3800 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.948+0000 7fd297e47640 1 -- 172.21.15.135:0/1476954891 >> 172.21.15.135:0/1476954891 conn(0x7fd2900c6ed0 msgr2=0x7fd2900c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:10.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.948+0000 7fd297e47640 1 -- 172.21.15.135:0/1476954891 shutdown_connections 2024-04-26T22:00:10.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.948+0000 7fd297e47640 1 -- 172.21.15.135:0/1476954891 wait complete. 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 Processor -- start 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 -- start start 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd29014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2900c3d40 0x7fd29014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2900c4820 0x7fd2901e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900af0d0 con 0x7fd2900aecf0 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd290156c10 con 0x7fd2900c3d40 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd297e47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd2900af250 con 0x7fd2900c4820 2024-04-26T22:00:10.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd295be3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd29014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd295be3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd29014e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40216/0 (socket says 172.21.15.135:40216) 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.949+0000 7fd295be3640 1 -- 172.21.15.135:0/2131050956 learned_addr learned my addr 172.21.15.135:0/2131050956 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd295be3640 1 -- 172.21.15.135:0/2131050956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2900c4820 msgr2=0x7fd2901e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd295be3640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2900c4820 0x7fd2901e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd295be3640 1 -- 172.21.15.135:0/2131050956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2900c3d40 msgr2=0x7fd29014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd295be3640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2900c3d40 0x7fd29014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd295be3640 1 -- 172.21.15.135:0/2131050956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290150d00 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd295be3640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd29014e1b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd28c000c00 tx=0x7fd28c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c037040 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c042070 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd290150fe0 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c030bb0 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.950+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2901515a0 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.951+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2900ad270 con 0x7fd2900aecf0 2024-04-26T22:00:10.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.952+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd28c047460 con 0x7fd2900aecf0 2024-04-26T22:00:10.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.952+0000 7fd2867fc640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd27406c920 0x7fd27406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:10.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.952+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd28c0aca60 con 0x7fd2900aecf0 2024-04-26T22:00:10.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.952+0000 7fd2953e2640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd27406c920 0x7fd27406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:10.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.952+0000 7fd2953e2640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd27406c920 0x7fd27406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd2800079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:10.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:10.956+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd28c045360 con 0x7fd2900aecf0 2024-04-26T22:00:11.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.168+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2900507a0 con 0x7fd2900aecf0 2024-04-26T22:00:11.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.169+0000 7fd2867fc640 1 -- 172.21.15.135:0/2131050956 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7fd28c07abf0 con 0x7fd2900aecf0 2024-04-26T22:00:11.169 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:11.169 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:11.169 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:11.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd27406c920 msgr2=0x7fd27406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd27406c920 0x7fd27406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd2800079c0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 msgr2=0x7fd29014e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd29014e1b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd28c000c00 tx=0x7fd28c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 shutdown_connections 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd27406c920 0x7fd27406ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2900c4820 0x7fd2901e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2900c3d40 0x7fd29014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 --2- 172.21.15.135:0/2131050956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2900aecf0 0x7fd29014e1b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 >> 172.21.15.135:0/2131050956 conn(0x7fd2900c6ed0 msgr2=0x7fd2900ab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 shutdown_connections 2024-04-26T22:00:11.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.171+0000 7fd297e47640 1 -- 172.21.15.135:0/2131050956 wait complete. 2024-04-26T22:00:11.182 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:11.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.279+0000 7f3376d78640 1 Processor -- start 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.279+0000 7f3376d78640 1 -- start start 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.279+0000 7f3376d78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 0x7f33700720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.279+0000 7f3376d78640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 0x7f33700c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3376d78640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3370154ba0 con 0x7f33700c40e0 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3376d78640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3370154e80 con 0x7f33700c46e0 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3376d78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3370155160 con 0x7f33700c3060 2024-04-26T22:00:11.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3375315640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 0x7f33700c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3374b14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 0x7f33700720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3375315640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 0x7f33700c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33172/0 (socket says 172.21.15.135:33172) 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f336ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f33700c40e0 0x7f3370151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57202/0 (socket says 172.21.15.135:57202) 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3374b14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 0x7f33700720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:33686/0 (socket says 172.21.15.135:33686) 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3375315640 1 -- 172.21.15.135:0/1379653596 learned_addr learned my addr 172.21.15.135:0/1379653596 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3374b14640 1 -- 172.21.15.135:0/1379653596 >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 msgr2=0x7f33700c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3374b14640 1 --2- 172.21.15.135:0/1379653596 >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 0x7f33700c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3374b14640 1 -- 172.21.15.135:0/1379653596 >> v1:172.21.15.139:6789/0 conn(0x7f33700c40e0 legacy=0x7f3370151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3374b14640 1 -- 172.21.15.135:0/1379653596 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3370155440 con 0x7f33700c3060 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.280+0000 7f3375315640 1 --2- 172.21.15.135:0/1379653596 >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 0x7f33700c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.281+0000 7f3374b14640 1 --2- 172.21.15.135:0/1379653596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 0x7f33700720a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33640097a0 tx=0x7f336402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d1147c21edd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.281+0000 7f336f7fe640 1 -- 172.21.15.135:0/1379653596 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364037070 con 0x7f33700c3060 2024-04-26T22:00:11.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.281+0000 7f336f7fe640 1 -- 172.21.15.135:0/1379653596 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336402ae50 con 0x7f33700c3060 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.281+0000 7f336f7fe640 1 -- 172.21.15.135:0/1379653596 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364032660 con 0x7f33700c3060 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 -- 172.21.15.135:0/1379653596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 msgr2=0x7f33700720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 --2- 172.21.15.135:0/1379653596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 0x7f33700720a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33640097a0 tx=0x7f336402a100 comp rx=0 tx=0).stop 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 -- 172.21.15.135:0/1379653596 shutdown_connections 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 --2- 172.21.15.135:0/1379653596 >> v2:172.21.15.139:3300/0 conn(0x7f33700c46e0 0x7f33700c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 --2- 172.21.15.135:0/1379653596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c3060 0x7f33700720a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 -- 172.21.15.135:0/1379653596 >> 172.21.15.135:0/1379653596 conn(0x7f337006c310 msgr2=0x7f337006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 -- 172.21.15.135:0/1379653596 shutdown_connections 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.282+0000 7f3376d78640 1 -- 172.21.15.135:0/1379653596 wait complete. 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.283+0000 7f3376d78640 1 Processor -- start 2024-04-26T22:00:11.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.283+0000 7f3376d78640 1 -- start start 2024-04-26T22:00:11.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 0x7f3370158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33700c40e0 0x7f33700c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c46e0 0x7f33700c1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33700c3440 con 0x7f33700c3060 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33700c3aa0 con 0x7f33700c40e0 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33700c35c0 con 0x7f33700c46e0 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 0x7f3370158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 0x7f3370158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40218/0 (socket says 172.21.15.135:40218) 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 -- 172.21.15.135:0/3412429627 learned_addr learned my addr 172.21.15.135:0/3412429627 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3375315640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c46e0 0x7f33700c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f336ffff640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33700c40e0 0x7f33700c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 -- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c46e0 msgr2=0x7f33700c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c46e0 0x7f33700c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 -- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33700c40e0 msgr2=0x7f33700c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33700c40e0 0x7f33700c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 -- 172.21.15.135:0/3412429627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33701eabe0 con 0x7f33700c3060 2024-04-26T22:00:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3374b14640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 0x7f3370158240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3364035040 tx=0x7f33640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364041020 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336402ac50 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364037040 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337014ebd0 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.284+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33701eb140 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.285+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33700507a0 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.286+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3364044020 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.286+0000 7f336d7fa640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f335406c8f0 0x7f335406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.286+0000 7f336ffff640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f335406c8f0 0x7f335406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.286+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f33640ac260 con 0x7f33700c3060 2024-04-26T22:00:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.287+0000 7f336ffff640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f335406c8f0 0x7f335406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3370153980 tx=0x7f335c007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.290+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f336404b030 con 0x7f33700c3060 2024-04-26T22:00:11.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.466+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f337014f770 con 0x7f33700c3060 2024-04-26T22:00:11.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.467+0000 7f336d7fa640 1 -- 172.21.15.135:0/3412429627 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f3364080030 con 0x7f33700c3060 2024-04-26T22:00:11.467 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:11.467 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:11.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f335406c8f0 msgr2=0x7f335406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f335406c8f0 0x7f335406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3370153980 tx=0x7f335c007e80 comp rx=0 tx=0).stop 2024-04-26T22:00:11.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 msgr2=0x7f3370158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 0x7f3370158240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3364035040 tx=0x7f33640046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 shutdown_connections 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f335406c8f0 0x7f335406edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33700c46e0 0x7f33700c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33700c40e0 0x7f33700c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 --2- 172.21.15.135:0/3412429627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33700c3060 0x7f3370158240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 >> 172.21.15.135:0/3412429627 conn(0x7f337006c310 msgr2=0x7f337006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 shutdown_connections 2024-04-26T22:00:11.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.469+0000 7f3376d78640 1 -- 172.21.15.135:0/3412429627 wait complete. 2024-04-26T22:00:11.480 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:00:11.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:11.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.577+0000 7f79c57e2640 1 Processor -- start 2024-04-26T22:00:11.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.577+0000 7f79c57e2640 1 -- start start 2024-04-26T22:00:11.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79c57e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c00c3c70 0x7f79c00c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79c57e2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f79c0157010 0x7f79c01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79c57e2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c015a230 con 0x7f79c013fe20 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79c57e2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00c39b0 con 0x7f79c0157010 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79c57e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79c015b120 con 0x7f79c00c3c70 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f79c0157010 0x7f79c01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f79c0157010 0x7f79c01574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40226/0 (socket says 172.21.15.135:40226) 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 -- 172.21.15.135:0/3576610102 learned_addr learned my addr 172.21.15.135:0/3576610102 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79be7fc640 1 --2- 172.21.15.135:0/3576610102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c00c3c70 0x7f79c00c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 -- 172.21.15.135:0/3576610102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c00c3c70 msgr2=0x7f79c00c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 --2- 172.21.15.135:0/3576610102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c00c3c70 0x7f79c00c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 -- 172.21.15.135:0/3576610102 >> v1:172.21.15.139:6789/0 conn(0x7f79c013fe20 legacy=0x7f79c006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:11.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.578+0000 7f79bf7fe640 1 -- 172.21.15.135:0/3576610102 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c015a510 con 0x7f79c0157010 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.579+0000 7f79bf7fe640 1 --2- 172.21.15.135:0/3576610102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c0157010 0x7f79c01574a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009870 tx=0x7f79b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f4ee1478a4bc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.579+0000 7f79bdffb640 1 -- 172.21.15.135:0/3576610102 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8037070 con 0x7f79c0157010 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.579+0000 7f79bdffb640 1 -- 172.21.15.135:0/3576610102 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8032440 con 0x7f79c0157010 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.579+0000 7f79bdffb640 1 -- 172.21.15.135:0/3576610102 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8032810 con 0x7f79c0157010 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 -- 172.21.15.135:0/3576610102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c0157010 msgr2=0x7f79c01574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 --2- 172.21.15.135:0/3576610102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c0157010 0x7f79c01574a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009870 tx=0x7f79b802a210 comp rx=0 tx=0).stop 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 -- 172.21.15.135:0/3576610102 shutdown_connections 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 --2- 172.21.15.135:0/3576610102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c0157010 0x7f79c01574a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 --2- 172.21.15.135:0/3576610102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c00c3c70 0x7f79c00c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 -- 172.21.15.135:0/3576610102 >> 172.21.15.135:0/3576610102 conn(0x7f79c00c5d70 msgr2=0x7f79c00c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 -- 172.21.15.135:0/3576610102 shutdown_connections 2024-04-26T22:00:11.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.580+0000 7f79c57e2640 1 -- 172.21.15.135:0/3576610102 wait complete. 2024-04-26T22:00:11.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.581+0000 7f79c57e2640 1 Processor -- start 2024-04-26T22:00:11.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.581+0000 7f79c57e2640 1 -- start start 2024-04-26T22:00:11.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 0x7f79c00c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c013fe20 0x7f79c00c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c0157010 0x7f79c00c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f79c015adb0 con 0x7f79c00c3c70 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f79c015aad0 con 0x7f79c013fe20 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79c0140200 con 0x7f79c0157010 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 0x7f79c00c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 0x7f79c00c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40242/0 (socket says 172.21.15.135:40242) 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 -- 172.21.15.135:0/2621872863 learned_addr learned my addr 172.21.15.135:0/2621872863 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79be7fc640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c013fe20 0x7f79c00c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79bf7fe640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c0157010 0x7f79c00c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 -- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c0157010 msgr2=0x7f79c00c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c0157010 0x7f79c00c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 -- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c013fe20 msgr2=0x7f79c00c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c013fe20 0x7f79c00c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 -- 172.21.15.135:0/2621872863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c00c78e0 con 0x7f79c00c3c70 2024-04-26T22:00:11.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79beffd640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 0x7f79c00c8800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79b000c7f0 tx=0x7f79b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0016020 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b000aa50 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b000ae40 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c0074bd0 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.582+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c01ecae0 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.583+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c00507a0 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.584+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79b0019020 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.584+0000 7f79a77fe640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f79ac06c8f0 0x7f79ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.584+0000 7f79be7fc640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f79ac06c8f0 0x7f79ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.584+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f79b008ba60 con 0x7f79c00c3c70 2024-04-26T22:00:11.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.585+0000 7f79be7fc640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f79ac06c8f0 0x7f79ac06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79c0156b80 tx=0x7f79b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.589+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79b0020070 con 0x7f79c00c3c70 2024-04-26T22:00:11.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.764+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79c006f9c0 con 0x7f79ac06c8f0 2024-04-26T22:00:11.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.775+0000 7f79a77fe640 1 -- 172.21.15.135:0/2621872863 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490417 (secure 0 0 0) 0x7f79c006f9c0 con 0x7f79ac06c8f0 2024-04-26T22:00:11.781 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:11.784 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f79ac06c8f0 msgr2=0x7f79ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f79ac06c8f0 0x7f79ac06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79c0156b80 tx=0x7f79b4008040 comp rx=0 tx=0).stop 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 msgr2=0x7f79c00c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 0x7f79c00c8800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79b000c7f0 tx=0x7f79b000ccb0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 shutdown_connections 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f79ac06c8f0 0x7f79ac06edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c0157010 0x7f79c00c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c013fe20 0x7f79c00c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 --2- 172.21.15.135:0/2621872863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79c00c3c70 0x7f79c00c8800 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 >> 172.21.15.135:0/2621872863 conn(0x7f79c00c5d70 msgr2=0x7f79c0158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 shutdown_connections 2024-04-26T22:00:11.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.780+0000 7f79c57e2640 1 -- 172.21.15.135:0/2621872863 wait complete. 2024-04-26T22:00:11.791 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-04-26T22:00:11.071480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":345,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000985"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809657+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:07.809657+0000","last_peered":"2024-04-26T22:00:07.809657+0000","last_clean":"2024-04-26T22:00:07.809657+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:07.809657+0000","last_undegraded":"2024-04-26T22:00:07.809657+0000","last_fullsized":"2024-04-26T22:00:07.809657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:11.799 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:11.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.927+0000 7fddb1f5f640 1 Processor -- start 2024-04-26T22:00:11.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.927+0000 7fddb1f5f640 1 -- start start 2024-04-26T22:00:11.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddb1f5f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fddac0740e0 0x7fddac157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddb1f5f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fddac0ca3a0 0x7fddac0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddb1f5f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddac15a530 con 0x7fddac0c6e20 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddb1f5f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fddac072930 con 0x7fddac0740e0 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddb1f5f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fddac15a810 con 0x7fddac0ca3a0 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fddac0740e0 0x7fddac157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fddac0740e0 0x7fddac157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40250/0 (socket says 172.21.15.135:40250) 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddab7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fddac0c6e20 0x7fddac156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52872/0 (socket says 172.21.15.135:52872) 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 -- 172.21.15.135:0/2225273958 learned_addr learned my addr 172.21.15.135:0/2225273958 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddabfff640 1 --2- 172.21.15.135:0/2225273958 >> v2:172.21.15.139:3300/0 conn(0x7fddac0ca3a0 0x7fddac0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 -- 172.21.15.135:0/2225273958 >> v2:172.21.15.139:3300/0 conn(0x7fddac0ca3a0 msgr2=0x7fddac0c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaa7fc640 1 -- 172.21.15.135:0/2225273958 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819727336 0 0) 0x7fddac15a530 con 0x7fddac0c6e20 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 --2- 172.21.15.135:0/2225273958 >> v2:172.21.15.139:3300/0 conn(0x7fddac0ca3a0 0x7fddac0c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 -- 172.21.15.135:0/2225273958 >> v1:172.21.15.135:6789/0 conn(0x7fddac0c6e20 legacy=0x7fddac156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddaaffd640 1 -- 172.21.15.135:0/2225273958 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac15aaf0 con 0x7fddac0740e0 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.928+0000 7fddabfff640 1 --2- 172.21.15.135:0/2225273958 >> v2:172.21.15.139:3300/0 conn(0x7fddac0ca3a0 0x7fddac0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.929+0000 7fddaaffd640 1 --2- 172.21.15.135:0/2225273958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0740e0 0x7fddac157630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddac0af150 tx=0x7fdda002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6ad96ed0280a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.929+0000 7fddaa7fc640 1 -- 172.21.15.135:0/2225273958 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0037070 con 0x7fddac0740e0 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.929+0000 7fddaa7fc640 1 -- 172.21.15.135:0/2225273958 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda0032440 con 0x7fddac0740e0 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.929+0000 7fddaa7fc640 1 -- 172.21.15.135:0/2225273958 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0032810 con 0x7fddac0740e0 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.929+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2225273958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0740e0 msgr2=0x7fddac157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2225273958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0740e0 0x7fddac157630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddac0af150 tx=0x7fdda002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2225273958 shutdown_connections 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2225273958 >> v2:172.21.15.139:3300/0 conn(0x7fddac0ca3a0 0x7fddac0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2225273958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0740e0 0x7fddac157630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2225273958 >> 172.21.15.135:0/2225273958 conn(0x7fddac0ae0b0 msgr2=0x7fddac0ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2225273958 shutdown_connections 2024-04-26T22:00:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.930+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2225273958 wait complete. 2024-04-26T22:00:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.931+0000 7fddb1f5f640 1 Processor -- start 2024-04-26T22:00:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.931+0000 7fddb1f5f640 1 -- start start 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.931+0000 7fddb1f5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddac0740e0 0x7fddac142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.931+0000 7fddb1f5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddac0c6e20 0x7fddac1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.931+0000 7fddb1f5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 0x7fddac141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddb1f5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddac154540 con 0x7fddac0ca3a0 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddb1f5f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1543c0 con 0x7fddac0740e0 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddb1f5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fddac15a0c0 con 0x7fddac0c6e20 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 0x7fddac141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 0x7fddac141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40252/0 (socket says 172.21.15.135:40252) 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 -- 172.21.15.135:0/2356637646 learned_addr learned my addr 172.21.15.135:0/2356637646 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddaaffd640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddac0c6e20 0x7fddac1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddab7fe640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddac0740e0 0x7fddac142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 -- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddac0c6e20 msgr2=0x7fddac1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddac0c6e20 0x7fddac1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 -- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddac0740e0 msgr2=0x7fddac142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddac0740e0 0x7fddac142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 -- 172.21.15.135:0/2356637646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac142a70 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddab7fe640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddac0740e0 0x7fddac142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddaaffd640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddac0c6e20 0x7fddac1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddabfff640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 0x7fddac141d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdda400cd50 tx=0x7fdda400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400f680 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda400f840 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddac152cd0 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400fc30 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.932+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddac153290 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.933+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddac0766c0 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.934+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdda400e3e0 con 0x7fddac0ca3a0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.934+0000 7fdd93fff640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdd9806c920 0x7fdd9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.934+0000 7fddab7fe640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdd9806c920 0x7fdd9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:11.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.934+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fdda408cc00 con 0x7fddac0ca3a0 2024-04-26T22:00:11.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.934+0000 7fddab7fe640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdd9806c920 0x7fdd9806ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c000f60 tx=0x7fdd9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:11.938+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdda4093080 con 0x7fddac0ca3a0 2024-04-26T22:00:12.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.112+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddac0507a0 con 0x7fdd9806c920 2024-04-26T22:00:12.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.123+0000 7fdd93fff640 1 -- 172.21.15.135:0/2356637646 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490417 (secure 0 0 0) 0x7fddac0507a0 con 0x7fdd9806c920 2024-04-26T22:00:12.127 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:12.132 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:12.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdd9806c920 msgr2=0x7fdd9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdd9806c920 0x7fdd9806ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c000f60 tx=0x7fdd9c009040 comp rx=0 tx=0).stop 2024-04-26T22:00:12.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 msgr2=0x7fddac141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 0x7fddac141d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdda400cd50 tx=0x7fdda400a650 comp rx=0 tx=0).stop 2024-04-26T22:00:12.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 shutdown_connections 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdd9806c920 0x7fdd9806ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddac0ca3a0 0x7fddac141d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddac0c6e20 0x7fddac1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 --2- 172.21.15.135:0/2356637646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddac0740e0 0x7fddac142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 >> 172.21.15.135:0/2356637646 conn(0x7fddac0ae0b0 msgr2=0x7fddac0caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 shutdown_connections 2024-04-26T22:00:12.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.127+0000 7fddb1f5f640 1 -- 172.21.15.135:0/2356637646 wait complete. 2024-04-26T22:00:12.137 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-04-26T22:00:11.071480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":345,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000985"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809657+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:07.809657+0000","last_peered":"2024-04-26T22:00:07.809657+0000","last_clean":"2024-04-26T22:00:07.809657+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:07.809657+0000","last_undegraded":"2024-04-26T22:00:07.809657+0000","last_fullsized":"2024-04-26T22:00:07.809657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:12.145 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:00:12.145 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T22:00:12.145 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T22:00:12.146 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T22:00:12.146 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T22:00:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.278+0000 7fc82d893640 1 Processor -- start 2024-04-26T22:00:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.278+0000 7fc82d893640 1 -- start start 2024-04-26T22:00:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc82d893640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc82813ff20 0x7fc82815a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc82d893640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8280c3720 0x7fc8280c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc82d893640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8281577b0 con 0x7fc8280c46c0 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc82d893640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc828157560 con 0x7fc82813ff20 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc82d893640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8280ab960 con 0x7fc8280c3720 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc82813ff20 0x7fc82815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc82813ff20 0x7fc82815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40264/0 (socket says 172.21.15.135:40264) 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 -- 172.21.15.135:0/1275498726 learned_addr learned my addr 172.21.15.135:0/1275498726 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc8277fe640 1 --2- 172.21.15.135:0/1275498726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8280c3720 0x7fc8280c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 -- 172.21.15.135:0/1275498726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8280c3720 msgr2=0x7fc8280c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 --2- 172.21.15.135:0/1275498726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8280c3720 0x7fc8280c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 -- 172.21.15.135:0/1275498726 >> v1:172.21.15.139:6789/0 conn(0x7fc8280c46c0 legacy=0x7fc82815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.279+0000 7fc826ffd640 1 -- 172.21.15.135:0/1275498726 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8280ab5f0 con 0x7fc82813ff20 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.280+0000 7fc826ffd640 1 --2- 172.21.15.135:0/1275498726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813ff20 0x7fc82815a280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8280b05c0 tx=0x7fc81802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9f0fb015045828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.280+0000 7fc825ffb640 1 -- 172.21.15.135:0/1275498726 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818036070 con 0x7fc82813ff20 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.280+0000 7fc825ffb640 1 -- 172.21.15.135:0/1275498726 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc818031600 con 0x7fc82813ff20 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.280+0000 7fc825ffb640 1 -- 172.21.15.135:0/1275498726 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180319d0 con 0x7fc82813ff20 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 -- 172.21.15.135:0/1275498726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813ff20 msgr2=0x7fc82815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 --2- 172.21.15.135:0/1275498726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813ff20 0x7fc82815a280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8280b05c0 tx=0x7fc81802a6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 -- 172.21.15.135:0/1275498726 shutdown_connections 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 --2- 172.21.15.135:0/1275498726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8280c3720 0x7fc8280c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 --2- 172.21.15.135:0/1275498726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813ff20 0x7fc82815a280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 -- 172.21.15.135:0/1275498726 >> 172.21.15.135:0/1275498726 conn(0x7fc8280ca2f0 msgr2=0x7fc8280ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 -- 172.21.15.135:0/1275498726 shutdown_connections 2024-04-26T22:00:12.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.281+0000 7fc82d893640 1 -- 172.21.15.135:0/1275498726 wait complete. 2024-04-26T22:00:12.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.282+0000 7fc82d893640 1 Processor -- start 2024-04-26T22:00:12.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.282+0000 7fc82d893640 1 -- start start 2024-04-26T22:00:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8280c3720 0x7fc828151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc82813ff20 0x7fc82813dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 0x7fc82813e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc828140300 con 0x7fc82813e9b0 2024-04-26T22:00:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8280ac710 con 0x7fc8280c3720 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc828140480 con 0x7fc82813ff20 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 0x7fc82813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 0x7fc82813e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40274/0 (socket says 172.21.15.135:40274) 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 -- 172.21.15.135:0/2711265081 learned_addr learned my addr 172.21.15.135:0/2711265081 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc826ffd640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8280c3720 0x7fc828151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8267fc640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc82813ff20 0x7fc82813dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 -- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc82813ff20 msgr2=0x7fc82813dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc82813ff20 0x7fc82813dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 -- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8280c3720 msgr2=0x7fc828151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8280c3720 0x7fc828151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 -- 172.21.15.135:0/2711265081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82813e760 con 0x7fc82813e9b0 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8267fc640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc82813ff20 0x7fc82813dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc8277fe640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 0x7fc82813e220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c820 tx=0x7fc82000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820015070 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820004020 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281f0980 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820004410 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.283+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc828154e80 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.284+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8280766c0 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.285+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc820007830 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.285+0000 7fc80ffff640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.285+0000 7fc826ffd640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.285+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc82008bc00 con 0x7fc82813e9b0 2024-04-26T22:00:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.286+0000 7fc826ffd640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc81406c8f0 0x7fc81406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc818033000 tx=0x7fc818009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.289+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc820092080 con 0x7fc82813e9b0 2024-04-26T22:00:12.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.509+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc82814d8c0 con 0x7fc82813e9b0 2024-04-26T22:00:12.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.512+0000 7fc80ffff640 1 -- 172.21.15.135:0/2711265081 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174144 (secure 0 0 0) 0x7fc820059d90 con 0x7fc82813e9b0 2024-04-26T22:00:12.513 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc81406c8f0 msgr2=0x7fc81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc81406c8f0 0x7fc81406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc818033000 tx=0x7fc818009000 comp rx=0 tx=0).stop 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 msgr2=0x7fc82813e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 0x7fc82813e220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c820 tx=0x7fc82000cce0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 shutdown_connections 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc82813e9b0 0x7fc82813e220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc82813ff20 0x7fc82813dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 --2- 172.21.15.135:0/2711265081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8280c3720 0x7fc828151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 >> 172.21.15.135:0/2711265081 conn(0x7fc8280ca2f0 msgr2=0x7fc8280c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 shutdown_connections 2024-04-26T22:00:12.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.514+0000 7fc82d893640 1 -- 172.21.15.135:0/2711265081 wait complete. 2024-04-26T22:00:12.527 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3236798717},{"type":"v1","addr":"172.21.15.135:6833","nonce":3236798717}]},"active_addr":"172.21.15.135:6833/3236798717","active_change":"2024-04-26T21:59:37.060028+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1050147649}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":3838188661}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1403484283}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":4237126386}]}]} 2024-04-26T22:00:12.530 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T22:00:12.530 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T22:00:12.531 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:12.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.627+0000 7fe4aa3e4640 1 Processor -- start 2024-04-26T22:00:12.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.627+0000 7fe4aa3e4640 1 -- start start 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.627+0000 7fe4aa3e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4a40c6e20 0x7fe4a40c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4aa3e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40ca3a0 0x7fe4a4157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4aa3e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a415a530 con 0x7fe4a40740e0 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4aa3e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a4072930 con 0x7fe4a40c6e20 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4aa3e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a415a810 con 0x7fe4a40ca3a0 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4a40c6e20 0x7fe4a40c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a8981640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40ca3a0 0x7fe4a4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a37fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe4a40740e0 0x7fe4a4156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57226/0 (socket says 172.21.15.135:57226) 2024-04-26T22:00:12.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4a40c6e20 0x7fe4a40c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33222/0 (socket says 172.21.15.135:33222) 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a37fe640 1 -- 172.21.15.135:0/1633275668 learned_addr learned my addr 172.21.15.135:0/1633275668 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40ca3a0 msgr2=0x7fe4a4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a3fff640 1 --2- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40ca3a0 0x7fe4a4157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1633275668 >> v1:172.21.15.139:6789/0 conn(0x7fe4a40740e0 legacy=0x7fe4a4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1633275668 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a415aaf0 con 0x7fe4a40c6e20 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.628+0000 7fe4a8981640 1 --2- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40ca3a0 0x7fe4a4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.629+0000 7fe4a3fff640 1 --2- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40c6e20 0x7fe4a40c7200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe4940097a0 tx=0x7fe49402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808aaec41f8761c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.629+0000 7fe4a2ffd640 1 -- 172.21.15.135:0/1633275668 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494037070 con 0x7fe4a40c6e20 2024-04-26T22:00:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.629+0000 7fe4a2ffd640 1 -- 172.21.15.135:0/1633275668 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49402adb0 con 0x7fe4a40c6e20 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.629+0000 7fe4a2ffd640 1 -- 172.21.15.135:0/1633275668 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494032660 con 0x7fe4a40c6e20 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40c6e20 msgr2=0x7fe4a40c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40c6e20 0x7fe4a40c7200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe4940097a0 tx=0x7fe49402a060 comp rx=0 tx=0).stop 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1633275668 shutdown_connections 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40ca3a0 0x7fe4a4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1633275668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40c6e20 0x7fe4a40c7200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1633275668 >> 172.21.15.135:0/1633275668 conn(0x7fe4a40ae0b0 msgr2=0x7fe4a40ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1633275668 shutdown_connections 2024-04-26T22:00:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.630+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1633275668 wait complete. 2024-04-26T22:00:12.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.631+0000 7fe4aa3e4640 1 Processor -- start 2024-04-26T22:00:12.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.631+0000 7fe4aa3e4640 1 -- start start 2024-04-26T22:00:12.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 0x7fe4a41ed890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40c6e20 0x7fe4a4159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40ca3a0 0x7fe4a41e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4154540 con 0x7fe4a40740e0 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41543c0 con 0x7fe4a40ca3a0 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a415a0c0 con 0x7fe4a40c6e20 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 0x7fe4a41ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 0x7fe4a41ed890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40280/0 (socket says 172.21.15.135:40280) 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1710245028 learned_addr learned my addr 172.21.15.135:0/1710245028 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a37fe640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40c6e20 0x7fe4a4159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a8981640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40ca3a0 0x7fe4a41e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40c6e20 msgr2=0x7fe4a4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40c6e20 0x7fe4a4159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40ca3a0 msgr2=0x7fe4a41e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40ca3a0 0x7fe4a41e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 -- 172.21.15.135:0/1710245028 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a41e01d0 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a3fff640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 0x7fe4a41ed890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe494035040 tx=0x7fe4940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494041020 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49403c670 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494037040 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a41e04b0 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.632+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a41e1e80 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.634+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a40766c0 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.634+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe494044020 con 0x7fe4a40740e0 2024-04-26T22:00:12.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.634+0000 7fe4a0ff9640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe49006c920 0x7fe49006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.634+0000 7fe4a37fe640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe49006c920 0x7fe49006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.634+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe4940ac4d0 con 0x7fe4a40740e0 2024-04-26T22:00:12.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.635+0000 7fe4a37fe640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe49006c920 0x7fe49006ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40c3250 tx=0x7fe498007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.639+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe49404b030 con 0x7fe4a40740e0 2024-04-26T22:00:12.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.821+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4a40cbfe0 con 0x7fe4a40740e0 2024-04-26T22:00:12.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.822+0000 7fe4a0ff9640 1 -- 172.21.15.135:0/1710245028 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7fe49407a5d0 con 0x7fe4a40740e0 2024-04-26T22:00:12.822 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:12.822 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:12.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe49006c920 msgr2=0x7fe49006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe49006c920 0x7fe49006ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40c3250 tx=0x7fe498007e80 comp rx=0 tx=0).stop 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 msgr2=0x7fe4a41ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 0x7fe4a41ed890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe494035040 tx=0x7fe4940046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 shutdown_connections 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe49006c920 0x7fe49006ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4a40ca3a0 0x7fe4a41e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4a40c6e20 0x7fe4a4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 --2- 172.21.15.135:0/1710245028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a40740e0 0x7fe4a41ed890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.824+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 >> 172.21.15.135:0/1710245028 conn(0x7fe4a40ae0b0 msgr2=0x7fe4a4149f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.825+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 shutdown_connections 2024-04-26T22:00:12.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.825+0000 7fe4aa3e4640 1 -- 172.21.15.135:0/1710245028 wait complete. 2024-04-26T22:00:12.837 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T22:00:12.837 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:12.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.935+0000 7fc199be4640 1 Processor -- start 2024-04-26T22:00:12.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc199be4640 1 -- start start 2024-04-26T22:00:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc199be4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 0x7fc1940c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc199be4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc194154e10 con 0x7fc19406d590 2024-04-26T22:00:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc199be4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1941550b0 con 0x7fc194070120 2024-04-26T22:00:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc199be4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc1941559e0 con 0x7fc1940af140 2024-04-26T22:00:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc1937fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc19406d590 0x7fc194158d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52882/0 (socket says 172.21.15.135:52882) 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.936+0000 7fc1937fe640 1 -- 172.21.15.135:0/3647992338 learned_addr learned my addr 172.21.15.135:0/3647992338 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc193fff640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 0x7fc1940c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383289198 0 0) 0x7fc194154e10 con 0x7fc19406d590 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc180003520 con 0x7fc19406d590 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687272861 0 0) 0x7fc1941550b0 con 0x7fc194070120 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc194154e10 con 0x7fc194070120 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 153082673 0 0) 0x7fc180003520 con 0x7fc19406d590 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 msgr2=0x7fc1940c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 0x7fc1940c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 >> v1:172.21.15.139:6789/0 conn(0x7fc194070120 legacy=0x7fc1940aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc194155390 con 0x7fc19406d590 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fc184001690 con 0x7fc19406d590 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 >> v1:172.21.15.135:6789/0 conn(0x7fc19406d590 legacy=0x7fc194158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 0x7fc180003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004120 0x7fc180004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc180004d40 0x7fc1800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941559e0 con 0x7fc180004120 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc18000bb80 con 0x7fc180004d40 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc18000bd80 con 0x7fc1940af140 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc18affd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004120 0x7fc180004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc1937fe640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 0x7fc180003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc193fff640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc180004d40 0x7fc1800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc18affd640 1 -- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 msgr2=0x7fc180003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc18affd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1940af140 0x7fc180003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc18affd640 1 -- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc180004d40 msgr2=0x7fc1800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc18affd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc180004d40 0x7fc1800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.937+0000 7fc18affd640 1 -- 172.21.15.135:0/3647992338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc180003520 con 0x7fc180004120 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.938+0000 7fc18affd640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004120 0x7fc180004800 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0018d0 tx=0x7fc180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dd2e6020ad3f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.938+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0014f0 con 0x7fc180004120 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.938+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc17c0052d0 con 0x7fc180004120 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.938+0000 7fc192ffd640 1 -- 172.21.15.135:0/3647992338 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c02aca0 con 0x7fc180004120 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc199be4640 1 -- 172.21.15.135:0/3647992338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004120 msgr2=0x7fc180004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc199be4640 1 --2- 172.21.15.135:0/3647992338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004120 0x7fc180004800 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0018d0 tx=0x7fc180005830 comp rx=0 tx=0).stop 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc193fff640 1 -- 172.21.15.135:0/3647992338 reap_dead start 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc199be4640 1 -- 172.21.15.135:0/3647992338 shutdown_connections 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc199be4640 1 -- 172.21.15.135:0/3647992338 >> 172.21.15.135:0/3647992338 conn(0x7fc1940c5ce0 msgr2=0x7fc1940c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc199be4640 1 -- 172.21.15.135:0/3647992338 shutdown_connections 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.939+0000 7fc199be4640 1 -- 172.21.15.135:0/3647992338 wait complete. 2024-04-26T22:00:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.940+0000 7fc199be4640 1 Processor -- start 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.940+0000 7fc199be4640 1 -- start start 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc180004120 0x7fc19414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 0x7fc1940ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19414e620 0x7fc19414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc19406daf0 con 0x7fc1941ee540 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc19406d970 con 0x7fc19414e620 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc1940c3be0 con 0x7fc180004120 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 0x7fc1940ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 0x7fc1940ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40300/0 (socket says 172.21.15.135:40300) 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 -- 172.21.15.135:0/1305335083 learned_addr learned my addr 172.21.15.135:0/1305335083 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc193fff640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19414e620 0x7fc19414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc1937fe640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc180004120 0x7fc19414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 -- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc180004120 msgr2=0x7fc19414f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc180004120 0x7fc19414f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 -- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19414e620 msgr2=0x7fc19414ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19414e620 0x7fc19414ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 -- 172.21.15.135:0/1305335083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941deb90 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc18affd640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 0x7fc1940ad840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc17c005040 tx=0x7fc17c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c037040 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc17c006e50 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0014f0 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941dee70 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.941+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941dfd50 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.942+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc194068dc0 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.943+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc17c044020 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.943+0000 7fc190ff9640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc17406c920 0x7fc17406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.943+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc17c0abc10 con 0x7fc1941ee540 2024-04-26T22:00:12.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.943+0000 7fc1937fe640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc17406c920 0x7fc17406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:12.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.944+0000 7fc1937fe640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc17406c920 0x7fc17406ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc1940c6b60 tx=0x7fc184009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:12.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:12.948+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc17c079ec0 con 0x7fc1941ee540 2024-04-26T22:00:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.126+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1941de5a0 con 0x7fc1941ee540 2024-04-26T22:00:13.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.128+0000 7fc190ff9640 1 -- 172.21.15.135:0/1305335083 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7fc17c0797c0 con 0x7fc1941ee540 2024-04-26T22:00:13.128 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:13.128 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:13.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc17406c920 msgr2=0x7fc17406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc17406c920 0x7fc17406ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc1940c6b60 tx=0x7fc184009040 comp rx=0 tx=0).stop 2024-04-26T22:00:13.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 msgr2=0x7fc1940ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 0x7fc1940ad840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc17c005040 tx=0x7fc17c0012d0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 shutdown_connections 2024-04-26T22:00:13.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc17406c920 0x7fc17406ede0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19414e620 0x7fc19414ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1941ee540 0x7fc1940ad840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 --2- 172.21.15.135:0/1305335083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc180004120 0x7fc19414f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 >> 172.21.15.135:0/1305335083 conn(0x7fc1940c5ce0 msgr2=0x7fc1940c4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 shutdown_connections 2024-04-26T22:00:13.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.130+0000 7fc199be4640 1 -- 172.21.15.135:0/1305335083 wait complete. 2024-04-26T22:00:13.141 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T22:00:13.142 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T22:00:13.142 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T22:00:13.143 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T22:00:13.143 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T22:00:13.143 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T22:00:13.144 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T22:00:13.144 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T22:00:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.308+0000 7fe23792b640 1 Processor -- start 2024-04-26T22:00:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.309+0000 7fe23792b640 1 -- start start 2024-04-26T22:00:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe23792b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe230151430 0x7fe2300be440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe23792b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe230157d20 con 0x7fe2300adda0 2024-04-26T22:00:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe23792b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe230158000 con 0x7fe2300c5390 2024-04-26T22:00:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe23792b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe23006f030 con 0x7fe230151430 2024-04-26T22:00:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f67f3b19640 1 Processor -- start 2024-04-26T22:00:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7f30ccb08640 1 Processor -- start 2024-04-26T22:00:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7f30ccb08640 1 -- start start 2024-04-26T22:00:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7f20e41c6640 1 Processor -- start 2024-04-26T22:00:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7f20e41c6640 1 -- start start 2024-04-26T22:00:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe235ec8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe2300c5390 0x7fe23006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57256/0 (socket says 172.21.15.135:57256) 2024-04-26T22:00:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe235ec8640 1 -- 172.21.15.135:0/2799164839 learned_addr learned my addr 172.21.15.135:0/2799164839 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.310+0000 7fe2356c7640 1 --2- 172.21.15.135:0/2799164839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe230151430 0x7fe2300be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7fe227fff640 1 -- 172.21.15.135:0/2799164839 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3666309357 0 0) 0x7fe230158000 con 0x7fe2300c5390 2024-04-26T22:00:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7fe227fff640 1 -- 172.21.15.135:0/2799164839 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe21c003520 con 0x7fe2300c5390 2024-04-26T22:00:13.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f20e1761640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f20dc0c3f00 0x7f20dc0c4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52900/0 (socket says 172.21.15.135:52900) 2024-04-26T22:00:13.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f20e41c6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f20dc1500b0 0x7f20dc150520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f20e41c6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc06fbe0 con 0x7f20dc0c3f00 2024-04-26T22:00:13.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f20e41c6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc0ac270 con 0x7f20dc06f710 2024-04-26T22:00:13.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f20e41c6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0c54e0 con 0x7f20dc1500b0 2024-04-26T22:00:13.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f20e1761640 1 -- 172.21.15.135:0/585828810 learned_addr learned my addr 172.21.15.135:0/585828810 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f30ccb08640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30c00d3b40 0x7f30c0046b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30ccb08640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f30c00d6710 0x7f30c00d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30c6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f30c004b110 0x7f30c0006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57274/0 (socket says 172.21.15.135:57274) 2024-04-26T22:00:13.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30c6ffd640 1 -- 172.21.15.135:0/981345554 learned_addr learned my addr 172.21.15.135:0/981345554 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f373e6b0640 1 Processor -- start 2024-04-26T22:00:13.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e2763640 1 --2- 172.21.15.135:0/585828810 >> v2:172.21.15.135:3300/0 conn(0x7f20dc1500b0 0x7f20dc150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c0047050 con 0x7f30c004b110 2024-04-26T22:00:13.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f78d2f35640 1 Processor -- start 2024-04-26T22:00:13.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139762661 0 0) 0x7f20dc06fbe0 con 0x7f20dc0c3f00 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20c8003520 con 0x7f20dc0c3f00 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306484975 0 0) 0x7f20dc0ac270 con 0x7f20dc06f710 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20dc06fbe0 con 0x7f20dc06f710 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7f67f3b19640 1 -- start start 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f3b19640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67e40dbd10 0x7f67e40474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f3b19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e40df1b0 0x7f67e4006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f3b19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f67e4007410 0x7f67e400a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f3b19640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f67e400a450 con 0x7f67e40dbd10 2024-04-26T22:00:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f3b19640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f67e400bb80 con 0x7f67e4007410 2024-04-26T22:00:13.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f3b19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67e4048090 con 0x7f67e40df1b0 2024-04-26T22:00:13.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7fe2356c7640 1 -- 172.21.15.135:0/2799164839 >> v1:172.21.15.139:6789/0 conn(0x7fe2300c5390 legacy=0x7fe23006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7fe2356c7640 1 -- 172.21.15.135:0/2799164839 >> v1:172.21.15.135:6789/0 conn(0x7fe2300adda0 legacy=0x7fe2300bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30c7fff640 1 --2- 172.21.15.135:0/981345554 >> v2:172.21.15.135:3300/0 conn(0x7f30c00d6710 0x7f30c00d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00d9c60 con 0x7f30c00d6710 2024-04-26T22:00:13.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00d9f40 con 0x7f30c00d3b40 2024-04-26T22:00:13.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7fe227fff640 1 -- 172.21.15.135:0/2799164839 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878071775 0 0) 0x7fe230157d20 con 0x7fe2300adda0 2024-04-26T22:00:13.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.311+0000 7fe2356c7640 1 -- 172.21.15.135:0/2799164839 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23006f280 con 0x7fe230151430 2024-04-26T22:00:13.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30c77fe640 1 --2- 172.21.15.135:0/981345554 >> v2:172.21.15.139:3300/0 conn(0x7f30c00d3b40 0x7f30c0046b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614499715 0 0) 0x7f30c0047050 con 0x7f30c004b110 2024-04-26T22:00:13.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30b8003520 con 0x7f30c004b110 2024-04-26T22:00:13.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7fe2356c7640 1 --2- 172.21.15.135:0/2799164839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe230151430 0x7fe2300be440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2200097d0 tx=0x7fe22002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c987d6a5c41e1b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f20b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f67e4007410 0x7f67e400a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f67f10b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e40df1b0 0x7f67e4006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119296941 0 0) 0x7f30b8003520 con 0x7f30c004b110 2024-04-26T22:00:13.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e2763640 1 -- 172.21.15.135:0/585828810 >> v1:172.21.15.139:6789/0 conn(0x7f20dc06f710 legacy=0x7f20dc06e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e2763640 1 -- 172.21.15.135:0/585828810 >> v1:172.21.15.135:6789/0 conn(0x7f20dc0c3f00 legacy=0x7f20dc0c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f20e2763640 1 -- 172.21.15.135:0/585828810 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc0ac550 con 0x7f20dc1500b0 2024-04-26T22:00:13.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 >> v2:172.21.15.139:3300/0 conn(0x7f30c00d3b40 msgr2=0x7f30c0046b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 --2- 172.21.15.135:0/981345554 >> v2:172.21.15.139:3300/0 conn(0x7f30c00d3b40 0x7f30c0046b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 >> v2:172.21.15.135:3300/0 conn(0x7f30c00d6710 msgr2=0x7f30c00d6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7fe227fff640 1 -- 172.21.15.135:0/2799164839 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220037070 con 0x7fe230151430 2024-04-26T22:00:13.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7fe227fff640 1 -- 172.21.15.135:0/2799164839 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe220032440 con 0x7fe230151430 2024-04-26T22:00:13.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 --2- 172.21.15.135:0/981345554 >> v2:172.21.15.135:3300/0 conn(0x7f30c00d6710 0x7f30c00d6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f70ca8d9640 1 Processor -- start 2024-04-26T22:00:13.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00da220 con 0x7f30c004b110 2024-04-26T22:00:13.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f30b4001670 con 0x7f30c004b110 2024-04-26T22:00:13.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 >> v1:172.21.15.139:6789/0 conn(0x7f30c004b110 legacy=0x7f30c0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f67f20b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f67e4007410 0x7f67e400a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33256/0 (socket says 172.21.15.135:33256) 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f67f20b6640 1 -- 172.21.15.135:0/1692548042 learned_addr learned my addr 172.21.15.135:0/1692548042 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f373e6b0640 1 -- start start 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f373e6b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37300d7910 0x7f373000eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f373e6b0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37300d7cf0 con 0x7f373000fce0 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f373e6b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37300db240 con 0x7f3730050710 2024-04-26T22:00:13.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f373e6b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f373004b9f0 con 0x7f37300d7910 2024-04-26T22:00:13.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.312+0000 7f78d2f35640 1 -- start start 2024-04-26T22:00:13.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f373cc4d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f373000fce0 0x7f37300069c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52916/0 (socket says 172.21.15.135:52916) 2024-04-26T22:00:13.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f373cc4d640 1 -- 172.21.15.135:0/4198589639 learned_addr learned my addr 172.21.15.135:0/4198589639 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f20e2763640 1 --2- 172.21.15.135:0/585828810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc1500b0 0x7f20dc150520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009d90 tx=0x7f20d402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d86fa5cfb554d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b80039b0 0x7f30b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f3737fff640 1 --2- 172.21.15.135:0/4198589639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37300d7910 0x7f373000eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f78d2f35640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 0x7f78c400acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f78d2f35640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c400bb40 con 0x7f78c40dbcd0 2024-04-26T22:00:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f78d2f35640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c4048050 con 0x7f78c4047490 2024-04-26T22:00:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f78d2f35640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f78c400a410 con 0x7f78c400a8a0 2024-04-26T22:00:13.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b8004800 0x7f30b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00d9c60 con 0x7f30b80051c0 2024-04-26T22:00:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00d9f40 con 0x7f30b8004800 2024-04-26T22:00:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f30b800c170 con 0x7f30b80039b0 2024-04-26T22:00:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f70ca8d9640 1 -- start start 2024-04-26T22:00:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f6493346640 1 Processor -- start 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f3736ffd640 1 -- 172.21.15.135:0/4198589639 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876751395 0 0) 0x7f37300db240 con 0x7f3730050710 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f3736ffd640 1 -- 172.21.15.135:0/4198589639 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3728003520 con 0x7f3730050710 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.313+0000 7f67f18b5640 1 --2- 172.21.15.135:0/1692548042 >> v2:172.21.15.135:3300/0 conn(0x7f67e40dbd10 0x7f67e40474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f67f10b4640 1 -- 172.21.15.135:0/1692548042 >> v2:172.21.15.139:3300/0 conn(0x7f67e4007410 msgr2=0x7f67e400a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f67f10b4640 1 --2- 172.21.15.135:0/1692548042 >> v2:172.21.15.139:3300/0 conn(0x7f67e4007410 0x7f67e400a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f67f10b4640 1 -- 172.21.15.135:0/1692548042 >> v2:172.21.15.135:3300/0 conn(0x7f67e40dbd10 msgr2=0x7f67e40474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f67f10b4640 1 --2- 172.21.15.135:0/1692548042 >> v2:172.21.15.135:3300/0 conn(0x7f67e40dbd10 0x7f67e40474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f67f10b4640 1 -- 172.21.15.135:0/1692548042 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e400a120 con 0x7f67e40df1b0 2024-04-26T22:00:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f6493346640 1 -- start start 2024-04-26T22:00:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 -- 172.21.15.135:0/2799164839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe230151430 msgr2=0x7fe2300be440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 --2- 172.21.15.135:0/2799164839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe230151430 0x7fe2300be440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2200097d0 tx=0x7fe22002a180 comp rx=0 tx=0).stop 2024-04-26T22:00:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 -- 172.21.15.135:0/2799164839 shutdown_connections 2024-04-26T22:00:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 --2- 172.21.15.135:0/2799164839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe230151430 0x7fe2300be440 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2200097d0 tx=0x7fe22002a180 comp rx=0 tx=0).stop 2024-04-26T22:00:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 -- 172.21.15.135:0/2799164839 >> 172.21.15.135:0/2799164839 conn(0x7fe2300c91e0 msgr2=0x7fe2300c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f3737fff640 1 -- 172.21.15.135:0/4198589639 >> v1:172.21.15.139:6789/0 conn(0x7f3730050710 legacy=0x7f37300062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f3737fff640 1 -- 172.21.15.135:0/4198589639 >> v1:172.21.15.135:6789/0 conn(0x7f373000fce0 legacy=0x7f37300069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f3737fff640 1 -- 172.21.15.135:0/4198589639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373004bcd0 con 0x7f37300d7910 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d2734640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 0x7f78c400acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d2734640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 0x7f78c400acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:33808/0 (socket says 172.21.15.135:33808) 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d2734640 1 -- 172.21.15.135:0/3218249344 learned_addr learned my addr 172.21.15.135:0/3218249344 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4031600 con 0x7f20dc1500b0 2024-04-26T22:00:13.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d40317a0 con 0x7f20dc1500b0 2024-04-26T22:00:13.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 --2- 172.21.15.135:0/981345554 >> v2:172.21.15.135:3300/0 conn(0x7f30c00d6710 0x7f30c00d6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:13.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d1732640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f78c4047490 0x7f78c4006e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57298/0 (socket says 172.21.15.135:57298) 2024-04-26T22:00:13.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c77fe640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b80039b0 0x7f30b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c6ffd640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b8004800 0x7f30b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 -- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b80039b0 msgr2=0x7f30b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f67f18b5640 1 --2- 172.21.15.135:0/1692548042 >> v2:172.21.15.135:3300/0 conn(0x7f67e40dbd10 0x7f67e40474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:13.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b80039b0 0x7f30b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 -- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b8004800 msgr2=0x7f30b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b8004800 0x7f30b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 -- 172.21.15.135:0/2799164839 shutdown_connections 2024-04-26T22:00:13.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f30c7fff640 1 -- 172.21.15.135:0/981345554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00da220 con 0x7f30b80051c0 2024-04-26T22:00:13.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d0f31640 1 -- 172.21.15.135:0/3218249344 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2014643554 0 0) 0x7f78c400bb40 con 0x7f78c40dbcd0 2024-04-26T22:00:13.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d0f31640 1 -- 172.21.15.135:0/3218249344 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78b8003520 con 0x7f78c40dbcd0 2024-04-26T22:00:13.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f30c6ffd640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b8004800 0x7f30b8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f30c77fe640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b80039b0 0x7f30b8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d2734640 1 -- 172.21.15.135:0/3218249344 >> v1:172.21.15.139:6789/0 conn(0x7f78c4047490 legacy=0x7f78c4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7f78d2734640 1 -- 172.21.15.135:0/3218249344 >> v1:172.21.15.135:6789/0 conn(0x7f78c40dbcd0 legacy=0x7f78c40df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f78d2734640 1 -- 172.21.15.135:0/3218249344 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c400a0e0 con 0x7f78c400a8a0 2024-04-26T22:00:13.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e0f60640 1 -- 172.21.15.135:0/585828810 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4031b70 con 0x7f20dc1500b0 2024-04-26T22:00:13.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e41c6640 1 -- 172.21.15.135:0/585828810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc1500b0 msgr2=0x7f20dc150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e41c6640 1 --2- 172.21.15.135:0/585828810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc1500b0 0x7f20dc150520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009d90 tx=0x7f20d402a5d0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.314+0000 7fe23792b640 1 -- 172.21.15.135:0/2799164839 wait complete. 2024-04-26T22:00:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f3737fff640 1 --2- 172.21.15.135:0/4198589639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37300d7910 0x7f373000eda0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097d0 tx=0x7f372c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc94c1ada21fdb85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e41c6640 1 -- 172.21.15.135:0/585828810 shutdown_connections 2024-04-26T22:00:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e41c6640 1 --2- 172.21.15.135:0/585828810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc1500b0 0x7f20dc150520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e41c6640 1 -- 172.21.15.135:0/585828810 >> 172.21.15.135:0/585828810 conn(0x7f20dc06c310 msgr2=0x7f20dc06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f67f10b4640 1 --2- 172.21.15.135:0/1692548042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e40df1b0 0x7f67e4006ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67e400a2f0 tx=0x7f67dc029fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2258f4cd72fc50c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f3736ffd640 1 -- 172.21.15.135:0/4198589639 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c037070 con 0x7f37300d7910 2024-04-26T22:00:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f20e41c6640 1 -- 172.21.15.135:0/585828810 shutdown_connections 2024-04-26T22:00:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f3736ffd640 1 -- 172.21.15.135:0/4198589639 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372c032440 con 0x7f37300d7910 2024-04-26T22:00:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f3736ffd640 1 -- 172.21.15.135:0/4198589639 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c032810 con 0x7f37300d7910 2024-04-26T22:00:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f67f08b3640 1 -- 172.21.15.135:0/1692548042 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc037070 con 0x7f67e40df1b0 2024-04-26T22:00:13.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f6493346640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f648c1573c0 0x7f648c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f67f08b3640 1 -- 172.21.15.135:0/1692548042 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67dc02ad00 con 0x7f67e40df1b0 2024-04-26T22:00:13.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.315+0000 7f67f08b3640 1 -- 172.21.15.135:0/1692548042 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc032660 con 0x7f67e40df1b0 2024-04-26T22:00:13.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f30c7fff640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30b800b9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0098a0 tx=0x7f30b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d8a4258aa21242 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f30c7fff640 1 -- 172.21.15.135:0/981345554 reap_dead start 2024-04-26T22:00:13.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f6493346640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f648c0cbc70 0x7f648c161310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f6493346640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648c0c6f10 con 0x7f648c150990 2024-04-26T22:00:13.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f64910e2640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f648c150990 0x7f648c06c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57302/0 (socket says 172.21.15.135:57302) 2024-04-26T22:00:13.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f6493346640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f648c154620 con 0x7f648c1573c0 2024-04-26T22:00:13.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f6493346640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f648c0c6cc0 con 0x7f648c0cbc70 2024-04-26T22:00:13.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f64910e2640 1 -- 172.21.15.135:0/2628457228 learned_addr learned my addr 172.21.15.135:0/2628457228 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7fe23792b640 1 Processor -- start 2024-04-26T22:00:13.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f78d2734640 1 --2- 172.21.15.135:0/3218249344 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 0x7f78c400acf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78c00098a0 tx=0x7f78c002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69e1387adb40c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164683160 0 0) 0x7f648c0c6f10 con 0x7f648c150990 2024-04-26T22:00:13.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f78d0f31640 1 -- 172.21.15.135:0/3218249344 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0037070 con 0x7f78c400a8a0 2024-04-26T22:00:13.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6478003520 con 0x7f648c150990 2024-04-26T22:00:13.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f78d0f31640 1 -- 172.21.15.135:0/3218249344 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c0032440 con 0x7f78c400a8a0 2024-04-26T22:00:13.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f373e6b0640 1 -- 172.21.15.135:0/4198589639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37300d7910 msgr2=0x7f373000eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f78d0f31640 1 -- 172.21.15.135:0/3218249344 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c00327f0 con 0x7f78c400a8a0 2024-04-26T22:00:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f373e6b0640 1 --2- 172.21.15.135:0/4198589639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37300d7910 0x7f373000eda0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097d0 tx=0x7f372c02a180 comp rx=0 tx=0).stop 2024-04-26T22:00:13.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7fe23792b640 1 -- start start 2024-04-26T22:00:13.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f373e6b0640 1 -- 172.21.15.135:0/4198589639 shutdown_connections 2024-04-26T22:00:13.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f373e6b0640 1 --2- 172.21.15.135:0/4198589639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37300d7910 0x7f373000eda0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f373e6b0640 1 -- 172.21.15.135:0/4198589639 >> 172.21.15.135:0/4198589639 conn(0x7f373004fbf0 msgr2=0x7f3730050000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f20e41c6640 1 -- 172.21.15.135:0/585828810 wait complete. 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70ca8d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70c4158600 0x7f70c40c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70ca8d9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70c406c310 0x7f70c40ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70ca8d9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c415e3b0 0x7f70c415e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70ca8d9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f70c4074590 con 0x7f70c4158600 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70ca8d9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f70c40abe80 con 0x7f70c406c310 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70ca8d9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f70c40747e0 con 0x7f70c415e3b0 2024-04-26T22:00:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70c3fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70c4158600 0x7f70c40c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70c3fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70c4158600 0x7f70c40c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40348/0 (socket says 172.21.15.135:40348) 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70c3fff640 1 -- 172.21.15.135:0/1495837804 learned_addr learned my addr 172.21.15.135:0/1495837804 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f70c8e76640 1 --2- 172.21.15.135:0/1495837804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c415e3b0 0x7f70c415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3747734601 0 0) 0x7f6478003520 con 0x7f648c150990 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 >> v2:172.21.15.139:3300/0 conn(0x7f648c0cbc70 msgr2=0x7f648c161310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 --2- 172.21.15.135:0/2628457228 >> v2:172.21.15.139:3300/0 conn(0x7f648c0cbc70 0x7f648c161310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 >> v2:172.21.15.135:3300/0 conn(0x7f648c1573c0 msgr2=0x7f648c0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 --2- 172.21.15.135:0/2628457228 >> v2:172.21.15.135:3300/0 conn(0x7f648c1573c0 0x7f648c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c0c71f0 con 0x7f648c150990 2024-04-26T22:00:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f647c001670 con 0x7f648c150990 2024-04-26T22:00:13.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f373e6b0640 1 -- 172.21.15.135:0/4198589639 shutdown_connections 2024-04-26T22:00:13.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 >> v1:172.21.15.139:6789/0 conn(0x7f648c150990 legacy=0x7f648c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f6478003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478004800 0x7f6478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64780052e0 0x7f647800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f648c154620 con 0x7f64780039b0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f648c0c6cc0 con 0x7f6478004800 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f647800c260 con 0x7f64780052e0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f64910e2640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f6478003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f64918e3640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64780052e0 0x7f647800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f64908e1640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478004800 0x7f6478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc037070 con 0x7f30b80051c0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc02ad30 con 0x7f30b80051c0 2024-04-26T22:00:13.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.316+0000 7f30c67fc640 1 -- 172.21.15.135:0/981345554 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00f040 con 0x7f30b80051c0 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f70c3fff640 1 -- 172.21.15.135:0/1495837804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c415e3b0 msgr2=0x7f70c415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f70c3fff640 1 --2- 172.21.15.135:0/1495837804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c415e3b0 0x7f70c415e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f70c3fff640 1 -- 172.21.15.135:0/1495837804 >> v2:172.21.15.139:3300/0 conn(0x7f70c406c310 msgr2=0x7f70c40ac5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f70c3fff640 1 --2- 172.21.15.135:0/1495837804 >> v2:172.21.15.139:3300/0 conn(0x7f70c406c310 0x7f70c40ac5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f70c3fff640 1 -- 172.21.15.135:0/1495837804 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c4074ac0 con 0x7f70c4158600 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe23792b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2300adda0 0x7fe230153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe23792b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2300c5390 0x7fe230154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 msgr2=0x7f30b800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f30ccb08640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30b800b9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0098a0 tx=0x7f30b8005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f64910e2640 1 -- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64780052e0 msgr2=0x7f647800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f78d2f35640 1 -- 172.21.15.135:0/3218249344 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 msgr2=0x7f78c400acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f78d2f35640 1 --2- 172.21.15.135:0/3218249344 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 0x7f78c400acf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78c00098a0 tx=0x7f78c002a200 comp rx=0 tx=0).stop 2024-04-26T22:00:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe23792b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2301522c0 0x7fe2301553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe23792b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe23006da30 con 0x7fe2300adda0 2024-04-26T22:00:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe23792b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2300be980 con 0x7fe2300c5390 2024-04-26T22:00:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe23792b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe23006dbb0 con 0x7fe2301522c0 2024-04-26T22:00:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe2356c7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2300adda0 0x7fe230153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7fe234ec6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2300c5390 0x7fe230154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f64910e2640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64780052e0 0x7f647800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f78d2f35640 1 -- 172.21.15.135:0/3218249344 shutdown_connections 2024-04-26T22:00:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f78d2f35640 1 --2- 172.21.15.135:0/3218249344 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c400a8a0 0x7f78c400acf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f78d2f35640 1 -- 172.21.15.135:0/3218249344 >> 172.21.15.135:0/3218249344 conn(0x7f78c4048240 msgr2=0x7f78c4048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f64910e2640 1 -- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478004800 msgr2=0x7f6478004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f64910e2640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478004800 0x7f6478004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f67f3b19640 1 -- 172.21.15.135:0/1692548042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e40df1b0 msgr2=0x7f67e4006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f67f3b19640 1 --2- 172.21.15.135:0/1692548042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e40df1b0 0x7f67e4006ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67e400a2f0 tx=0x7f67dc029fb0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 -- 172.21.15.135:0/1692548042 shutdown_connections 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 --2- 172.21.15.135:0/1692548042 >> v2:172.21.15.139:3300/0 conn(0x7f67e4007410 0x7f67e400a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 --2- 172.21.15.135:0/1692548042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e40df1b0 0x7f67e4006ed0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67e400a2f0 tx=0x7f67dc029fb0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 --2- 172.21.15.135:0/1692548042 >> v2:172.21.15.135:3300/0 conn(0x7f67e40dbd10 0x7f67e40474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 -- 172.21.15.135:0/1692548042 >> 172.21.15.135:0/1692548042 conn(0x7f67e4048280 msgr2=0x7f67e4048690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2300adda0 0x7fe230153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40358/0 (socket says 172.21.15.135:40358) 2024-04-26T22:00:13.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 -- 172.21.15.135:0/1108040097 learned_addr learned my addr 172.21.15.135:0/1108040097 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f64918e3640 1 -- 172.21.15.135:0/2628457228 reap_dead start 2024-04-26T22:00:13.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f64910e2640 1 -- 172.21.15.135:0/2628457228 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c0c71f0 con 0x7f64780039b0 2024-04-26T22:00:13.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f20e41c6640 1 Processor -- start 2024-04-26T22:00:13.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe235ec8640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2301522c0 0x7fe2301553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.317+0000 7f373e6b0640 1 -- 172.21.15.135:0/4198589639 wait complete. 2024-04-26T22:00:13.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f64908e1640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6478004800 0x7f6478004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f78d2f35640 1 -- 172.21.15.135:0/3218249344 shutdown_connections 2024-04-26T22:00:13.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f78d2f35640 1 -- 172.21.15.135:0/3218249344 wait complete. 2024-04-26T22:00:13.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 -- 172.21.15.135:0/1108040097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2301522c0 msgr2=0x7fe2301553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2301522c0 0x7fe2301553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 -- 172.21.15.135:0/1108040097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2300c5390 msgr2=0x7fe230154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2300c5390 0x7fe230154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 -- 172.21.15.135:0/1108040097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2301e91d0 con 0x7fe2300adda0 2024-04-26T22:00:13.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 -- 172.21.15.135:0/1692548042 shutdown_connections 2024-04-26T22:00:13.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 shutdown_connections 2024-04-26T22:00:13.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f70c3fff640 1 --2- 172.21.15.135:0/1495837804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c4158600 0x7f70c40c4f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b0490 tx=0x7f70b802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f77711f1796805 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f30ccb08640 1 --2- 172.21.15.135:0/981345554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30b800b9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 >> 172.21.15.135:0/981345554 conn(0x7f30c0048250 msgr2=0x7f30c0048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f20e41c6640 1 -- start start 2024-04-26T22:00:13.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7fe2356c7640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2300adda0 0x7fe230153f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe220009720 tx=0x7fe2200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220040030 con 0x7fe2300adda0 2024-04-26T22:00:13.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7fe23792b640 1 -- 172.21.15.135:0/1108040097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe230157070 con 0x7fe2300adda0 2024-04-26T22:00:13.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f70c2ffd640 1 -- 172.21.15.135:0/1495837804 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8037070 con 0x7f70c4158600 2024-04-26T22:00:13.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f70c2ffd640 1 -- 172.21.15.135:0/1495837804 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b8032440 con 0x7f70c4158600 2024-04-26T22:00:13.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f64910e2640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f6478003dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c290 tx=0x7f6478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd915007a7f809f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 shutdown_connections 2024-04-26T22:00:13.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f30ccb08640 1 -- 172.21.15.135:0/981345554 wait complete. 2024-04-26T22:00:13.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f70ca8d9640 1 -- 172.21.15.135:0/1495837804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c4158600 msgr2=0x7f70c40c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/1495837804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c4158600 0x7f70c40c4f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b0490 tx=0x7f70b802a160 comp rx=0 tx=0).stop 2024-04-26T22:00:13.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f373e6b0640 1 Processor -- start 2024-04-26T22:00:13.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f373e6b0640 1 -- start start 2024-04-26T22:00:13.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f78d2f35640 1 Processor -- start 2024-04-26T22:00:13.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f78d2f35640 1 -- start start 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e41c6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20dc06f710 0x7f20dc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e41c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc0c3f00 0x7f20dc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e41c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20dc1500b0 0x7f20dc14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e41c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0abe20 con 0x7f20dc0c3f00 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e41c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc150a60 con 0x7f20dc1500b0 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e41c6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc0abfa0 con 0x7f20dc06f710 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e2763640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20dc1500b0 0x7f20dc14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e2763640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20dc1500b0 0x7f20dc14c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33304/0 (socket says 172.21.15.135:33304) 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e2763640 1 -- 172.21.15.135:0/2571721135 learned_addr learned my addr 172.21.15.135:0/2571721135 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e1761640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc0c3f00 0x7f20dc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.319+0000 7f20e1f62640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20dc06f710 0x7f20dc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.318+0000 7f67f3b19640 1 -- 172.21.15.135:0/1692548042 wait complete. 2024-04-26T22:00:13.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f67f3b19640 1 Processor -- start 2024-04-26T22:00:13.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f20e2763640 1 -- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20dc06f710 msgr2=0x7f20dc15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f20e2763640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20dc06f710 0x7f20dc15c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f20e2763640 1 -- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc0c3f00 msgr2=0x7f20dc153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f20e2763640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20dc0c3f00 0x7f20dc153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f20e2763640 1 -- 172.21.15.135:0/2571721135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc0c70d0 con 0x7f20dc1500b0 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 -- 172.21.15.135:0/1495837804 shutdown_connections 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/1495837804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c415e3b0 0x7f70c415e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/1495837804 >> v2:172.21.15.139:3300/0 conn(0x7f70c406c310 0x7f70c40ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/1495837804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c4158600 0x7f70c40c4f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 -- 172.21.15.135:0/1495837804 >> 172.21.15.135:0/1495837804 conn(0x7f70c40c6cf0 msgr2=0x7f70c40c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f20e2763640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20dc1500b0 0x7f20dc14c5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009eb0 tx=0x7f20d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 -- 172.21.15.135:0/1495837804 shutdown_connections 2024-04-26T22:00:13.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c036070 con 0x7f64780039b0 2024-04-26T22:00:13.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f647c005ad0 con 0x7f64780039b0 2024-04-26T22:00:13.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f373e6b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373000fce0 0x7f37300303a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f373e6b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3730050710 0x7f37300cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f6483fff640 1 -- 172.21.15.135:0/2628457228 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c007d80 con 0x7f64780039b0 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f373e6b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37300d7910 0x7f37300ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f373e6b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37300dad10 con 0x7f37300d7910 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f373e6b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f373004c290 con 0x7f373000fce0 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f373e6b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f37300dae90 con 0x7f3730050710 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f3737fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373000fce0 0x7f37300303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f3737fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373000fce0 0x7f37300303a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33318/0 (socket says 172.21.15.135:33318) 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f3737fff640 1 -- 172.21.15.135:0/4024511884 learned_addr learned my addr 172.21.15.135:0/4024511884 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f37377fe640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3730050710 0x7f37300cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d2f35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78c400a8a0 0x7f78c4030820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f373cc4d640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37300d7910 0x7f37300ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f67f3b19640 1 -- start start 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f3b19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e4007410 0x7f67e4044df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f3b19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67e40dbd10 0x7f67e40dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f3b19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e4045730 0x7f67e402edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f37377fe640 1 -- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373000fce0 msgr2=0x7f37300303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f37377fe640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373000fce0 0x7f37300303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f37377fe640 1 -- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37300d7910 msgr2=0x7f37300ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f37377fe640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37300d7910 0x7f37300ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f37377fe640 1 -- 172.21.15.135:0/4024511884 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37300cde60 con 0x7f3730050710 2024-04-26T22:00:13.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f3737fff640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373000fce0 0x7f37300303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f3b19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40dc0f0 con 0x7f67e40dbd10 2024-04-26T22:00:13.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d2f35640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c4047490 0x7f78c40d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f3b19640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4047db0 con 0x7f67e4045730 2024-04-26T22:00:13.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f3b19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67e40df5f0 con 0x7f67e4007410 2024-04-26T22:00:13.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f373cc4d640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37300d7910 0x7f37300ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:13.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f20b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e4045730 0x7f67e402edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f20b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e4045730 0x7f67e402edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33320/0 (socket says 172.21.15.135:33320) 2024-04-26T22:00:13.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f67f20b6640 1 -- 172.21.15.135:0/702469266 learned_addr learned my addr 172.21.15.135:0/702469266 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d1f33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78c400a8a0 0x7f78c4030820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d1f33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78c400a8a0 0x7f78c4030820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33326/0 (socket says 172.21.15.135:33326) 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d1f33640 1 -- 172.21.15.135:0/4221660115 learned_addr learned my addr 172.21.15.135:0/4221660115 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d2f35640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78c40dbcd0 0x7f78c40d8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f37377fe640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3730050710 0x7f37300cc5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f373806b030 tx=0x7f3738069f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f6493346640 1 -- 172.21.15.135:0/2628457228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 msgr2=0x7f6478003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f78d1732640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c4047490 0x7f78c40d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f6493346640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f6478003dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c290 tx=0x7f6478005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f10b4640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67e40dbd10 0x7f67e40dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe220038040 con 0x7fe2300adda0 2024-04-26T22:00:13.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f20b6640 1 -- 172.21.15.135:0/702469266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e4007410 msgr2=0x7f67e4044df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:00:13.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.320+0000 7f70ca8d9640 1 -- 172.21.15.135:0/1495837804 wait complete. 2024-04-26T22:00:13.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f70ca8d9640 1 Processor -- start 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f20b6640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e4007410 0x7f67e4044df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f20b6640 1 -- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67e40dbd10 msgr2=0x7f67e40dacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f78d2734640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78c40dbcd0 0x7f78c40d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f30ccb08640 1 Processor -- start 2024-04-26T22:00:13.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.321+0000 7f30ccb08640 1 -- start start 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f20b6640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67e40dbd10 0x7f67e40dacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78c400b230 con 0x7f78c40dbcd0 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4047d70 con 0x7f78c400a8a0 2024-04-26T22:00:13.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40dc0b0 con 0x7f78c4047490 2024-04-26T22:00:13.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f20b6640 1 -- 172.21.15.135:0/702469266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e40421f0 con 0x7f67e4045730 2024-04-26T22:00:13.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f10b4640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67e40dbd10 0x7f67e40dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738091070 con 0x7f3730050710 2024-04-26T22:00:13.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67f20b6640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e4045730 0x7f67e402edf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67ec051fc0 tx=0x7f67ec069190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373806f900 con 0x7f3730050710 2024-04-26T22:00:13.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f30ccb08640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30c002fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f6493346640 1 -- 172.21.15.135:0/2628457228 shutdown_connections 2024-04-26T22:00:13.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f6493346640 1 --2- 172.21.15.135:0/2628457228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f6478003dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f6493346640 1 -- 172.21.15.135:0/2628457228 >> 172.21.15.135:0/2628457228 conn(0x7f648c0cc340 msgr2=0x7f648c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30ccb08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c0030bd0 0x7f30c015fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30ccb08640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c01601b0 0x7f30c00d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30c77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30c002fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30c77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30c002fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40406/0 (socket says 172.21.15.135:40406) 2024-04-26T22:00:13.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f78d1f33640 1 -- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c4047490 msgr2=0x7f78c40d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c0030bd0 0x7f30c015fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c0030bd0 0x7f30c015fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33338/0 (socket says 172.21.15.135:33338) 2024-04-26T22:00:13.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f78d1f33640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78c4047490 0x7f78c40d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f78d1f33640 1 -- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78c40dbcd0 msgr2=0x7f78c40d8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30c77fe640 1 -- 172.21.15.135:0/3303518614 learned_addr learned my addr 172.21.15.135:0/3303518614 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00d9930 con 0x7f30b80051c0 2024-04-26T22:00:13.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec06f430 con 0x7f67e4045730 2024-04-26T22:00:13.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ec06f5d0 con 0x7f67e4045730 2024-04-26T22:00:13.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30c7fff640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c01601b0 0x7f30c00d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f70ca8d9640 1 -- start start 2024-04-26T22:00:13.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00da7e0 con 0x7f30c0030bd0 2024-04-26T22:00:13.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00d3f20 con 0x7f30c01601b0 2024-04-26T22:00:13.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f78d1f33640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78c40dbcd0 0x7f78c40d8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f78d1f33640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c40dcc70 con 0x7f78c400a8a0 2024-04-26T22:00:13.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec06f980 con 0x7f67e4045730 2024-04-26T22:00:13.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e4041150 con 0x7f67e4045730 2024-04-26T22:00:13.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e4041710 con 0x7f67e4045730 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f6493346640 1 -- 172.21.15.135:0/2628457228 shutdown_connections 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f6493346640 1 -- 172.21.15.135:0/2628457228 wait complete. 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7fe23792b640 1 -- 172.21.15.135:0/1108040097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe230152020 con 0x7fe2300adda0 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220037070 con 0x7fe2300adda0 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe220043070 con 0x7fe2300adda0 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7fe225ffb640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe21406c970 0x7fe21406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7fe23792b640 1 -- 172.21.15.135:0/1108040097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2301e8f80 con 0x7fe2300adda0 2024-04-26T22:00:13.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7fe234ec6640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe21406c970 0x7fe21406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70ca8d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c406c310 0x7f70c41ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70ca8d9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c4158600 0x7f70c40c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70ca8d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c415e3b0 0x7f70c40ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70ca8d9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70c406c7a0 con 0x7f70c415e3b0 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70ca8d9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41589e0 con 0x7f70c406c310 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70ca8d9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f70c406c920 con 0x7f70c4158600 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c4158600 0x7f70c40c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c4158600 0x7f70c40c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:33890/0 (socket says 172.21.15.135:33890) 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f70c37fe640 1 -- 172.21.15.135:0/3240233129 learned_addr learned my addr 172.21.15.135:0/3240233129 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373806fcb0 con 0x7f3730050710 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37300cdad0 con 0x7f3730050710 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3730032930 con 0x7f3730050710 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.323+0000 7f78d2734640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78c40dbcd0 0x7f78c40d8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f78d1f33640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78c400a8a0 0x7f78c4030820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c490 tx=0x7f78c800c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4040070 con 0x7f20dc1500b0 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d403c8f0 con 0x7f20dc1500b0 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d403cca0 con 0x7f20dc1500b0 2024-04-26T22:00:13.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f20e41c6640 1 -- 172.21.15.135:0/2571721135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc157330 con 0x7f20dc1500b0 2024-04-26T22:00:13.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.322+0000 7f20e41c6640 1 -- 172.21.15.135:0/2571721135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1e98f0 con 0x7f20dc1500b0 2024-04-26T22:00:13.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe220077860 con 0x7fe2300adda0 2024-04-26T22:00:13.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7fe225ffb640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7fe214077190 0x7fe2140795b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 --> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe220001d30 con 0x7fe214077190 2024-04-26T22:00:13.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c80052d0 con 0x7f78c400a8a0 2024-04-26T22:00:13.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c8005470 con 0x7f78c400a8a0 2024-04-26T22:00:13.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8012870 con 0x7f78c400a8a0 2024-04-26T22:00:13.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7fe235ec8640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7fe214077190 0x7fe2140795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7fe234ec6640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe21406c970 0x7fe21406ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2300c3900 tx=0x7fe228009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f70c3fff640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c406c310 0x7f70c41ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f70c8e76640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c415e3b0 0x7f70c40ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f70c8e76640 1 -- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c4158600 msgr2=0x7f70c40c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f70c8e76640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70c4158600 0x7f70c40c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f70c8e76640 1 -- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c406c310 msgr2=0x7f70c41ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f70c8e76640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c406c310 0x7f70c41ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f70c8e76640 1 -- 172.21.15.135:0/3240233129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c40c9f40 con 0x7f70c415e3b0 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f70c8e76640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c415e3b0 0x7f70c40ae9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00eab0 tx=0x7f70bc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67e40053c0 con 0x7f67e4045730 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f30c7fff640 1 -- 172.21.15.135:0/3303518614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c0030bd0 msgr2=0x7f30c015fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f30c7fff640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c0030bd0 0x7f30c015fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f30c7fff640 1 -- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 msgr2=0x7f30c002fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f30c7fff640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30b80051c0 0x7f30c002fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f30c7fff640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c016ffc0 con 0x7f30c01601b0 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7fe235ec8640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7fe214077190 0x7fe2140795b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe220075a60 con 0x7fe2300adda0 2024-04-26T22:00:13.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.324+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20d4039040 con 0x7f20dc1500b0 2024-04-26T22:00:13.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3730005380 con 0x7f3730050710 2024-04-26T22:00:13.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c40dcf50 con 0x7f78c400a8a0 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20dc154080 con 0x7f20dc1500b0 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f20d27fc640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c006c9c0 0x7f20c006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f20d40ad390 con 0x7f20dc1500b0 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f20d27fc640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f20c0077260 0x7f20c0079680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 --> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20c0079d30 con 0x7f20c0077260 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20d40b4050 con 0x7f20dc1500b0 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f20e1f62640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c006c9c0 0x7f20c006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c40dd510 con 0x7f78c400a8a0 2024-04-26T22:00:13.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f6493346640 1 Processor -- start 2024-04-26T22:00:13.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f6493346640 1 -- start start 2024-04-26T22:00:13.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f20e1f62640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c006c9c0 0x7f20c006ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f20cc001010 tx=0x7f20cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.325+0000 7f30c7fff640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c01601b0 0x7f30c00d22b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f30c0048fa0 tx=0x7f30bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc041020 con 0x7f30c01601b0 2024-04-26T22:00:13.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc00f000 con 0x7f30c01601b0 2024-04-26T22:00:13.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc018020 con 0x7f70c415e3b0 2024-04-26T22:00:13.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== osd.3 v2:172.21.15.135:6800/3564981873 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe220001d30 con 0x7fe214077190 2024-04-26T22:00:13.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37380b3690 con 0x7f3730050710 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41f9d70 con 0x7f70c415e3b0 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.326+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41fc9e0 con 0x7f70c415e3b0 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70bc00bc00 con 0x7f70c415e3b0 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc014400 con 0x7f70c415e3b0 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc037040 con 0x7f30c01601b0 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c0042ab0 con 0x7f30c01601b0 2024-04-26T22:00:13.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c0042630 con 0x7f30c01601b0 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f6493346640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f648c0ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f6493346640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f648c158340 0x7f648c1e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f64910e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f648c0ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f64910e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f648c0ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40418/0 (socket says 172.21.15.135:40418) 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f6493346640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f648c15ab30 0x7f648c1e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f3734ff9640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f371c06c9c0 0x7f371c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f3738108540 con 0x7f3730050710 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f3734ff9640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f371c077260 0x7f371c079680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 --> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3738098e70 con 0x7f371c077260 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3738108930 con 0x7f3730050710 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f6493346640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f648c150d70 con 0x7f64780039b0 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f6493346640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f648c0cb210 con 0x7f648c15ab30 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f6493346640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f648c157870 con 0x7f648c158340 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f64910e2640 1 -- 172.21.15.135:0/750688684 learned_addr learned my addr 172.21.15.135:0/750688684 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70a4001030 con 0x7f70c415e3b0 2024-04-26T22:00:13.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f3737fff640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f371c06c9c0 0x7f371c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64908e1640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f648c158340 0x7f648c1e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64918e3640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f648c15ab30 0x7f648c1e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f20e1761640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f20c0077260 0x7f20c0079680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.327+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67ec06fb20 con 0x7f67e4045730 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f67e27fc640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f67d006c9c0 0x7f67d006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f67ec108bb0 con 0x7f67e4045730 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f67e27fc640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f67d0077250 0x7f67d0079670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 --> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67ec099e50 con 0x7f67d0077250 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f67ec108fa0 con 0x7f67e4045730 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f3737fff640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f371c06c9c0 0x7f371c06ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f372c000c00 tx=0x7f372c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f373cc4d640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f371c077260 0x7f371c079680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64908e1640 1 -- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f648c15ab30 msgr2=0x7f648c1e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64908e1640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f648c15ab30 0x7f648c1e97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64908e1640 1 -- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 msgr2=0x7f648c0ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64908e1640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f648c0ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.328+0000 7f64908e1640 1 -- 172.21.15.135:0/750688684 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c156540 con 0x7f648c158340 2024-04-26T22:00:13.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f64908e1640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f648c158340 0x7f648c1e9290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f648c063ef0 tx=0x7f648800ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f67f10b4640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f67d0077250 0x7f67d0079670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f64910e2640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64780039b0 0x7f648c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30c00428a0 con 0x7f30c01601b0 2024-04-26T22:00:13.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f78affff640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78c4005380 con 0x7f78c400a8a0 2024-04-26T22:00:13.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f20e1761640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f20c0077260 0x7f20c0079680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30bc044070 con 0x7f30c01601b0 2024-04-26T22:00:13.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488016070 con 0x7f648c158340 2024-04-26T22:00:13.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648800ee50 con 0x7f648c158340 2024-04-26T22:00:13.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488009460 con 0x7f648c158340 2024-04-26T22:00:13.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c1ecbb0 con 0x7f648c158340 2024-04-26T22:00:13.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f373cc4d640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f371c077260 0x7f371c079680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f30b3fff640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30ac070e00 0x7f30ac0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c154980 con 0x7f648c158340 2024-04-26T22:00:13.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f30bc0ab070 con 0x7f30c01601b0 2024-04-26T22:00:13.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70bc01b020 con 0x7f70c415e3b0 2024-04-26T22:00:13.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c0ff9640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f70a806c940 0x7f70a806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f70bc08c790 con 0x7f70c415e3b0 2024-04-26T22:00:13.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f67f18b5640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f67d006c9c0 0x7f67d006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.329+0000 7f67f10b4640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f67d0077250 0x7f67d0079670 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f30b3fff640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f30ac07b620 0x7f30ac07da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30bc035e70 con 0x7f30ac07b620 2024-04-26T22:00:13.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30bc0b2050 con 0x7f30c01601b0 2024-04-26T22:00:13.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f30c77fe640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30ac070e00 0x7f30ac0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c0ff9640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f70a80771e0 0x7f70a8079600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 --> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70bc00df10 con 0x7f70a80771e0 2024-04-26T22:00:13.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78c8012a10 con 0x7f78c400a8a0 2024-04-26T22:00:13.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f70bc08cc70 con 0x7f70c415e3b0 2024-04-26T22:00:13.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c3fff640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f70a806c940 0x7f70a806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f70c37fe640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f70a80771e0 0x7f70a8079600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.330+0000 7f30c6ffd640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f30ac07b620 0x7f30ac07da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== osd.7 v2:172.21.15.139:6824/3012659507 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20c0079d30 con 0x7f20c0077260 2024-04-26T22:00:13.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== osd.5 v2:172.21.15.139:6800/2023089101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3738098e70 con 0x7f371c077260 2024-04-26T22:00:13.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f30c77fe640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30ac070e00 0x7f30ac0732c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f30c806bd80 tx=0x7f30c808f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f646f7fe640 1 -- 172.21.15.135:0/750688684 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f648c0ca3d0 con 0x7f648c158340 2024-04-26T22:00:13.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f70c37fe640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f70a80771e0 0x7f70a8079600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f78be7fc640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f78b006c9f0 0x7f78b006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f70c3fff640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f70a806c940 0x7f70a806ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f70b8034000 tx=0x7f70b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f78c808ba40 con 0x7f78c400a8a0 2024-04-26T22:00:13.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f78be7fc640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7f78b0077290 0x7f78b00796b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78c8004ec0 con 0x7f78b0077290 2024-04-26T22:00:13.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78c80902a0 con 0x7f78c400a8a0 2024-04-26T22:00:13.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f30c6ffd640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f30ac07b620 0x7f30ac07da40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== osd.0 v2:172.21.15.135:6816/3857357919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67ec099e50 con 0x7f67d0077250 2024-04-26T22:00:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f67f18b5640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f67d006c9c0 0x7f67d006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b7e0 tx=0x7f67e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.331+0000 7f78d1732640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f78b006c9f0 0x7f78b006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f78d2734640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7f78b0077290 0x7f78b00796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6488019070 con 0x7f648c158340 2024-04-26T22:00:13.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== osd.4 v2:172.21.15.139:6808/3666291794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f70bc00df10 con 0x7f70a80771e0 2024-04-26T22:00:13.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f78d1732640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f78b006c9f0 0x7f78b006eeb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f78cc051470 tx=0x7f78cc063d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f78d2734640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7f78b0077290 0x7f78b00796b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== osd.1 v2:172.21.15.135:6808/3325535972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30bc035e70 con 0x7f30ac07b620 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f6481ffb640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f647006c9f0 0x7f647006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.332+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f648808b5f0 con 0x7f648c158340 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f64910e2640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f647006c9f0 0x7f647006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f6481ffb640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6470077200 0x7f6470079620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 --> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f648801ee50 con 0x7f6470077200 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f64918e3640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6470077200 0x7f6470079620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6488092050 con 0x7f648c158340 2024-04-26T22:00:13.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f64910e2640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f647006c9f0 0x7f647006eeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f647c001820 tx=0x7f647c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.333+0000 7f64918e3640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6470077200 0x7f6470079620 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.335+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== osd.2 v2:172.21.15.135:6824/626924944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f648801ee50 con 0x7f6470077200 2024-04-26T22:00:13.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.336+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== osd.6 v2:172.21.15.139:6816/3816888463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78c8004ec0 con 0x7f78b0077290 2024-04-26T22:00:13.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.359+0000 7fe23792b640 1 -- 172.21.15.135:0/1108040097 --> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2300507a0 con 0x7fe214077190 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.360+0000 7fe225ffb640 1 -- 172.21.15.135:0/1108040097 <== osd.3 v2:172.21.15.135:6800/3564981873 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2300507a0 con 0x7fe214077190 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.363+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7fe214077190 msgr2=0x7fe2140795b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.363+0000 7fe21b7fe640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7fe214077190 0x7fe2140795b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.363+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe21406c970 msgr2=0x7fe21406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.363+0000 7fe21b7fe640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe21406c970 0x7fe21406ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2300c3900 tx=0x7fe228009040 comp rx=0 tx=0).stop 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2300adda0 msgr2=0x7fe230153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe21b7fe640 1 --2- 172.21.15.135:0/1108040097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2300adda0 0x7fe230153f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe220009720 tx=0x7fe2200046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe235ec8640 1 -- 172.21.15.135:0/1108040097 reap_dead start 2024-04-26T22:00:13.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 --> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20dc0507a0 con 0x7f20c0077260 2024-04-26T22:00:13.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 shutdown_connections 2024-04-26T22:00:13.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 >> 172.21.15.135:0/1108040097 conn(0x7fe2300c91e0 msgr2=0x7fe230070150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 shutdown_connections 2024-04-26T22:00:13.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7fe21b7fe640 1 -- 172.21.15.135:0/1108040097 wait complete. 2024-04-26T22:00:13.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.364+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 --> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37300daaf0 con 0x7f371c077260 2024-04-26T22:00:13.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.365+0000 7f20d27fc640 1 -- 172.21.15.135:0/2571721135 <== osd.7 v2:172.21.15.139:6824/3012659507 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f20dc0507a0 con 0x7f20c0077260 2024-04-26T22:00:13.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.365+0000 7f3734ff9640 1 -- 172.21.15.135:0/4024511884 <== osd.5 v2:172.21.15.139:6800/2023089101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f37300daaf0 con 0x7f371c077260 2024-04-26T22:00:13.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.365+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 --> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67e400b5c0 con 0x7f67d0077250 2024-04-26T22:00:13.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.365+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 --> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f70a4002dd0 con 0x7f70a80771e0 2024-04-26T22:00:13.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.365+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f20c0077260 msgr2=0x7f20c0079680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.365+0000 7f20bffff640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f20c0077260 0x7f20c0079680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c006c9c0 msgr2=0x7f20c006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20bffff640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c006c9c0 0x7f20c006ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f20cc001010 tx=0x7f20cc009040 comp rx=0 tx=0).stop 2024-04-26T22:00:13.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f70c0ff9640 1 -- 172.21.15.135:0/3240233129 <== osd.4 v2:172.21.15.139:6808/3666291794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f70a4002dd0 con 0x7f70a80771e0 2024-04-26T22:00:13.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20dc1500b0 msgr2=0x7f20dc14c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20bffff640 1 --2- 172.21.15.135:0/2571721135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20dc1500b0 0x7f20dc14c5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009eb0 tx=0x7f20d4004a40 comp rx=0 tx=0).stop 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f371c077260 msgr2=0x7f371c079680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f371c077260 0x7f371c079680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f371c06c9c0 msgr2=0x7f371c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f371c06c9c0 0x7f371c06ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f372c000c00 tx=0x7f372c02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3730050710 msgr2=0x7f37300cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 --2- 172.21.15.135:0/4024511884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3730050710 0x7f37300cc5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f373806b030 tx=0x7f3738069f90 comp rx=0 tx=0).stop 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373cc4d640 1 -- 172.21.15.135:0/4024511884 reap_dead start 2024-04-26T22:00:13.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 shutdown_connections 2024-04-26T22:00:13.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 >> 172.21.15.135:0/4024511884 conn(0x7f373004fbf0 msgr2=0x7f37300d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20e2763640 1 -- 172.21.15.135:0/2571721135 reap_dead start 2024-04-26T22:00:13.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 shutdown_connections 2024-04-26T22:00:13.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 >> 172.21.15.135:0/2571721135 conn(0x7f20dc06c310 msgr2=0x7f20dc06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 --> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30c0005390 con 0x7f30ac07b620 2024-04-26T22:00:13.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f67e27fc640 1 -- 172.21.15.135:0/702469266 <== osd.0 v2:172.21.15.135:6816/3857357919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f67e400b5c0 con 0x7f67d0077250 2024-04-26T22:00:13.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f70a80771e0 msgr2=0x7f70a8079600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f70a80771e0 0x7f70a8079600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.366+0000 7f30b3fff640 1 -- 172.21.15.135:0/3303518614 <== osd.1 v2:172.21.15.135:6808/3325535972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30c0005390 con 0x7f30ac07b620 2024-04-26T22:00:13.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f70a806c940 msgr2=0x7f70a806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f70a806c940 0x7f70a806ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f70b8034000 tx=0x7f70b802e000 comp rx=0 tx=0).stop 2024-04-26T22:00:13.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 shutdown_connections 2024-04-26T22:00:13.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c415e3b0 msgr2=0x7f70c40ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f70ca8d9640 1 --2- 172.21.15.135:0/3240233129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70c415e3b0 0x7f70c40ae9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00eab0 tx=0x7f70bc00ef70 comp rx=0 tx=0).stop 2024-04-26T22:00:13.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f70c8e76640 1 -- 172.21.15.135:0/3240233129 reap_dead start 2024-04-26T22:00:13.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f373e6b0640 1 -- 172.21.15.135:0/4024511884 wait complete. 2024-04-26T22:00:13.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f30ac07b620 msgr2=0x7f30ac07da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f30ccb08640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f30ac07b620 0x7f30ac07da40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 shutdown_connections 2024-04-26T22:00:13.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30ac070e00 msgr2=0x7f30ac0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f30ccb08640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30ac070e00 0x7f30ac0732c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f30c806bd80 tx=0x7f30c808f040 comp rx=0 tx=0).stop 2024-04-26T22:00:13.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c01601b0 msgr2=0x7f30c00d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f30ccb08640 1 --2- 172.21.15.135:0/3303518614 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c01601b0 0x7f30c00d22b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f30c0048fa0 tx=0x7f30bc004730 comp rx=0 tx=0).stop 2024-04-26T22:00:13.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f67d0077250 msgr2=0x7f67d0079670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f3b19640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f67d0077250 0x7f67d0079670 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f20bffff640 1 -- 172.21.15.135:0/2571721135 wait complete. 2024-04-26T22:00:13.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f30c7fff640 1 -- 172.21.15.135:0/3303518614 reap_dead start 2024-04-26T22:00:13.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f67d006c9c0 msgr2=0x7f67d006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f3b19640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f67d006c9c0 0x7f67d006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b7e0 tx=0x7f67e8008040 comp rx=0 tx=0).stop 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e4045730 msgr2=0x7f67e402edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f3b19640 1 --2- 172.21.15.135:0/702469266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e4045730 0x7f67e402edf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67ec051fc0 tx=0x7f67ec069190 comp rx=0 tx=0).stop 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.367+0000 7f67f20b6640 1 -- 172.21.15.135:0/702469266 reap_dead start 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 shutdown_connections 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 >> 172.21.15.135:0/3303518614 conn(0x7f30c0048250 msgr2=0x7f30c00493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 shutdown_connections 2024-04-26T22:00:13.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 >> 172.21.15.135:0/3240233129 conn(0x7f70c40c6cf0 msgr2=0x7f70c40c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 shutdown_connections 2024-04-26T22:00:13.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 >> 172.21.15.135:0/702469266 conn(0x7f67e4048280 msgr2=0x7f67e402cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 shutdown_connections 2024-04-26T22:00:13.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 shutdown_connections 2024-04-26T22:00:13.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.368+0000 7f67f3b19640 1 -- 172.21.15.135:0/702469266 wait complete. 2024-04-26T22:00:13.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.369+0000 7f30ccb08640 1 -- 172.21.15.135:0/3303518614 wait complete. 2024-04-26T22:00:13.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.373+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 shutdown_connections 2024-04-26T22:00:13.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.373+0000 7f70ca8d9640 1 -- 172.21.15.135:0/3240233129 wait complete. 2024-04-26T22:00:13.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.375+0000 7f78affff640 1 -- 172.21.15.135:0/4221660115 --> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78c4177520 con 0x7f78b0077290 2024-04-26T22:00:13.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.376+0000 7f78be7fc640 1 -- 172.21.15.135:0/4221660115 <== osd.6 v2:172.21.15.139:6816/3816888463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78c4177520 con 0x7f78b0077290 2024-04-26T22:00:13.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7f78b0077290 msgr2=0x7f78b00796b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7f78b0077290 0x7f78b00796b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f78b006c9f0 msgr2=0x7f78b006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f78b006c9f0 0x7f78b006eeb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f78cc051470 tx=0x7f78cc063d80 comp rx=0 tx=0).stop 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78c400a8a0 msgr2=0x7f78c4030820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 --2- 172.21.15.135:0/4221660115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78c400a8a0 0x7f78c4030820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c490 tx=0x7f78c800c950 comp rx=0 tx=0).stop 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2734640 1 -- 172.21.15.135:0/4221660115 reap_dead start 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 shutdown_connections 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.378+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 >> 172.21.15.135:0/4221660115 conn(0x7f78c4048240 msgr2=0x7f78c402bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.380+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 shutdown_connections 2024-04-26T22:00:13.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.383+0000 7f646f7fe640 1 -- 172.21.15.135:0/750688684 --> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f648c1ed0d0 con 0x7f6470077200 2024-04-26T22:00:13.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.384+0000 7f6481ffb640 1 -- 172.21.15.135:0/750688684 <== osd.2 v2:172.21.15.135:6824/626924944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f648c1ed0d0 con 0x7f6470077200 2024-04-26T22:00:13.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.383+0000 7f78d2f35640 1 -- 172.21.15.135:0/4221660115 wait complete. 2024-04-26T22:00:13.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6470077200 msgr2=0x7f6470079620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6470077200 0x7f6470079620 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f647006c9f0 msgr2=0x7f647006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f647006c9f0 0x7f647006eeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f647c001820 tx=0x7f647c009030 comp rx=0 tx=0).stop 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f648c158340 msgr2=0x7f648c1e9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 --2- 172.21.15.135:0/750688684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f648c158340 0x7f648c1e9290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f648c063ef0 tx=0x7f648800ec00 comp rx=0 tx=0).stop 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f64918e3640 1 -- 172.21.15.135:0/750688684 reap_dead start 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 shutdown_connections 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.385+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 >> 172.21.15.135:0/750688684 conn(0x7f648c0cc340 msgr2=0x7f648c14ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.386+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 shutdown_connections 2024-04-26T22:00:13.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.386+0000 7f6493346640 1 -- 172.21.15.135:0/750688684 wait complete. 2024-04-26T22:00:13.436 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:13.436 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T22:00:13.437 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:13.437 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T22:00:13.438 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-26T22:00:13.438 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T22:00:13.439 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:13.439 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T22:00:13.440 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:13.440 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T22:00:13.441 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:13.441 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T22:00:13.442 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-26T22:00:13.442 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T22:00:13.443 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:13.443 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T22:00:13.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.644+0000 7f816fac4640 1 Processor -- start 2024-04-26T22:00:13.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.644+0000 7f42fd8a9640 1 Processor -- start 2024-04-26T22:00:13.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.645+0000 7f9887af9640 1 Processor -- start 2024-04-26T22:00:13.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f9887af9640 1 -- start start 2024-04-26T22:00:13.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f8fa2f97640 1 Processor -- start 2024-04-26T22:00:13.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.645+0000 7f816fac4640 1 -- start start 2024-04-26T22:00:13.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f816fac4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81600071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f816fac4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f816000f9d0 con 0x7f816004dff0 2024-04-26T22:00:13.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f816fac4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81600da4b0 con 0x7f816004dae0 2024-04-26T22:00:13.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f816fac4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81600da790 con 0x7f81600d7ae0 2024-04-26T22:00:13.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7ffaa6652640 1 Processor -- start 2024-04-26T22:00:13.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f8fa2f97640 1 -- start start 2024-04-26T22:00:13.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7ffaa6652640 1 -- start start 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f816e061640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f816004dae0 0x7f81600e4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57318/0 (socket says 172.21.15.135:57318) 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f816e061640 1 -- 172.21.15.135:0/2951801882 learned_addr learned my addr 172.21.15.135:0/2951801882 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948189732 0 0) 0x7f81600da4b0 con 0x7f816004dae0 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8154003520 con 0x7f816004dae0 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816d860640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81600071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4030779561 0 0) 0x7f8154003520 con 0x7f816004dae0 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 msgr2=0x7f81600071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816c85e640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81600071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 >> v1:172.21.15.135:6789/0 conn(0x7f816004dff0 legacy=0x7f816004e7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.645+0000 7f42fd8a9640 1 -- start start 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f42fd8a9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42f80c4850 0x7f42f80aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f42fd8a9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f80ac010 con 0x7f42f80c3860 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f42fd8a9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f80ac2f0 con 0x7f42f806e4c0 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f42fd8a9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f42f80ab7d0 con 0x7f42f80c4850 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f42f806e4c0 0x7f42f806e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57322/0 (socket says 172.21.15.135:57322) 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f77fe640 1 -- 172.21.15.135:0/1289534736 learned_addr learned my addr 172.21.15.135:0/1289534736 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871584577 0 0) 0x7f42f80ac2f0 con 0x7f42f806e4c0 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42e4003520 con 0x7f42f806e4c0 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42effff640 1 --2- 172.21.15.135:0/1289534736 >> v2:172.21.15.139:3300/0 conn(0x7f42f80c4850 0x7f42f80aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3578972661 0 0) 0x7f42e4003520 con 0x7f42f806e4c0 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 >> v2:172.21.15.139:3300/0 conn(0x7f42f80c4850 msgr2=0x7f42f80aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f67fc640 1 --2- 172.21.15.135:0/1289534736 >> v2:172.21.15.139:3300/0 conn(0x7f42f80c4850 0x7f42f80aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 >> v1:172.21.15.135:6789/0 conn(0x7f42f80c3860 legacy=0x7f42f815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f80ab460 con 0x7f42f806e4c0 2024-04-26T22:00:13.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f42f00019f0 con 0x7f42f806e4c0 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7ffaa6652640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffaa00c33f0 0x7ffaa00c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7ffaa6652640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa015bb40 con 0x7ffaa0158610 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7ffaa6652640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa00c3920 con 0x7ffaa00c4e30 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7ffaa6652640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa01589f0 con 0x7ffaa00c33f0 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ffaa0158610 0x7ffaa006f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52968/0 (socket says 172.21.15.135:52968) 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9ffff640 1 -- 172.21.15.135:0/1775710217 learned_addr learned my addr 172.21.15.135:0/1775710217 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f8fa2f97640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8f9c14fe70 0x7f8f9c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7fc55dc09640 1 Processor -- start 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7fc55dc09640 1 -- start start 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7fc55dc09640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc5500dbce0 0x7fc5500474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7fc55dc09640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc55000bb50 con 0x7fc5500df180 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7fc55dc09640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc550048060 con 0x7fc55000a8b0 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7fc55dc09640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc55000a420 con 0x7fc5500dbce0 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7fc557fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc5500df180 0x7fc550006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52990/0 (socket says 172.21.15.135:52990) 2024-04-26T22:00:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7fc557fff640 1 -- 172.21.15.135:0/626628462 learned_addr learned my addr 172.21.15.135:0/626628462 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f9887af9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9880158270 0x7f98800c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f9887af9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98800abe80 0x7f9880151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f9887af9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f98800c5170 0x7f9880152310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f9887af9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f98800bfca0 con 0x7f98800abe80 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f9887af9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f98800bff80 con 0x7f98800c5170 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f9887af9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f98800c62c0 con 0x7f9880158270 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98800abe80 0x7f9880151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98800abe80 0x7f9880151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40426/0 (socket says 172.21.15.135:40426) 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 -- 172.21.15.135:0/2418728357 learned_addr learned my addr 172.21.15.135:0/2418728357 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.646+0000 7f29cffff640 1 Processor -- start 2024-04-26T22:00:13.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.647+0000 7f29cffff640 1 -- start start 2024-04-26T22:00:13.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 >> v1:172.21.15.139:6789/0 conn(0x7f42f806e4c0 legacy=0x7f42f806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8f9bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8f9c0694b0 0x7f8f9c06eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52984/0 (socket says 172.21.15.135:52984) 2024-04-26T22:00:13.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e4003a00 0x7f42e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8f9bfff640 1 -- 172.21.15.135:0/1544188771 learned_addr learned my addr 172.21.15.135:0/1544188771 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/1544188771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0c3440 0x7f8f9c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885895640 1 --2- 172.21.15.135:0/2418728357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9880158270 0x7f98800c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f29cffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29d006f4b0 0x7f29d006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42e4004800 0x7f42e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e40052e0 0x7f42e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8fa0d33640 1 --2- 172.21.15.135:0/1544188771 >> v2:172.21.15.139:3300/0 conn(0x7f8f9c14fe70 0x7f8f9c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80ab7d0 con 0x7f42e4003a00 2024-04-26T22:00:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9c0ab460 con 0x7f8f9c0694b0 2024-04-26T22:00:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8fa1534640 1 --2- 172.21.15.135:0/1544188771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0c3440 0x7f8f9c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f29cffff640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d006d390 con 0x7f29d006e410 2024-04-26T22:00:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f08582f8640 1 Processor -- start 2024-04-26T22:00:13.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f08582f8640 1 -- start start 2024-04-26T22:00:13.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42e400c230 con 0x7f42e40052e0 2024-04-26T22:00:13.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42e400c430 con 0x7f42e4004800 2024-04-26T22:00:13.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9c153a60 con 0x7f8f9c14fe70 2024-04-26T22:00:13.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c0c3ba0 con 0x7f8f9c0c3440 2024-04-26T22:00:13.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f42f77fe640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e40052e0 0x7f42e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2554808521 0 0) 0x7fc55000bb50 con 0x7fc5500df180 2024-04-26T22:00:13.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc548003520 con 0x7fc5500df180 2024-04-26T22:00:13.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 -- 172.21.15.135:0/2418728357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9880158270 msgr2=0x7f98800c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 --2- 172.21.15.135:0/2418728357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9880158270 0x7f98800c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 -- 172.21.15.135:0/2418728357 >> v2:172.21.15.139:3300/0 conn(0x7f98800c5170 msgr2=0x7f9880152310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7f9885094640 1 --2- 172.21.15.135:0/2418728357 >> v2:172.21.15.139:3300/0 conn(0x7f98800c5170 0x7f9880152310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f9885094640 1 -- 172.21.15.135:0/2418728357 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98800c65a0 con 0x7f98800abe80 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f42effff640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42e4004800 0x7f42e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9f7fe640 1 --2- 172.21.15.135:0/1775710217 >> v2:172.21.15.139:3300/0 conn(0x7ffaa00c33f0 0x7ffaa00c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864670874 0 0) 0x7ffaa015bb40 con 0x7ffaa0158610 2024-04-26T22:00:13.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa88003520 con 0x7ffaa0158610 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829766762 0 0) 0x7ffaa00c3920 con 0x7ffaa00c4e30 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa015bb40 con 0x7ffaa00c4e30 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9f7fe640 1 -- 172.21.15.135:0/1775710217 >> v1:172.21.15.139:6789/0 conn(0x7ffaa00c4e30 legacy=0x7ffaa00c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9f7fe640 1 -- 172.21.15.135:0/1775710217 >> v1:172.21.15.135:6789/0 conn(0x7ffaa0158610 legacy=0x7ffaa006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.648+0000 7ffa9f7fe640 1 -- 172.21.15.135:0/1775710217 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa006ee80 con 0x7ffaa00c33f0 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98965599 0 0) 0x7fc550048060 con 0x7fc55000a8b0 2024-04-26T22:00:13.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc55000bb50 con 0x7fc55000a8b0 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f29cffff640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d00c2bf0 con 0x7f29d00c40c0 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f29cffff640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f29d00c3520 con 0x7f29d006f4b0 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f29cf7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f29d00c40c0 0x7f29d015d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57348/0 (socket says 172.21.15.135:57348) 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f29cf7fe640 1 -- 172.21.15.135:0/2410233901 learned_addr learned my addr 172.21.15.135:0/2410233901 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81600daa70 con 0x7f816004dae0 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f8168050be0 con 0x7f816004dae0 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 >> v1:172.21.15.139:6789/0 conn(0x7f816004dae0 legacy=0x7f81600e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8154003fc0 0x7f8154004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8154004d40 0x7f81540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81600da790 con 0x7f8154003fc0 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f815400bb00 con 0x7f8154004d40 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f815400bd00 con 0x7f81600d7ae0 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7fc55cc07640 1 --2- 172.21.15.135:0/626628462 >> v2:172.21.15.135:3300/0 conn(0x7fc5500dbce0 0x7fc5500474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f8fa1534640 1 -- 172.21.15.135:0/1544188771 >> v2:172.21.15.139:3300/0 conn(0x7f8f9c14fe70 msgr2=0x7f8f9c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f8fa1534640 1 --2- 172.21.15.135:0/1544188771 >> v2:172.21.15.139:3300/0 conn(0x7f8f9c14fe70 0x7f8f9c0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f8fa1534640 1 -- 172.21.15.135:0/1544188771 >> v1:172.21.15.135:6789/0 conn(0x7f8f9c0694b0 legacy=0x7f8f9c06eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f8fa1534640 1 -- 172.21.15.135:0/1544188771 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c0c3e80 con 0x7f8f9c0c3440 2024-04-26T22:00:13.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816e061640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8154004d40 0x7f81540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f08582f8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0848049f30 0x7f08480315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f08582f8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08480d6910 con 0x7f084804a440 2024-04-26T22:00:13.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f08582f8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08480d6bb0 con 0x7f0848007ad0 2024-04-26T22:00:13.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f08582f8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0848006940 con 0x7f0848049f30 2024-04-26T22:00:13.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f42effff640 1 -- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e40052e0 msgr2=0x7f42e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f42effff640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e40052e0 0x7f42e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f42effff640 1 -- 172.21.15.135:0/1289534736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e4003a00 msgr2=0x7f42e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f42effff640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e4003a00 0x7f42e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.649+0000 7f42effff640 1 -- 172.21.15.135:0/1289534736 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e4003520 con 0x7f42e4004800 2024-04-26T22:00:13.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7ffa9f7fe640 1 --2- 172.21.15.135:0/1775710217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa00c33f0 0x7ffaa00c6db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa900097a0 tx=0x7ffa9002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be116e5fdc91134 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f0856895640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0848007ad0 0x7f084800a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57350/0 (socket says 172.21.15.135:57350) 2024-04-26T22:00:13.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f0856895640 1 -- 172.21.15.135:0/1568606594 learned_addr learned my addr 172.21.15.135:0/1568606594 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:13.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285690372 0 0) 0x7f29d00c2bf0 con 0x7f29d00c40c0 2024-04-26T22:00:13.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29b8003520 con 0x7f29d00c40c0 2024-04-26T22:00:13.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f29ce7fc640 1 --2- 172.21.15.135:0/2410233901 >> v2:172.21.15.135:3300/0 conn(0x7f29d006f4b0 0x7f29d006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f0856094640 1 --2- 172.21.15.135:0/1568606594 >> v2:172.21.15.135:3300/0 conn(0x7f0848049f30 0x7f08480315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90037070 con 0x7ffaa00c33f0 2024-04-26T22:00:13.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9002ae50 con 0x7ffaa00c33f0 2024-04-26T22:00:13.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7ffa9effd640 1 -- 172.21.15.135:0/1775710217 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90032660 con 0x7ffaa00c33f0 2024-04-26T22:00:13.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006679679 0 0) 0x7f08480d6bb0 con 0x7f0848007ad0 2024-04-26T22:00:13.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f083c003520 con 0x7f0848007ad0 2024-04-26T22:00:13.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f8fa1534640 1 --2- 172.21.15.135:0/1544188771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0c3440 0x7f8f9c15e640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f940097a0 tx=0x7f8f9402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c36d2ccb5b834da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1911786032 0 0) 0x7fc55000bb50 con 0x7fc55000a8b0 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 >> v2:172.21.15.135:3300/0 conn(0x7fc5500dbce0 msgr2=0x7fc5500474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 --2- 172.21.15.135:0/626628462 >> v2:172.21.15.135:3300/0 conn(0x7fc5500dbce0 0x7fc5500474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 >> v1:172.21.15.135:6789/0 conn(0x7fc5500df180 legacy=0x7fc550006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55000a0f0 con 0x7fc55000a8b0 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fc54c0015a0 con 0x7fc55000a8b0 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 >> v1:172.21.15.139:6789/0 conn(0x7fc55000a8b0 legacy=0x7fc55000ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc548003b40 0x7fc548003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc548004800 0x7fc548004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5480051c0 0x7fc54800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc55000a420 con 0x7fc5480051c0 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc54800c090 con 0x7fc548003b40 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc54800c290 con 0x7fc548004800 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816d860640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816e061640 1 -- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 msgr2=0x7f81540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816e061640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816e061640 1 -- 172.21.15.135:0/2951801882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8154003fc0 msgr2=0x7f8154004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816e061640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8154003fc0 0x7f8154004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f816e061640 1 -- 172.21.15.135:0/2951801882 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8154003520 con 0x7f8154004d40 2024-04-26T22:00:13.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7fc55cc07640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc548003b40 0x7fc548003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f42effff640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42e4004800 0x7f42e4004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f42f80c4cb0 tx=0x7f42e4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6916e20116458b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7fc55d408640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5480051c0 0x7fc54800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1791546597 0 0) 0x7f29b8003520 con 0x7f29d00c40c0 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 >> v2:172.21.15.135:3300/0 conn(0x7f29d006f4b0 msgr2=0x7f29d006e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.650+0000 7f29cdffb640 1 --2- 172.21.15.135:0/2410233901 >> v2:172.21.15.135:3300/0 conn(0x7f29d006f4b0 0x7f29d006e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 >> v1:172.21.15.135:6789/0 conn(0x7f29d006e410 legacy=0x7f29d00c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00c2ed0 con 0x7f29d00c40c0 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f29c4001670 con 0x7f29d00c40c0 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 >> v1:172.21.15.139:6789/0 conn(0x7f29d00c40c0 legacy=0x7f29d015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b80039b0 0x7f29b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29b80053c0 0x7f29b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00c3520 con 0x7f29b80053c0 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29b800c2a0 con 0x7f29b80039b0 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29b800c4a0 con 0x7f29b8004800 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f8f9b7fe640 1 -- 172.21.15.135:0/1544188771 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94037070 con 0x7f8f9c0c3440 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f8f9b7fe640 1 -- 172.21.15.135:0/1544188771 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f94032440 con 0x7f8f9c0c3440 2024-04-26T22:00:13.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f8f9b7fe640 1 -- 172.21.15.135:0/1544188771 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94032810 con 0x7f8f9c0c3440 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 79482091 0 0) 0x7f083c003520 con 0x7f0848007ad0 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 >> v2:172.21.15.135:3300/0 conn(0x7f0848049f30 msgr2=0x7f08480315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 --2- 172.21.15.135:0/1568606594 >> v2:172.21.15.135:3300/0 conn(0x7f0848049f30 0x7f08480315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 >> v1:172.21.15.135:6789/0 conn(0x7f084804a440 legacy=0x7f0848006230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480d6e90 con 0x7f0848007ad0 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f0840001670 con 0x7f0848007ad0 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 >> v1:172.21.15.139:6789/0 conn(0x7f0848007ad0 legacy=0x7f084800a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f083c0039b0 0x7f083c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c004800 0x7f083c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f083c0051c0 0x7f083c0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0848006940 con 0x7f083c004800 2024-04-26T22:00:13.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f083c014900 con 0x7f083c0051c0 2024-04-26T22:00:13.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f083c014b00 con 0x7f083c0039b0 2024-04-26T22:00:13.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f816d860640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81600d7ae0 0x7f81540039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7ffaa6652640 1 -- 172.21.15.135:0/1775710217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa00c33f0 msgr2=0x7ffaa00c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7ffaa6652640 1 --2- 172.21.15.135:0/1775710217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa00c33f0 0x7ffaa00c6db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa900097a0 tx=0x7ffa9002a100 comp rx=0 tx=0).stop 2024-04-26T22:00:13.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7fc557fff640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc548004800 0x7fc548004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f29ce7fc640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0856094640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f083c0039b0 0x7f083c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0855893640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c004800 0x7f083c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f0856895640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f083c0051c0 0x7f083c0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f42f77fe640 1 -- 172.21.15.135:0/1289534736 reap_dead start 2024-04-26T22:00:13.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0c3440 msgr2=0x7f8f9c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/1544188771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0c3440 0x7f8f9c15e640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f940097a0 tx=0x7f8f9402a180 comp rx=0 tx=0).stop 2024-04-26T22:00:13.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f9885094640 1 --2- 172.21.15.135:0/2418728357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98800abe80 0x7f9880151dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9880076870 tx=0x7f987002e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f80d725698e170 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0037070 con 0x7f42e4004800 2024-04-26T22:00:13.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 shutdown_connections 2024-04-26T22:00:13.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/1544188771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0c3440 0x7f8f9c15e640 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/1544188771 >> v2:172.21.15.139:3300/0 conn(0x7f8f9c14fe70 0x7f8f9c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e002ab50 con 0x7f42e4004800 2024-04-26T22:00:13.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 >> 172.21.15.135:0/1544188771 conn(0x7f8f9c0c5ab0 msgr2=0x7f8f9c06d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29cf7fe640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29b80053c0 0x7f29b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29ceffd640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b80039b0 0x7f29b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f42f67fc640 1 -- 172.21.15.135:0/1289534736 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e000a590 con 0x7f42e4004800 2024-04-26T22:00:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7ffaa6652640 1 -- 172.21.15.135:0/1775710217 shutdown_connections 2024-04-26T22:00:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7ffaa6652640 1 --2- 172.21.15.135:0/1775710217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa00c33f0 0x7ffaa00c6db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.651+0000 7ffaa6652640 1 -- 172.21.15.135:0/1775710217 >> 172.21.15.135:0/1775710217 conn(0x7ffaa00c89f0 msgr2=0x7ffaa00c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7ffaa6652640 1 -- 172.21.15.135:0/1775710217 shutdown_connections 2024-04-26T22:00:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7ffaa6652640 1 -- 172.21.15.135:0/1775710217 wait complete. 2024-04-26T22:00:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f9884893640 1 -- 172.21.15.135:0/2418728357 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987003a070 con 0x7f98800abe80 2024-04-26T22:00:13.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29ce7fc640 1 -- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b80039b0 msgr2=0x7f29b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29ce7fc640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b80039b0 0x7f29b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 shutdown_connections 2024-04-26T22:00:13.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f8fa2f97640 1 -- 172.21.15.135:0/1544188771 wait complete. 2024-04-26T22:00:13.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f9884893640 1 -- 172.21.15.135:0/2418728357 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9870033710 con 0x7f98800abe80 2024-04-26T22:00:13.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29ce7fc640 1 -- 172.21.15.135:0/2410233901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29b80053c0 msgr2=0x7f29b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29ce7fc640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29b80053c0 0x7f29b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f9884893640 1 -- 172.21.15.135:0/2418728357 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987003a070 con 0x7f98800abe80 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29ce7fc640 1 -- 172.21.15.135:0/2410233901 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00c2ed0 con 0x7f29b8004800 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856094640 1 -- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f083c0051c0 msgr2=0x7f083c0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856094640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f083c0051c0 0x7f083c0141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856094640 1 -- 172.21.15.135:0/1568606594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c004800 msgr2=0x7f083c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856094640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c004800 0x7f083c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856094640 1 -- 172.21.15.135:0/1568606594 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480d6e90 con 0x7f083c0039b0 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856895640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f083c0051c0 0x7f083c0141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f0856895640 1 -- 172.21.15.135:0/1568606594 reap_dead start 2024-04-26T22:00:13.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7fc55d408640 1 -- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc548004800 msgr2=0x7fc548004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7f29cf7fe640 1 -- 172.21.15.135:0/2410233901 reap_dead start 2024-04-26T22:00:13.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7fc55d408640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc548004800 0x7fc548004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f816e061640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8154004d40 0x7f81540051c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8168051fc0 tx=0x7f8154005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c80b302b902429 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7fc55d408640 1 -- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc548003b40 msgr2=0x7fc548003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7fc55d408640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc548003b40 0x7fc548003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.652+0000 7fc55d408640 1 -- 172.21.15.135:0/626628462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55000bb50 con 0x7fc5480051c0 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7fc55cc07640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc548003b40 0x7fc548003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7fc55d408640 1 -- 172.21.15.135:0/626628462 reap_dead start 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7fc557fff640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc548004800 0x7fc548004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168050f10 con 0x7f8154004d40 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168076310 con 0x7f8154004d40 2024-04-26T22:00:13.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f816c85e640 1 -- 172.21.15.135:0/2951801882 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81680756f0 con 0x7f8154004d40 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7fc55d408640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5480051c0 0x7fc54800b9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc54c007eb0 tx=0x7fc548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1275008539ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 -- 172.21.15.135:0/2418728357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98800abe80 msgr2=0x7f9880151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 --2- 172.21.15.135:0/2418728357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98800abe80 0x7f9880151dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9880076870 tx=0x7f987002e570 comp rx=0 tx=0).stop 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 -- 172.21.15.135:0/2418728357 shutdown_connections 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 --2- 172.21.15.135:0/2418728357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98800abe80 0x7f9880151dd0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9880076870 tx=0x7f987002e570 comp rx=0 tx=0).stop 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 --2- 172.21.15.135:0/2418728357 >> v2:172.21.15.139:3300/0 conn(0x7f98800c5170 0x7f9880152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 --2- 172.21.15.135:0/2418728357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9880158270 0x7f98800c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f9887af9640 1 -- 172.21.15.135:0/2418728357 >> 172.21.15.135:0/2418728357 conn(0x7f98800caad0 msgr2=0x7f98800c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c036070 con 0x7fc5480051c0 2024-04-26T22:00:13.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54c031620 con 0x7fc5480051c0 2024-04-26T22:00:13.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7fc5577fe640 1 -- 172.21.15.135:0/626628462 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544009690 con 0x7fc5480051c0 2024-04-26T22:00:13.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.653+0000 7f0856094640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f083c0039b0 0x7f083c003db0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f084c00ba00 tx=0x7f083c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb48519a24ac7375 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c030070 con 0x7f083c0039b0 2024-04-26T22:00:13.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c02ad80 con 0x7f083c0039b0 2024-04-26T22:00:13.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f0855092640 1 -- 172.21.15.135:0/1568606594 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00b600 con 0x7f083c0039b0 2024-04-26T22:00:13.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f42fd8a9640 1 -- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42e4004800 msgr2=0x7f42e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f9887af9640 1 -- 172.21.15.135:0/2418728357 shutdown_connections 2024-04-26T22:00:13.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f9887af9640 1 -- 172.21.15.135:0/2418728357 wait complete. 2024-04-26T22:00:13.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42e4004800 0x7f42e4004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f42f80c4cb0 tx=0x7f42e4005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f42fd8a9640 1 -- 172.21.15.135:0/1289534736 shutdown_connections 2024-04-26T22:00:13.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/1289534736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42e4004800 0x7f42e4004c80 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f42f80c4cb0 tx=0x7f42e4005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f816fac4640 1 -- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8154004d40 msgr2=0x7f81540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f816fac4640 1 --2- 172.21.15.135:0/2951801882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8154004d40 0x7f81540051c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8168051fc0 tx=0x7f8154005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7ffaa6652640 1 Processor -- start 2024-04-26T22:00:13.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7ffaa6652640 1 -- start start 2024-04-26T22:00:13.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f8fa2f97640 1 Processor -- start 2024-04-26T22:00:13.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f816e061640 1 -- 172.21.15.135:0/2951801882 reap_dead start 2024-04-26T22:00:13.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f816fac4640 1 -- 172.21.15.135:0/2951801882 shutdown_connections 2024-04-26T22:00:13.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f42fd8a9640 1 -- 172.21.15.135:0/1289534736 >> 172.21.15.135:0/1289534736 conn(0x7f42f80c5ab0 msgr2=0x7f42f806d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f816fac4640 1 -- 172.21.15.135:0/2951801882 >> 172.21.15.135:0/2951801882 conn(0x7f816004c660 msgr2=0x7f816004ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f8fa2f97640 1 -- start start 2024-04-26T22:00:13.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 -- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f083c0039b0 msgr2=0x7f083c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f42fd8a9640 1 -- 172.21.15.135:0/1289534736 shutdown_connections 2024-04-26T22:00:13.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7fc55dc09640 1 -- 172.21.15.135:0/626628462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5480051c0 msgr2=0x7fc54800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7fc55dc09640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5480051c0 0x7fc54800b9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc54c007eb0 tx=0x7fc548005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f083c0039b0 0x7f083c003db0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f084c00ba00 tx=0x7f083c005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f29ce7fc640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29b8004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f29c000ba30 tx=0x7f29b8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7124a14a7349ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0030070 con 0x7f29b8004800 2024-04-26T22:00:13.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c002ade0 con 0x7f29b8004800 2024-04-26T22:00:13.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.654+0000 7f29cdffb640 1 -- 172.21.15.135:0/2410233901 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c000b600 con 0x7f29b8004800 2024-04-26T22:00:13.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7fc55dc09640 1 -- 172.21.15.135:0/626628462 shutdown_connections 2024-04-26T22:00:13.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7fc55dc09640 1 --2- 172.21.15.135:0/626628462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5480051c0 0x7fc54800b9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7fc55dc09640 1 -- 172.21.15.135:0/626628462 >> 172.21.15.135:0/626628462 conn(0x7fc550048250 msgr2=0x7fc550048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f816fac4640 1 -- 172.21.15.135:0/2951801882 shutdown_connections 2024-04-26T22:00:13.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 -- 172.21.15.135:0/1568606594 shutdown_connections 2024-04-26T22:00:13.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 --2- 172.21.15.135:0/1568606594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f083c0039b0 0x7f083c003db0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 -- 172.21.15.135:0/1568606594 >> 172.21.15.135:0/1568606594 conn(0x7f084802db60 msgr2=0x7f084802df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 -- 172.21.15.135:0/1568606594 shutdown_connections 2024-04-26T22:00:13.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f08582f8640 1 -- 172.21.15.135:0/1568606594 wait complete. 2024-04-26T22:00:13.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7fc55dc09640 1 -- 172.21.15.135:0/626628462 shutdown_connections 2024-04-26T22:00:13.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f816fac4640 1 -- 172.21.15.135:0/2951801882 wait complete. 2024-04-26T22:00:13.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7f42fd8a9640 1 -- 172.21.15.135:0/1289534736 wait complete. 2024-04-26T22:00:13.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa6652640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffaa00c33f0 0x7ffaa0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa6652640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffaa00c4e30 0x7ffaa014d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa6652640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 0x7ffaa014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa6652640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa006fd90 con 0x7ffaa00c33f0 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa6652640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa006f1f0 con 0x7ffaa0158610 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa6652640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa015bde0 con 0x7ffaa00c4e30 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa4bef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 0x7ffaa014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa4bef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 0x7ffaa014dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33460/0 (socket says 172.21.15.135:33460) 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.655+0000 7ffaa4bef640 1 -- 172.21.15.135:0/707893995 learned_addr learned my addr 172.21.15.135:0/707893995 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7fc55dc09640 1 -- 172.21.15.135:0/626628462 wait complete. 2024-04-26T22:00:13.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f8fa2f97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 0x7f8f9c147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f8fa2f97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 0x7f8f9c0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f29cffff640 1 -- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 msgr2=0x7f29b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f29cffff640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29b8004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f29c000ba30 tx=0x7f29b8005830 comp rx=0 tx=0).stop 2024-04-26T22:00:13.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f8fa0d33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 0x7f8f9c147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f9887af9640 1 Processor -- start 2024-04-26T22:00:13.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f29cffff640 1 -- 172.21.15.135:0/2410233901 shutdown_connections 2024-04-26T22:00:13.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f29cffff640 1 --2- 172.21.15.135:0/2410233901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29b8004c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f29cffff640 1 -- 172.21.15.135:0/2410233901 >> 172.21.15.135:0/2410233901 conn(0x7f29d00c80c0 msgr2=0x7f29d00c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:13.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f9887af9640 1 -- start start 2024-04-26T22:00:13.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f8fa0d33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 0x7f8f9c147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:33992/0 (socket says 172.21.15.135:33992) 2024-04-26T22:00:13.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7ffa9ffff640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffaa00c33f0 0x7ffaa0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.656+0000 7f42fd8a9640 1 Processor -- start 2024-04-26T22:00:13.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8f9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 0x7f8f9c0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8f9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 0x7f8f9c0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40508/0 (socket says 172.21.15.135:40508) 2024-04-26T22:00:13.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffaa4bef640 1 -- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffaa00c4e30 msgr2=0x7ffaa014d840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:00:13.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffaa4bef640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffaa00c4e30 0x7ffaa014d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffaa4bef640 1 -- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffaa00c33f0 msgr2=0x7ffaa0155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f08582f8640 1 Processor -- start 2024-04-26T22:00:13.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8fa0d33640 1 -- 172.21.15.135:0/3285110627 learned_addr learned my addr 172.21.15.135:0/3285110627 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c14fe70 0x7f8f9c1eab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffaa4bef640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffaa00c33f0 0x7ffaa0155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffaa4bef640 1 -- 172.21.15.135:0/707893995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa0151950 con 0x7ffaa0158610 2024-04-26T22:00:13.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffa9ffff640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffaa00c33f0 0x7ffaa0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c150250 con 0x7f8f9c0c3440 2024-04-26T22:00:13.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c153710 con 0x7f8f9c14fe70 2024-04-26T22:00:13.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8fa1534640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c14fe70 0x7f8f9c1eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f29cffff640 1 -- 172.21.15.135:0/2410233901 shutdown_connections 2024-04-26T22:00:13.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c1503d0 con 0x7f8f9c0694b0 2024-04-26T22:00:13.702 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f29cffff640 1 -- 172.21.15.135:0/2410233901 wait complete. 2024-04-26T22:00:13.702 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f08582f8640 1 -- start start 2024-04-26T22:00:13.702 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f9887af9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 0x7f98801e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f08582f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 0x7f0848045bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f08582f8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08480425f0 0x7f084802f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f08582f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08480d3450 0x7f08480d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f08582f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f084800c1b0 con 0x7f083c0039b0 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f08582f8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08480d7450 con 0x7f08480d3450 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f08582f8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f084800c330 con 0x7f08480425f0 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7f42fd8a9640 1 -- start start 2024-04-26T22:00:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42fd8a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f81f0760 0x7f42f8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42fd8a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42f8154260 0x7f42f80c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42fd8a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81f0b40 0x7f42f81536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42fd8a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80c3dc0 con 0x7f42f8154260 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42fd8a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80c3c40 con 0x7f42f81f0b40 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42fd8a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42f80aa650 con 0x7f42f81f0760 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7fc55dc09640 1 Processor -- start 2024-04-26T22:00:13.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f816fac4640 1 Processor -- start 2024-04-26T22:00:13.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7fc55dc09640 1 -- start start 2024-04-26T22:00:13.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f0856094640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 0x7f0848045bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f0856094640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 0x7f0848045bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40520/0 (socket says 172.21.15.135:40520) 2024-04-26T22:00:13.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f0856094640 1 -- 172.21.15.135:0/741768843 learned_addr learned my addr 172.21.15.135:0/741768843 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f0855893640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08480425f0 0x7f084802f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f0856895640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08480d3450 0x7f08480d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42f6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f81f0760 0x7f42f8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42f6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f81f0760 0x7f42f8072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:33998/0 (socket says 172.21.15.135:33998) 2024-04-26T22:00:13.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42f6ffd640 1 -- 172.21.15.135:0/2256674234 learned_addr learned my addr 172.21.15.135:0/2256674234 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f42effff640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42f8154260 0x7f42f80c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f9887af9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9880158270 0x7f98801ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f9887af9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f988014de50 0x7f988014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 0x7f98801e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 0x7f98801e88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34014/0 (socket says 172.21.15.135:34014) 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 -- 172.21.15.135:0/2108909923 learned_addr learned my addr 172.21.15.135:0/2108909923 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98800c5c40 con 0x7f9880158270 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885094640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9880158270 0x7f98801ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9886096640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f988014de50 0x7f988014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98800c5ac0 con 0x7f988014de50 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f98800c5dc0 con 0x7f98800c5170 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f42effff640 1 -- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f81f0760 msgr2=0x7f42f8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f42f77fe640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81f0b40 0x7f42f81536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f42effff640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f81f0760 0x7f42f8072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f42effff640 1 -- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81f0b40 msgr2=0x7f42f81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f42effff640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81f0b40 0x7f42f81536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0856094640 1 -- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08480425f0 msgr2=0x7f084802f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0856094640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08480425f0 0x7f084802f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0856094640 1 -- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08480d3450 msgr2=0x7f08480d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0856094640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08480d3450 0x7f08480d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0856094640 1 -- 172.21.15.135:0/741768843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480da790 con 0x7f083c0039b0 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0856094640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 0x7f0848045bc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f084c00bb20 tx=0x7f084c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f42effff640 1 -- 172.21.15.135:0/2256674234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8071ad0 con 0x7f42f8154260 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f8f9bfff640 1 -- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 msgr2=0x7f8f9c147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f8f9bfff640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 0x7f8f9c147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f8f9bfff640 1 -- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c14fe70 msgr2=0x7f8f9c1eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f8f9bfff640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c14fe70 0x7f8f9c1eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f8f9bfff640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c156650 con 0x7f8f9c0c3440 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f8fa0d33640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 0x7f8f9c147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f8fa1534640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c14fe70 0x7f8f9c1eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:13.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f29cffff640 1 Processor -- start 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7f816fac4640 1 -- start start 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816fac4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8154004d40 0x7f81600d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816fac4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 0x7f8160030870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816fac4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81600475c0 0x7f816016cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816fac4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f816004e4b0 con 0x7f81601704a0 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816fac4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81600db270 con 0x7f81600475c0 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816fac4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f816000fdc0 con 0x7f8154004d40 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f42effff640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42f8154260 0x7f42f80c7250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42e400c600 tx=0x7f42e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f29cffff640 1 -- start start 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 -- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f988014de50 msgr2=0x7f988014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f988014de50 0x7f988014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 -- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9880158270 msgr2=0x7f98801ddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9880158270 0x7f98801ddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f9885895640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988014fcd0 con 0x7f98800c5170 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f816d05f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 0x7f8160030870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d860640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8154004d40 0x7f81600d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 0x7f8160030870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40540/0 (socket says 172.21.15.135:40540) 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 -- 172.21.15.135:0/4285620259 learned_addr learned my addr 172.21.15.135:0/4285620259 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.657+0000 7ffaa4bef640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 0x7ffaa014dd80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c7d0 tx=0x7ffa9800cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98003b70 con 0x7ffaa0158610 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa0071dd0 con 0x7ffaa0158610 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.658+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa0156280 con 0x7ffaa0158610 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa98003d30 con 0x7ffaa0158610 2024-04-26T22:00:13.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800aa90 con 0x7ffaa0158610 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f9885895640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 0x7f98801e88c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e870 tx=0x7f987c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c043020 con 0x7f083c0039b0 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.659+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c035e90 con 0x7f083c0039b0 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00b600 con 0x7f083c0039b0 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0040020 con 0x7f42f8154260 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e0038040 con 0x7f42f8154260 2024-04-26T22:00:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0037070 con 0x7f42f8154260 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7fc55dc09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5480051c0 0x7fc5500d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7fc55dc09640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5500da1e0 0x7fc5501693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7fc55dc09640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc55016b830 0x7fc5501698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7fc55dc09640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500dc0c0 con 0x7fc5500da1e0 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7fc55dc09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc550047d80 con 0x7fc5480051c0 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7fc55dc09640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc5500df5f0 con 0x7fc55016b830 2024-04-26T22:00:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa9800ac30 con 0x7ffaa0158610 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f98800c5170 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f29cffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29d0072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816e061640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81600475c0 0x7f816016cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 -- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8154004d40 msgr2=0x7f81600d8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8154004d40 0x7f81600d8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 -- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81600475c0 msgr2=0x7f816016cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81600475c0 0x7f816016cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f816d05f640 1 -- 172.21.15.135:0/4285620259 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81600d9e70 con 0x7f81601704a0 2024-04-26T22:00:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f816d05f640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 0x7f8160030870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f815800c440 tx=0x7f815800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f8f9bfff640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 0x7f8f9c0c7fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c830 tx=0x7f8f8400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c00c940 con 0x7f98800c5170 2024-04-26T22:00:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988014f430 con 0x7f98800c5170 2024-04-26T22:00:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f988014f9f0 con 0x7f98800c5170 2024-04-26T22:00:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f29cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29d00c7110 0x7f29d01eba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f29cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 0x7f29d0143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f29cffff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0152a70 con 0x7f29d00c7110 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f29cffff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0152e30 con 0x7f29d01ebf60 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f29cffff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29d0152bf0 con 0x7f29b8004800 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7ffa867fc640 1 -- 172.21.15.135:0/707893995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa80005290 con 0x7ffaa0158610 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7ffa9cff9640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffa8c06c8f0 0x7ffa8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08480daa70 con 0x7f083c0039b0 2024-04-26T22:00:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.660+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0848041330 con 0x7f083c0039b0 2024-04-26T22:00:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 0x7f29d0143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 0x7f29d0143450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33530/0 (socket says 172.21.15.135:33530) 2024-04-26T22:00:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 -- 172.21.15.135:0/1905658288 learned_addr learned my addr 172.21.15.135:0/1905658288 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84017070 con 0x7f8f9c0c3440 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f84005ab0 con 0x7f8f9c0c3440 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7fc55d408640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc55016b830 0x7fc5501698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7fc55cc07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5480051c0 0x7fc5500d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7fc55d408640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc55016b830 0x7fc5501698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34032/0 (socket says 172.21.15.135:34032) 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7fc55d408640 1 -- 172.21.15.135:0/2724385004 learned_addr learned my addr 172.21.15.135:0/2724385004 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:13.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400ce20 con 0x7f8f9c0c3440 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7fc557fff640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5500da1e0 0x7fc5501693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc55cc07640 1 -- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc55016b830 msgr2=0x7fc5501698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc55cc07640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc55016b830 0x7fc5501698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc55cc07640 1 -- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5500da1e0 msgr2=0x7fc5501693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc55cc07640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5500da1e0 0x7fc5501693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc55cc07640 1 -- 172.21.15.135:0/2724385004 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc550167dd0 con 0x7fc5480051c0 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc557fff640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5500da1e0 0x7fc5501693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f8071db0 con 0x7f42f8154260 2024-04-26T22:00:13.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.661+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f80c6a50 con 0x7f42f8154260 2024-04-26T22:00:13.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29ceffd640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29d0072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0848005f70 con 0x7f083c0039b0 2024-04-26T22:00:13.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc005290 con 0x7f42f8154260 2024-04-26T22:00:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c156910 con 0x7f8f9c0c3440 2024-04-26T22:00:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c156ed0 con 0x7f8f9c0c3440 2024-04-26T22:00:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c00cd10 con 0x7f98800c5170 2024-04-26T22:00:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7ffa9ffff640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffa8c06c8f0 0x7ffa8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7fc55cc07640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5480051c0 0x7fc5500d9ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc544011960 tx=0x7fc544011d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815801d3f0 con 0x7f81601704a0 2024-04-26T22:00:13.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.663+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815801d590 con 0x7f81601704a0 2024-04-26T22:00:13.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.663+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815801d3f0 con 0x7f81601704a0 2024-04-26T22:00:13.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.663+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81600da150 con 0x7f81601704a0 2024-04-26T22:00:13.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.663+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81600d8590 con 0x7f81601704a0 2024-04-26T22:00:13.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ffa9808bd40 con 0x7ffaa0158610 2024-04-26T22:00:13.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.663+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98800506c0 con 0x7f98800c5170 2024-04-26T22:00:13.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f987c01a070 con 0x7f98800c5170 2024-04-26T22:00:13.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9c0506c0 con 0x7f8f9c0c3440 2024-04-26T22:00:13.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7ffa9ffff640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffa8c06c8f0 0x7ffa8c06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa940011e0 tx=0x7ffa94007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.663+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084c046020 con 0x7f083c0039b0 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f0846ffd640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f083406c9c0 0x7f083406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f084c0ac8e0 con 0x7f083c0039b0 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 -- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 msgr2=0x7f29d0072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29d0072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 -- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29d00c7110 msgr2=0x7f29d01eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29d00c7110 0x7f29d01eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 -- 172.21.15.135:0/1905658288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00731f0 con 0x7f29d01ebf60 2024-04-26T22:00:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.662+0000 7f29cf7fe640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 0x7f29d0143450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f29d00acfa0 tx=0x7f29c400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f0855893640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f083406c9c0 0x7f083406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f98767fc640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f986406c9f0 0x7f986406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.664+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42e0043020 con 0x7f42f8154260 2024-04-26T22:00:13.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f987c08bbc0 con 0x7f98800c5170 2024-04-26T22:00:13.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81600ceb00 con 0x7f81601704a0 2024-04-26T22:00:13.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f9885094640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f986406c9f0 0x7f986406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f0855893640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f083406c9c0 0x7f083406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0850075230 tx=0x7f085006bd40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f42f4ff9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f42d806c9c0 0x7f42d806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f42e00ac0d0 con 0x7f42f8154260 2024-04-26T22:00:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f42f6ffd640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f42d806c9c0 0x7f42d806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f9885094640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f986406c9f0 0x7f986406eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9870037000 tx=0x7f9870008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.665+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f8401a070 con 0x7f8f9c0c3440 2024-04-26T22:00:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.666+0000 7f42f6ffd640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f42d806c9c0 0x7f42d806ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e80018a0 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.666+0000 7f8f997fa640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f7c06c940 0x7f8f7c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.666+0000 7f8fa0d33640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f7c06c940 0x7f8f7c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.666+0000 7f8fa0d33640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f7c06c940 0x7f8f7c06ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c007520 tx=0x7f8f8c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544019070 con 0x7fc5480051c0 2024-04-26T22:00:13.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c400cb40 con 0x7f29d01ebf60 2024-04-26T22:00:13.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544004240 con 0x7fc5480051c0 2024-04-26T22:00:13.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54400f7c0 con 0x7fc5480051c0 2024-04-26T22:00:13.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c4016070 con 0x7f29d01ebf60 2024-04-26T22:00:13.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7fc55dc09640 1 -- 172.21.15.135:0/2724385004 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5501786d0 con 0x7fc5480051c0 2024-04-26T22:00:13.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c400cb40 con 0x7f29d01ebf60 2024-04-26T22:00:13.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d00718d0 con 0x7f29d01ebf60 2024-04-26T22:00:13.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d0071e90 con 0x7f29d01ebf60 2024-04-26T22:00:13.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa98059e30 con 0x7ffaa0158610 2024-04-26T22:00:13.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.667+0000 7fc55dc09640 1 -- 172.21.15.135:0/2724385004 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc550044710 con 0x7fc5480051c0 2024-04-26T22:00:13.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.668+0000 7fc55dc09640 1 -- 172.21.15.135:0/2724385004 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc550005390 con 0x7fc5480051c0 2024-04-26T22:00:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.668+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d014ecb0 con 0x7f29d01ebf60 2024-04-26T22:00:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.669+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29c400cce0 con 0x7f29d01ebf60 2024-04-26T22:00:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.669+0000 7f29b7fff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29bc06caf0 0x7f29bc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.669+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f29c408bf70 con 0x7f29d01ebf60 2024-04-26T22:00:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.669+0000 7f29ceffd640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29bc06caf0 0x7f29bc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f987c0596e0 con 0x7f98800c5170 2024-04-26T22:00:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42e004a030 con 0x7f42f8154260 2024-04-26T22:00:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.669+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc54401c070 con 0x7fc5480051c0 2024-04-26T22:00:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7fc5557fa640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc53c06c8f0 0x7fc53c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f8f84057050 con 0x7f8f9c0c3440 2024-04-26T22:00:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7f29ceffd640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29bc06caf0 0x7f29bc06efb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29d0072c40 tx=0x7f29c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc54408c3e0 con 0x7fc5480051c0 2024-04-26T22:00:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.670+0000 7fc557fff640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc53c06c8f0 0x7fc53c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.671+0000 7fc557fff640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc53c06c8f0 0x7fc53c06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc558065e50 tx=0x7fc558073040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.673+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc54405a000 con 0x7fc5480051c0 2024-04-26T22:00:13.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.675+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f084c04d030 con 0x7f083c0039b0 2024-04-26T22:00:13.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.677+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8158007020 con 0x7f81601704a0 2024-04-26T22:00:13.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.677+0000 7f815e7fc640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f814c06cd20 0x7f814c06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:13.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.678+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29d014ecb0 con 0x7f29d01ebf60 2024-04-26T22:00:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.678+0000 7f816d860640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f814c06cd20 0x7f814c06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.678+0000 7f816d860640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f814c06cd20 0x7f814c06f1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8164006fd0 tx=0x7f8164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.681+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f8158015c90 con 0x7f81601704a0 2024-04-26T22:00:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.681+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f84021020 con 0x7f8f9c0c3440 2024-04-26T22:00:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:13.689+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81600ceb00 con 0x7f81601704a0 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.088+0000 7ffa867fc640 1 -- 172.21.15.135:0/707893995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffa80005580 con 0x7ffaa0158610 2024-04-26T22:00:14.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.089+0000 7ffa9cff9640 1 -- 172.21.15.135:0/707893995 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa98059730 con 0x7ffaa0158610 2024-04-26T22:00:14.089 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.092+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffa8c06c8f0 msgr2=0x7ffa8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.092+0000 7ffaa6652640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffa8c06c8f0 0x7ffa8c06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa940011e0 tx=0x7ffa94007d30 comp rx=0 tx=0).stop 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.092+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 msgr2=0x7ffaa014dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.092+0000 7ffaa6652640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 0x7ffaa014dd80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c7d0 tx=0x7ffa9800cc90 comp rx=0 tx=0).stop 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 shutdown_connections 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffa8c06c8f0 0x7ffa8c06edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa0158610 0x7ffaa014dd80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffaa00c4e30 0x7ffaa014d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 --2- 172.21.15.135:0/707893995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffaa00c33f0 0x7ffaa0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 >> 172.21.15.135:0/707893995 conn(0x7ffaa00c89f0 msgr2=0x7ffaa006f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 shutdown_connections 2024-04-26T22:00:14.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.093+0000 7ffaa6652640 1 -- 172.21.15.135:0/707893995 wait complete. 2024-04-26T22:00:14.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.097+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f98800ab7f0 con 0x7f98800c5170 2024-04-26T22:00:14.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.098+0000 7f98767fc640 1 -- 172.21.15.135:0/2108909923 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f987c05d9e0 con 0x7f98800c5170 2024-04-26T22:00:14.098 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:14.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.100+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f29d00506c0 con 0x7f29d01ebf60 2024-04-26T22:00:14.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.100+0000 7f29b7fff640 1 -- 172.21.15.135:0/1905658288 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29d014ecb0 con 0x7f29d01ebf60 2024-04-26T22:00:14.100 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f986406c9f0 msgr2=0x7f986406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f986406c9f0 0x7f986406eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9870037000 tx=0x7f9870008030 comp rx=0 tx=0).stop 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 msgr2=0x7f98801e88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 0x7f98801e88c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e870 tx=0x7f987c00ed30 comp rx=0 tx=0).stop 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 shutdown_connections 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f986406c9f0 0x7f986406eeb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f988014de50 0x7f988014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9880158270 0x7f98801ddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 --2- 172.21.15.135:0/2108909923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98800c5170 0x7f98801e88c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.101+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 >> 172.21.15.135:0/2108909923 conn(0x7f98800caad0 msgr2=0x7f9880151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.102+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 shutdown_connections 2024-04-26T22:00:14.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.102+0000 7f9887af9640 1 -- 172.21.15.135:0/2108909923 wait complete. 2024-04-26T22:00:14.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.103+0000 7fc55dc09640 1 -- 172.21.15.135:0/2724385004 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc55000b590 con 0x7fc5480051c0 2024-04-26T22:00:14.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.103+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f084800bf90 con 0x7f083c0039b0 2024-04-26T22:00:14.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.104+0000 7f0846ffd640 1 -- 172.21.15.135:0/741768843 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f084c07a9c0 con 0x7f083c0039b0 2024-04-26T22:00:14.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.104+0000 7fc5557fa640 1 -- 172.21.15.135:0/2724385004 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc54405e300 con 0x7fc5480051c0 2024-04-26T22:00:14.105 INFO:teuthology.orchestra.run.smithi135.stdout:55834574854 2024-04-26T22:00:14.105 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:14.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.104+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f42dc005850 con 0x7f42f8154260 2024-04-26T22:00:14.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f42f4ff9640 1 -- 172.21.15.135:0/2256674234 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42e0079bf0 con 0x7f42f8154260 2024-04-26T22:00:14.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29bc06caf0 msgr2=0x7f29bc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f29cffff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29bc06caf0 0x7f29bc06efb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29d0072c40 tx=0x7f29c8009040 comp rx=0 tx=0).stop 2024-04-26T22:00:14.106 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 msgr2=0x7f29d0143450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f29cffff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 0x7f29d0143450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f29d00acfa0 tx=0x7f29c400ecc0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 shutdown_connections 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.105+0000 7f29cffff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29bc06caf0 0x7f29bc06efb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.106+0000 7f29cffff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29d01ebf60 0x7f29d0143450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.106+0000 7f29cffff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29d00c7110 0x7f29d01eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.106+0000 7f29cffff640 1 --2- 172.21.15.135:0/1905658288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29b8004800 0x7f29d0072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.106+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 >> 172.21.15.135:0/1905658288 conn(0x7f29d00c80c0 msgr2=0x7f29d006e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.106+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 shutdown_connections 2024-04-26T22:00:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.106+0000 7f29cffff640 1 -- 172.21.15.135:0/1905658288 wait complete. 2024-04-26T22:00:14.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.107+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8f9c0c3220 con 0x7f8f9c0c3440 2024-04-26T22:00:14.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f8f997fa640 1 -- 172.21.15.135:0/3285110627 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f840596b0 con 0x7f8f9c0c3440 2024-04-26T22:00:14.109 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:14.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7fc542ffd640 1 -- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc53c06c8f0 msgr2=0x7fc53c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7fc542ffd640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc53c06c8f0 0x7fc53c06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc558065e50 tx=0x7fc558073040 comp rx=0 tx=0).stop 2024-04-26T22:00:14.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7fc542ffd640 1 -- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5480051c0 msgr2=0x7fc5500d9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7fc542ffd640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5480051c0 0x7fc5500d9ca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc544011960 tx=0x7fc544011d60 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f083406c9c0 msgr2=0x7f083406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f083406c9c0 0x7f083406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0850075230 tx=0x7f085006bd40 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 msgr2=0x7f0848045bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 0x7f0848045bc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f084c00bb20 tx=0x7f084c004730 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 shutdown_connections 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f083406c9c0 0x7f083406ee80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08480d3450 0x7f08480d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08480425f0 0x7f084802f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 --2- 172.21.15.135:0/741768843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f083c0039b0 0x7f0848045bc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.108+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 >> 172.21.15.135:0/741768843 conn(0x7f084802db60 msgr2=0x7f08480d5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 shutdown_connections 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7f08582f8640 1 -- 172.21.15.135:0/741768843 wait complete. 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7fc542ffd640 1 -- 172.21.15.135:0/2724385004 shutdown_connections 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc53c06c8f0 0x7fc53c06edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc55016b830 0x7fc5501698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5500da1e0 0x7fc5501693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 --2- 172.21.15.135:0/2724385004 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5480051c0 0x7fc5500d9ca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 -- 172.21.15.135:0/2724385004 >> 172.21.15.135:0/2724385004 conn(0x7fc550048250 msgr2=0x7fc5500de730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 -- 172.21.15.135:0/2724385004 shutdown_connections 2024-04-26T22:00:14.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7fc542ffd640 1 -- 172.21.15.135:0/2724385004 wait complete. 2024-04-26T22:00:14.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f42d806c9c0 msgr2=0x7f42d806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f42d806c9c0 0x7f42d806ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e80018a0 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2024-04-26T22:00:14.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42f8154260 msgr2=0x7f42f80c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.109+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42f8154260 0x7f42f80c7250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42e400c600 tx=0x7f42e0004730 comp rx=0 tx=0).stop 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 shutdown_connections 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f42d806c9c0 0x7f42d806ee80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f81f0b40 0x7f42f81536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42f8154260 0x7f42f80c7250 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 --2- 172.21.15.135:0/2256674234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f81f0760 0x7f42f8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 >> 172.21.15.135:0/2256674234 conn(0x7f42f80c5ab0 msgr2=0x7f42f815acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 shutdown_connections 2024-04-26T22:00:14.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.110+0000 7f42fd8a9640 1 -- 172.21.15.135:0/2256674234 wait complete. 2024-04-26T22:00:14.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.111+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f7c06c940 msgr2=0x7f8f7c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.111+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f7c06c940 0x7f8f7c06ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c007520 tx=0x7f8f8c00b040 comp rx=0 tx=0).stop 2024-04-26T22:00:14.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.111+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 msgr2=0x7f8f9c0c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.116 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-04-26T22:00:14.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 0x7f8f9c0c7fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c830 tx=0x7f8f8400ccf0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 shutdown_connections 2024-04-26T22:00:14.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f7c06c940 0x7f8f7c06ee00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f9c14fe70 0x7f8f9c1eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f9c0c3440 0x7f8f9c0c7fb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 --2- 172.21.15.135:0/3285110627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f9c0694b0 0x7f8f9c147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.112+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 >> 172.21.15.135:0/3285110627 conn(0x7f8f9c0c5ab0 msgr2=0x7f8f9c14f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.113+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 shutdown_connections 2024-04-26T22:00:14.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.113+0000 7f8fa2f97640 1 -- 172.21.15.135:0/3285110627 wait complete. 2024-04-26T22:00:14.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.119+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f816000fba0 con 0x7f81601704a0 2024-04-26T22:00:14.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.120+0000 7f815e7fc640 1 -- 172.21.15.135:0/4285620259 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f81600ceb00 con 0x7f81601704a0 2024-04-26T22:00:14.120 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:14.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.124+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f814c06cd20 msgr2=0x7f814c06f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.124+0000 7f816fac4640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f814c06cd20 0x7f814c06f1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8164006fd0 tx=0x7f8164008040 comp rx=0 tx=0).stop 2024-04-26T22:00:14.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.124+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 msgr2=0x7f8160030870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.124+0000 7f816fac4640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 0x7f8160030870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f815800c440 tx=0x7f815800c900 comp rx=0 tx=0).stop 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.125+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 shutdown_connections 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.125+0000 7f816fac4640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f814c06cd20 0x7f814c06f1e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.125+0000 7f816fac4640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81600475c0 0x7f816016cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.125+0000 7f816fac4640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81601704a0 0x7f8160030870 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f815800c440 tx=0x7f815800c900 comp rx=0 tx=0).stop 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.125+0000 7f816fac4640 1 --2- 172.21.15.135:0/4285620259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8154004d40 0x7f81600d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:14.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.125+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 >> 172.21.15.135:0/4285620259 conn(0x7f816004c660 msgr2=0x7f81600d69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:14.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-04-26T22:00:14.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.126+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 shutdown_connections 2024-04-26T22:00:14.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:14.126+0000 7f816fac4640 1 -- 172.21.15.135:0/4285620259 wait complete. 2024-04-26T22:00:14.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-26T22:00:14.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-04-26T22:00:14.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-04-26T22:00:14.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-04-26T22:00:14.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-04-26T22:00:14.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-04-26T22:00:15.117 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T22:00:15.127 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T22:00:15.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T22:00:15.130 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T22:00:15.133 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T22:00:15.133 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T22:00:15.135 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T22:00:15.141 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.277+0000 7f8ded7f0640 1 Processor -- start 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.277+0000 7f8ded7f0640 1 -- start start 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.278+0000 7f8ded7f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8de004c2a0 0x7f8de000bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.278+0000 7f8ded7f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8de0049050 0x7f8de0008790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.278+0000 7f8ded7f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de004c680 con 0x7f8de000c020 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.278+0000 7f8ded7f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8de00d4a70 con 0x7f8de0049050 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.278+0000 7f8ded7f0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8de0047b40 con 0x7f8de004c2a0 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8de0049050 0x7f8de0008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8de0049050 0x7f8de0008790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40574/0 (socket says 172.21.15.135:40574) 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 -- 172.21.15.135:0/845498416 learned_addr learned my addr 172.21.15.135:0/845498416 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 -- 172.21.15.135:0/845498416 >> v2:172.21.15.139:3300/0 conn(0x7f8de004c2a0 msgr2=0x7f8de000bae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:15.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 --2- 172.21.15.135:0/845498416 >> v2:172.21.15.139:3300/0 conn(0x7f8de004c2a0 0x7f8de000bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 -- 172.21.15.135:0/845498416 >> v1:172.21.15.139:6789/0 conn(0x7f8de000c020 legacy=0x7f8de0008080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.279+0000 7f8de7fff640 1 -- 172.21.15.135:0/845498416 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de00d7a70 con 0x7f8de0049050 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.280+0000 7f8de7fff640 1 --2- 172.21.15.135:0/845498416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0049050 0x7f8de0008790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000b60 tx=0x7f8de80a75d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a8624356f661e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.281+0000 7f8de5ffb640 1 -- 172.21.15.135:0/845498416 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de806e650 con 0x7f8de0049050 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.281+0000 7f8de5ffb640 1 -- 172.21.15.135:0/845498416 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de806e830 con 0x7f8de0049050 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.282+0000 7f8ded7f0640 1 -- 172.21.15.135:0/845498416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0049050 msgr2=0x7f8de0008790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.282+0000 7f8ded7f0640 1 --2- 172.21.15.135:0/845498416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0049050 0x7f8de0008790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000b60 tx=0x7f8de80a75d0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.282+0000 7f8ded7f0640 1 -- 172.21.15.135:0/845498416 shutdown_connections 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.282+0000 7f8ded7f0640 1 --2- 172.21.15.135:0/845498416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0049050 0x7f8de0008790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.282+0000 7f8ded7f0640 1 --2- 172.21.15.135:0/845498416 >> v2:172.21.15.139:3300/0 conn(0x7f8de004c2a0 0x7f8de000bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.282+0000 7f8ded7f0640 1 -- 172.21.15.135:0/845498416 >> 172.21.15.135:0/845498416 conn(0x7f8de0049520 msgr2=0x7f8de0049930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.283+0000 7f8ded7f0640 1 -- 172.21.15.135:0/845498416 shutdown_connections 2024-04-26T22:00:15.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.283+0000 7f8ded7f0640 1 -- 172.21.15.135:0/845498416 wait complete. 2024-04-26T22:00:15.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.284+0000 7f8ded7f0640 1 Processor -- start 2024-04-26T22:00:15.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.285+0000 7f8ded7f0640 1 -- start start 2024-04-26T22:00:15.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8ded7f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de000c020 0x7f8de0195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8ded7f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 0x7f8de00cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8ded7f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 0x7f8de00d6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8de67fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 0x7f8de00cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8de67fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 0x7f8de00cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34060/0 (socket says 172.21.15.135:34060) 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8de7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 0x7f8de00d6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8de7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 0x7f8de00d6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40588/0 (socket says 172.21.15.135:40588) 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8de6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de000c020 0x7f8de0195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8de67fc640 1 -- 172.21.15.135:0/3132310475 learned_addr learned my addr 172.21.15.135:0/3132310475 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8de000ca90 con 0x7f8de00d25f0 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8de000c7b0 con 0x7f8de000c020 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.286+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8de000cd70 con 0x7f8de004c2a0 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.287+0000 7f8de67fc640 1 -- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de000c020 msgr2=0x7f8de0195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.287+0000 7f8de67fc640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de000c020 0x7f8de0195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.287+0000 7f8de67fc640 1 -- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 msgr2=0x7f8de00d6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.287+0000 7f8de67fc640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 0x7f8de00d6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.287+0000 7f8de67fc640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de00d4f90 con 0x7f8de004c2a0 2024-04-26T22:00:15.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.287+0000 7f8de7fff640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 0x7f8de00d6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:15.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.288+0000 7f8de67fc640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 0x7f8de00cd000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800b480 tx=0x7f8dd800b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.288+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8036070 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.288+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd80328a0 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.288+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8032c90 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.288+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de00d4410 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.288+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de00cd540 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.290+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dd8039070 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.291+0000 7f8dcf7fe640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8dd406c9f0 0x7f8dd406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.291+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f8dd80adbe0 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.292+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc8005290 con 0x7f8de004c2a0 2024-04-26T22:00:15.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.292+0000 7f8de6ffd640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8dd406c9f0 0x7f8dd406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.293+0000 7f8de6ffd640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8dd406c9f0 0x7f8dd406eeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00a730 tx=0x7f8ddc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.298+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8dd8040030 con 0x7f8de004c2a0 2024-04-26T22:00:15.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.332+0000 7fc7edf4d640 1 Processor -- start 2024-04-26T22:00:15.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.332+0000 7f0e91802640 1 Processor -- start 2024-04-26T22:00:15.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.332+0000 7fc7edf4d640 1 -- start start 2024-04-26T22:00:15.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.333+0000 7f4c70df0640 1 Processor -- start 2024-04-26T22:00:15.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.333+0000 7f4c70df0640 1 -- start start 2024-04-26T22:00:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f48733f0640 1 Processor -- start 2024-04-26T22:00:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.333+0000 7f0e91802640 1 -- start start 2024-04-26T22:00:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c70df0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c6c156740 0x7f4c6c0ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c70df0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c6c0c4530 0x7f4c6c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c70df0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c150520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c6affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c6c0c4530 0x7f4c6c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c70df0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6c0ac100 con 0x7f4c6c156740 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c70df0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6c0acad0 con 0x7f4c6c0c4530 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c70df0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c0ad030 con 0x7f4c6c1500d0 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c6bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c6bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34076/0 (socket says 172.21.15.135:34076) 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f4c6bfff640 1 -- 172.21.15.135:0/1148609297 learned_addr learned my addr 172.21.15.135:0/1148609297 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f48733f0640 1 -- start start 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f0e91802640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e8c0aa7c0 0x7f0e8c0c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f0e91802640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0abdc0 0x7f0e8c0ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f0e91802640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8c06dec0 con 0x7f0e8c15b940 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f0e91802640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0e8c0c2ac0 con 0x7f0e8c0aa7c0 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7f0e91802640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c06e1a0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e8c0aa7c0 0x7f0e8c0c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e8c0aa7c0 0x7f0e8c0c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33568/0 (socket says 172.21.15.135:33568) 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 -- 172.21.15.135:0/3100705559 learned_addr learned my addr 172.21.15.135:0/3100705559 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e91001640 1 --2- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0abdc0 0x7f0e8c0ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7edf4d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80c3d40 0x7fc7e80aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7edf4d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc7e80ac490 0x7fc7e815b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7edf4d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc7e815b950 0x7fc7e806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7edf4d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e815bde0 con 0x7fc7e80ac490 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7edf4d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e80ad610 con 0x7fc7e815b950 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7edf4d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e80aca00 con 0x7fc7e80c3d40 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7ed74c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc7e815b950 0x7fc7e806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7ed74c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc7e815b950 0x7fc7e806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33560/0 (socket says 172.21.15.135:33560) 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.334+0000 7fc7ed74c640 1 -- 172.21.15.135:0/852275955 learned_addr learned my addr 172.21.15.135:0/852275955 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7fc7ecf4b640 1 --2- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80c3d40 0x7fc7e80aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7fc7ed74c640 1 -- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80c3d40 msgr2=0x7fc7e80aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7fc7ed74c640 1 --2- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80c3d40 0x7fc7e80aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7fc7ed74c640 1 -- 172.21.15.135:0/852275955 >> v2:172.21.15.135:3300/0 conn(0x7fc7e80ac490 msgr2=0x7fc7e815b410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7fc7ed74c640 1 --2- 172.21.15.135:0/852275955 >> v2:172.21.15.135:3300/0 conn(0x7fc7e80ac490 0x7fc7e815b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 -- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0abdc0 msgr2=0x7f0e8c0ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 --2- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0abdc0 0x7f0e8c0ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 -- 172.21.15.135:0/3100705559 >> v1:172.21.15.139:6789/0 conn(0x7f0e8c15b940 legacy=0x7f0e8c0c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f0e8bfff640 1 -- 172.21.15.135:0/3100705559 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c06e480 con 0x7f0e8c0aa7c0 2024-04-26T22:00:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f4c6bfff640 1 -- 172.21.15.135:0/1148609297 >> v2:172.21.15.139:3300/0 conn(0x7f4c6c0c4530 msgr2=0x7f4c6c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f4c6bfff640 1 --2- 172.21.15.135:0/1148609297 >> v2:172.21.15.139:3300/0 conn(0x7f4c6c0c4530 0x7f4c6c06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f4c6bfff640 1 -- 172.21.15.135:0/1148609297 >> v2:172.21.15.135:3300/0 conn(0x7f4c6c156740 msgr2=0x7f4c6c0ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f4c6bfff640 1 --2- 172.21.15.135:0/1148609297 >> v2:172.21.15.135:3300/0 conn(0x7f4c6c156740 0x7f4c6c0ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f4c6bfff640 1 -- 172.21.15.135:0/1148609297 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c0ad2f0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f48733f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f486c158730 0x7f486c0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f48733f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c075b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f48733f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f486c15b4d0 0x7f486c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f48733f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0c5280 con 0x7f486c158730 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f48733f0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0acbd0 con 0x7f486c15b4d0 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7f48733f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f486c15e9d0 con 0x7f486c0756f0 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f486c15b4d0 0x7f486c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f486c15b4d0 0x7f486c15b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33570/0 (socket says 172.21.15.135:33570) 2024-04-26T22:00:15.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 -- 172.21.15.135:0/3294708829 learned_addr learned my addr 172.21.15.135:0/3294708829 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.335+0000 7fc7ed74c640 1 -- 172.21.15.135:0/852275955 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e80acce0 con 0x7fc7e815b950 2024-04-26T22:00:15.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7ff858c62640 1 Processor -- start 2024-04-26T22:00:15.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7ff858c62640 1 -- start start 2024-04-26T22:00:15.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487118c640 1 --2- 172.21.15.135:0/3294708829 >> v2:172.21.15.135:3300/0 conn(0x7f486c158730 0x7f486c0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f0e8bfff640 1 --2- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0aa7c0 0x7f0e8c0c3940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009e50 tx=0x7f0e8402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a517be66dfa109 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 -- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 msgr2=0x7f486c075b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f0e8b7fe640 1 -- 172.21.15.135:0/3100705559 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84036070 con 0x7f0e8c0aa7c0 2024-04-26T22:00:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f0e8b7fe640 1 -- 172.21.15.135:0/3100705559 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e84031600 con 0x7f0e8c0aa7c0 2024-04-26T22:00:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f0e8b7fe640 1 -- 172.21.15.135:0/3100705559 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84031a10 con 0x7f0e8c0aa7c0 2024-04-26T22:00:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 --2- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c075b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 -- 172.21.15.135:0/3294708829 >> v2:172.21.15.135:3300/0 conn(0x7f486c158730 msgr2=0x7f486c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 --2- 172.21.15.135:0/3294708829 >> v2:172.21.15.135:3300/0 conn(0x7f486c158730 0x7f486c0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.336+0000 7f487198d640 1 -- 172.21.15.135:0/3294708829 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c15ecb0 con 0x7f486c15b4d0 2024-04-26T22:00:15.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f4c6bfff640 1 --2- 172.21.15.135:0/1148609297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c150520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0c2990 tx=0x7f4c6402ead0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc3a78b0f4c46fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7fc7ed74c640 1 --2- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e815b950 0x7fc7e806d4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80c5990 tx=0x7fc7e003b170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718ea33bd0c4a012 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7fa524d11640 1 Processor -- start 2024-04-26T22:00:15.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7fc7e77fe640 1 -- 172.21.15.135:0/852275955 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0048070 con 0x7fc7e815b950 2024-04-26T22:00:15.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7fc7e77fe640 1 -- 172.21.15.135:0/852275955 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e0043440 con 0x7fc7e815b950 2024-04-26T22:00:15.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7fc7e77fe640 1 -- 172.21.15.135:0/852275955 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0043810 con 0x7fc7e815b950 2024-04-26T22:00:15.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f4c6a7fc640 1 -- 172.21.15.135:0/1148609297 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64036070 con 0x7f4c6c1500d0 2024-04-26T22:00:15.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f4c6a7fc640 1 -- 172.21.15.135:0/1148609297 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c64008500 con 0x7f4c6c1500d0 2024-04-26T22:00:15.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f4c6a7fc640 1 -- 172.21.15.135:0/1148609297 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c640088d0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f494359e640 1 Processor -- start 2024-04-26T22:00:15.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7ff858c62640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff8540c4040 0x7ff85406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7ff858c62640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff85406e870 0x7ff8540c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7ff858c62640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8540c4420 con 0x7ff8540749a0 2024-04-26T22:00:15.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7ff858c62640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff85406df40 con 0x7ff8540c4040 2024-04-26T22:00:15.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7ff858c62640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff8540ab380 con 0x7ff85406e870 2024-04-26T22:00:15.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f0e91802640 1 -- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0aa7c0 msgr2=0x7f0e8c0c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff8540c4040 0x7ff85406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff8540c4040 0x7ff85406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40620/0 (socket says 172.21.15.135:40620) 2024-04-26T22:00:15.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 -- 172.21.15.135:0/3912931920 learned_addr learned my addr 172.21.15.135:0/3912931920 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f0e91802640 1 --2- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0aa7c0 0x7f0e8c0c3940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009e50 tx=0x7f0e8402a6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff852d9d640 1 --2- 172.21.15.135:0/3912931920 >> v2:172.21.15.139:3300/0 conn(0x7ff85406e870 0x7ff8540c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 -- 172.21.15.135:0/3912931920 >> v2:172.21.15.139:3300/0 conn(0x7ff85406e870 msgr2=0x7ff8540c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 --2- 172.21.15.135:0/3912931920 >> v2:172.21.15.139:3300/0 conn(0x7ff85406e870 0x7ff8540c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 -- 172.21.15.135:0/3912931920 >> v1:172.21.15.139:6789/0 conn(0x7ff8540749a0 legacy=0x7ff8540c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 -- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e815b950 msgr2=0x7fc7e806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e815b950 0x7fc7e806d4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80c5990 tx=0x7fc7e003b170 comp rx=0 tx=0).stop 2024-04-26T22:00:15.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7ff85259c640 1 -- 172.21.15.135:0/3912931920 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8540ab660 con 0x7ff8540c4040 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 -- 172.21.15.135:0/852275955 shutdown_connections 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/852275955 >> v2:172.21.15.135:3300/0 conn(0x7fc7e80ac490 0x7fc7e815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e815b950 0x7fc7e806d4c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/852275955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80c3d40 0x7fc7e80aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fc7edf4d640 1 -- 172.21.15.135:0/852275955 >> 172.21.15.135:0/852275955 conn(0x7fc7e80c4bc0 msgr2=0x7fc7e80c4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7fa524d11640 1 -- start start 2024-04-26T22:00:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f487198d640 1 --2- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c15b960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f486c0c40c0 tx=0x7f486402a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ba5c6bad2b0bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4863fff640 1 -- 172.21.15.135:0/3294708829 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864036070 con 0x7f486c15b4d0 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4863fff640 1 -- 172.21.15.135:0/3294708829 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4864031600 con 0x7f486c15b4d0 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa524d11640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa520158610 0x7fa5200c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa524d11640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5200c33f0 0x7fa5200c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa524d11640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52015bb40 con 0x7fa52006f680 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa524d11640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa5201589f0 con 0x7fa520158610 2024-04-26T22:00:15.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4863fff640 1 -- 172.21.15.135:0/3294708829 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640319d0 con 0x7f486c15b4d0 2024-04-26T22:00:15.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa524d11640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa52006f1f0 con 0x7fa5200c33f0 2024-04-26T22:00:15.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f0e91802640 1 -- 172.21.15.135:0/3100705559 shutdown_connections 2024-04-26T22:00:15.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f0e91802640 1 --2- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0abdc0 0x7f0e8c0ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f0e91802640 1 --2- 172.21.15.135:0/3100705559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0aa7c0 0x7f0e8c0c3940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f0e91802640 1 -- 172.21.15.135:0/3100705559 >> 172.21.15.135:0/3100705559 conn(0x7f0e8c0c80c0 msgr2=0x7f0e8c0c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa51dd9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5200c33f0 0x7fa5200c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7fa51ed9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa52006f680 0x7fa5200c5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53024/0 (socket says 172.21.15.135:53024) 2024-04-26T22:00:15.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51ed9d640 1 -- 172.21.15.135:0/3331637687 learned_addr learned my addr 172.21.15.135:0/3331637687 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:15.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51e59c640 1 --2- 172.21.15.135:0/3331637687 >> v2:172.21.15.135:3300/0 conn(0x7fa520158610 0x7fa5200c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.337+0000 7f494359e640 1 -- start start 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 -- 172.21.15.135:0/1148609297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 msgr2=0x7f4c6c150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 --2- 172.21.15.135:0/1148609297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c150520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0c2990 tx=0x7f4c6402ead0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 -- 172.21.15.135:0/1148609297 shutdown_connections 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 --2- 172.21.15.135:0/1148609297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c150520 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 --2- 172.21.15.135:0/1148609297 >> v2:172.21.15.139:3300/0 conn(0x7f4c6c0c4530 0x7f4c6c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 --2- 172.21.15.135:0/1148609297 >> v2:172.21.15.135:3300/0 conn(0x7f4c6c156740 0x7f4c6c0ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.338+0000 7f4c70df0640 1 -- 172.21.15.135:0/1148609297 >> 172.21.15.135:0/1148609297 conn(0x7f4c6c0c4bc0 msgr2=0x7f4c6c0c4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f494359e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49440aa7c0 0x7f49440c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fc7edf4d640 1 -- 172.21.15.135:0/852275955 shutdown_connections 2024-04-26T22:00:15.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fc7edf4d640 1 -- 172.21.15.135:0/852275955 wait complete. 2024-04-26T22:00:15.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51d59a640 1 -- 172.21.15.135:0/3331637687 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582324006 0 0) 0x7fa52015bb40 con 0x7fa52006f680 2024-04-26T22:00:15.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f494359e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49440abdc0 0x7f49440ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51d59a640 1 -- 172.21.15.135:0/3331637687 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa508003520 con 0x7fa52006f680 2024-04-26T22:00:15.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51dd9b640 1 -- 172.21.15.135:0/3331637687 >> v2:172.21.15.135:3300/0 conn(0x7fa520158610 msgr2=0x7fa5200c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51dd9b640 1 --2- 172.21.15.135:0/3331637687 >> v2:172.21.15.135:3300/0 conn(0x7fa520158610 0x7fa5200c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51dd9b640 1 -- 172.21.15.135:0/3331637687 >> v1:172.21.15.135:6789/0 conn(0x7fa52006f680 legacy=0x7fa5200c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49440aa7c0 0x7f49440c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f494359e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494406dec0 con 0x7f494415b940 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f494359e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f49440c2ac0 con 0x7f49440abdc0 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f494359e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f494406e1a0 con 0x7f49440aa7c0 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49440aa7c0 0x7f49440c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33594/0 (socket says 172.21.15.135:33594) 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 -- 172.21.15.135:0/741157039 learned_addr learned my addr 172.21.15.135:0/741157039 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7fa51dd9b640 1 -- 172.21.15.135:0/3331637687 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200c3920 con 0x7fa5200c33f0 2024-04-26T22:00:15.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f0e91802640 1 -- 172.21.15.135:0/3100705559 shutdown_connections 2024-04-26T22:00:15.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f0e91802640 1 -- 172.21.15.135:0/3100705559 wait complete. 2024-04-26T22:00:15.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f48733f0640 1 -- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 msgr2=0x7f486c15b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f48733f0640 1 --2- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c15b960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f486c0c40c0 tx=0x7f486402a610 comp rx=0 tx=0).stop 2024-04-26T22:00:15.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f4c70df0640 1 -- 172.21.15.135:0/1148609297 shutdown_connections 2024-04-26T22:00:15.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f4c70df0640 1 -- 172.21.15.135:0/1148609297 wait complete. 2024-04-26T22:00:15.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7ff85259c640 1 --2- 172.21.15.135:0/3912931920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8540c4040 0x7ff85406e330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff8540aac30 tx=0x7ff84c02a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c783564b6122afd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7ff85159a640 1 -- 172.21.15.135:0/3912931920 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c037070 con 0x7ff8540c4040 2024-04-26T22:00:15.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 -- 172.21.15.135:0/741157039 >> v2:172.21.15.135:3300/0 conn(0x7f49440abdc0 msgr2=0x7f49440ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 --2- 172.21.15.135:0/741157039 >> v2:172.21.15.135:3300/0 conn(0x7f49440abdc0 0x7f49440ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 -- 172.21.15.135:0/741157039 >> v1:172.21.15.135:6789/0 conn(0x7f494415b940 legacy=0x7f49440c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:15.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.339+0000 7f4941d9b640 1 -- 172.21.15.135:0/741157039 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494406e480 con 0x7f49440aa7c0 2024-04-26T22:00:15.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7ff85159a640 1 -- 172.21.15.135:0/3912931920 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c032460 con 0x7ff8540c4040 2024-04-26T22:00:15.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7ff85159a640 1 -- 172.21.15.135:0/3912931920 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c032830 con 0x7ff8540c4040 2024-04-26T22:00:15.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7fa51dd9b640 1 --2- 172.21.15.135:0/3331637687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5200c33f0 0x7fa5200c4e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097a0 tx=0x7fa51002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db4ca5339cba72c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f48733f0640 1 -- 172.21.15.135:0/3294708829 shutdown_connections 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f48733f0640 1 --2- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c15b960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f48733f0640 1 --2- 172.21.15.135:0/3294708829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c075b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f48733f0640 1 --2- 172.21.15.135:0/3294708829 >> v2:172.21.15.135:3300/0 conn(0x7f486c158730 0x7f486c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.340+0000 7f48733f0640 1 -- 172.21.15.135:0/3294708829 >> 172.21.15.135:0/3294708829 conn(0x7f486c0c9600 msgr2=0x7f486c0c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f48733f0640 1 -- 172.21.15.135:0/3294708829 shutdown_connections 2024-04-26T22:00:15.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f4941d9b640 1 --2- 172.21.15.135:0/741157039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440aa7c0 0x7f49440c3940 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4934009870 tx=0x7f493402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e8297f2e5d419 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7fa51d59a640 1 -- 172.21.15.135:0/3331637687 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510037070 con 0x7fa5200c33f0 2024-04-26T22:00:15.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7fa51d59a640 1 -- 172.21.15.135:0/3331637687 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51002adb0 con 0x7fa5200c33f0 2024-04-26T22:00:15.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7fa51d59a640 1 -- 172.21.15.135:0/3331637687 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510032660 con 0x7fa5200c33f0 2024-04-26T22:00:15.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f0e91802640 1 Processor -- start 2024-04-26T22:00:15.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7ff858c62640 1 -- 172.21.15.135:0/3912931920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8540c4040 msgr2=0x7ff85406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7ff858c62640 1 --2- 172.21.15.135:0/3912931920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8540c4040 0x7ff85406e330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff8540aac30 tx=0x7ff84c02a870 comp rx=0 tx=0).stop 2024-04-26T22:00:15.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f48733f0640 1 -- 172.21.15.135:0/3294708829 wait complete. 2024-04-26T22:00:15.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7ff858c62640 1 -- 172.21.15.135:0/3912931920 shutdown_connections 2024-04-26T22:00:15.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7ff858c62640 1 --2- 172.21.15.135:0/3912931920 >> v2:172.21.15.139:3300/0 conn(0x7ff85406e870 0x7ff8540c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f494159a640 1 -- 172.21.15.135:0/741157039 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934037070 con 0x7f49440aa7c0 2024-04-26T22:00:15.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f494159a640 1 -- 172.21.15.135:0/741157039 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4934032440 con 0x7f49440aa7c0 2024-04-26T22:00:15.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f494159a640 1 -- 172.21.15.135:0/741157039 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934032810 con 0x7f49440aa7c0 2024-04-26T22:00:15.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7ff858c62640 1 --2- 172.21.15.135:0/3912931920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8540c4040 0x7ff85406e330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7ff858c62640 1 -- 172.21.15.135:0/3912931920 >> 172.21.15.135:0/3912931920 conn(0x7ff8540c81d0 msgr2=0x7ff8540c85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f4c70df0640 1 Processor -- start 2024-04-26T22:00:15.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7fa524d11640 1 -- 172.21.15.135:0/3331637687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5200c33f0 msgr2=0x7fa5200c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7fa524d11640 1 --2- 172.21.15.135:0/3331637687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5200c33f0 0x7fa5200c4e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097a0 tx=0x7fa51002a060 comp rx=0 tx=0).stop 2024-04-26T22:00:15.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f4c70df0640 1 -- start start 2024-04-26T22:00:15.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f48733f0640 1 Processor -- start 2024-04-26T22:00:15.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f48733f0640 1 -- start start 2024-04-26T22:00:15.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7f0e91802640 1 -- start start 2024-04-26T22:00:15.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f0e91802640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0aa7c0 0x7f0e8c0c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f0e91802640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0abdc0 0x7f0e8c1e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f0e91802640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 0x7f0e8c1e7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f494359e640 1 -- 172.21.15.135:0/741157039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440aa7c0 msgr2=0x7f49440c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7f494359e640 1 --2- 172.21.15.135:0/741157039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440aa7c0 0x7f49440c3940 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4934009870 tx=0x7f493402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fa524d11640 1 -- 172.21.15.135:0/3331637687 shutdown_connections 2024-04-26T22:00:15.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fa524d11640 1 --2- 172.21.15.135:0/3331637687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5200c33f0 0x7fa5200c4e30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fa524d11640 1 --2- 172.21.15.135:0/3331637687 >> v2:172.21.15.135:3300/0 conn(0x7fa520158610 0x7fa5200c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fa524d11640 1 -- 172.21.15.135:0/3331637687 >> 172.21.15.135:0/3331637687 conn(0x7fa5200c89f0 msgr2=0x7fa5200c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f494359e640 1 -- 172.21.15.135:0/741157039 shutdown_connections 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f494359e640 1 --2- 172.21.15.135:0/741157039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440aa7c0 0x7f49440c3940 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f494359e640 1 --2- 172.21.15.135:0/741157039 >> v2:172.21.15.135:3300/0 conn(0x7f49440abdc0 0x7f49440ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f494359e640 1 -- 172.21.15.135:0/741157039 >> 172.21.15.135:0/741157039 conn(0x7f49440c80c0 msgr2=0x7f49440c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fa524d11640 1 -- 172.21.15.135:0/3331637687 shutdown_connections 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7ff858c62640 1 -- 172.21.15.135:0/3912931920 shutdown_connections 2024-04-26T22:00:15.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7ff858c62640 1 -- 172.21.15.135:0/3912931920 wait complete. 2024-04-26T22:00:15.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fa524d11640 1 -- 172.21.15.135:0/3331637687 wait complete. 2024-04-26T22:00:15.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e91001640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 0x7f0e8c1e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e91001640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 0x7f0e8c1e7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40656/0 (socket says 172.21.15.135:40656) 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e8bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0abdc0 0x7f0e8c1e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e91001640 1 -- 172.21.15.135:0/1349622756 learned_addr learned my addr 172.21.15.135:0/1349622756 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c15bea0 con 0x7f0e8c15b940 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c15bd20 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7fc7edf4d640 1 Processor -- start 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.341+0000 7fc7edf4d640 1 -- start start 2024-04-26T22:00:15.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7fc7edf4d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 0x7fc7e8156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c0ac780 con 0x7f0e8c0aa7c0 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7fc7edf4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e80c3d40 0x7fc7e814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7fc7edf4d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7e815b950 0x7fc7e814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.342+0000 7fc7edf4d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e80c4790 con 0x7fc7e815b950 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7edf4d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e80ad2a0 con 0x7fc7e80c3d40 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7edf4d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e80c4910 con 0x7fc7e80ac490 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7ecf4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 0x7fc7e8156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7ed74c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7e815b950 0x7fc7e814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7ecf4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 0x7fc7e8156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34090/0 (socket says 172.21.15.135:34090) 2024-04-26T22:00:15.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7ecf4b640 1 -- 172.21.15.135:0/2240303718 learned_addr learned my addr 172.21.15.135:0/2240303718 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f4c70df0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 0x7f4c6c0c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7fc7e7fff640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e80c3d40 0x7fc7e814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f48733f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c1e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f48733f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c158730 0x7f486c0be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7ff858c62640 1 Processor -- start 2024-04-26T22:00:15.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7ff858c62640 1 -- start start 2024-04-26T22:00:15.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f48733f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c1f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f48733f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0ae410 con 0x7f486c158730 2024-04-26T22:00:15.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f48733f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f486c15f270 con 0x7f486c15b4d0 2024-04-26T22:00:15.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f48733f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0ae590 con 0x7f486c0756f0 2024-04-26T22:00:15.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.343+0000 7f494359e640 1 -- 172.21.15.135:0/741157039 shutdown_connections 2024-04-26T22:00:15.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487118c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c1e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f4c6b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 0x7f4c6c0c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487118c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c1e54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34098/0 (socket says 172.21.15.135:34098) 2024-04-26T22:00:15.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f4c6b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 0x7f4c6c0c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40678/0 (socket says 172.21.15.135:40678) 2024-04-26T22:00:15.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f4c6b7fe640 1 -- 172.21.15.135:0/815705327 learned_addr learned my addr 172.21.15.135:0/815705327 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c1e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ecf4b640 1 -- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e80c3d40 msgr2=0x7fc7e814e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ecf4b640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e80c3d40 0x7fc7e814e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ecf4b640 1 -- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7e815b950 msgr2=0x7fc7e814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ecf4b640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7e815b950 0x7fc7e814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ecf4b640 1 -- 172.21.15.135:0/2240303718 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e814c730 con 0x7fc7e80ac490 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ed74c640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7e815b950 0x7fc7e814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c1f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c1f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33628/0 (socket says 172.21.15.135:33628) 2024-04-26T22:00:15.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 -- 172.21.15.135:0/4265178223 learned_addr learned my addr 172.21.15.135:0/4265178223 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f0e8bfff640 1 -- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0aa7c0 msgr2=0x7f0e8c0c34a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:15.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487098b640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c158730 0x7f486c0be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7ecf4b640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 0x7fc7e8156ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000c820 tx=0x7fc7d000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 -- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 msgr2=0x7f486c1e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0015070 con 0x7fc7e80ac490 2024-04-26T22:00:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c1e54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7ff858c62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 0x7ff8541513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d0004240 con 0x7fc7e80ac490 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000ce10 con 0x7fc7e80ac490 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81e8b10 con 0x7fc7e80ac490 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 -- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c158730 msgr2=0x7f486c0be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c158730 0x7f486c0be6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f487198d640 1 -- 172.21.15.135:0/4265178223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c1e4f00 con 0x7f486c15b4d0 2024-04-26T22:00:15.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f0e8bfff640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0aa7c0 0x7f0e8c0c34a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff858c62640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8540749a0 0x7ff85415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff858c62640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8540c4040 0x7ff854157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff858c62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540abf00 con 0x7ff85406e870 2024-04-26T22:00:15.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff858c62640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540abc20 con 0x7ff8540c4040 2024-04-26T22:00:15.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff858c62640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff8540ac1e0 con 0x7ff8540749a0 2024-04-26T22:00:15.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e814efe0 con 0x7fc7e80ac490 2024-04-26T22:00:15.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 0x7ff8541513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 0x7ff8541513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40692/0 (socket says 172.21.15.135:40692) 2024-04-26T22:00:15.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 -- 172.21.15.135:0/2397469414 learned_addr learned my addr 172.21.15.135:0/2397469414 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff852d9d640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8540c4040 0x7ff854157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f487198d640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c1f3f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f486402ac00 tx=0x7f4864004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff851d9b640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8540749a0 0x7ff85415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 -- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8540749a0 msgr2=0x7ff85415e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8540749a0 0x7ff85415e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 -- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8540c4040 msgr2=0x7ff854157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f0e8bfff640 1 -- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 msgr2=0x7f0e8c1e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f0e8bfff640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 0x7f0e8c1e7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f0e8bfff640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c155550 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f0e91001640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 0x7f0e8c1e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f0e8bfff640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0abdc0 0x7f0e8c1e7230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e84030000 tx=0x7f0e84004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8540c4040 0x7ff854157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 -- 172.21.15.135:0/2397469414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8541f0850 con 0x7ff85406e870 2024-04-26T22:00:15.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84041020 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7fa524d11640 1 Processor -- start 2024-04-26T22:00:15.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7fa524d11640 1 -- start start 2024-04-26T22:00:15.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7ff85259c640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 0x7ff8541513a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b290 tx=0x7ff84c004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e840087b0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84036040 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c1552b0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1f47c0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.344+0000 7f494359e640 1 -- 172.21.15.135:0/741157039 wait complete. 2024-04-26T22:00:15.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f494359e640 1 Processor -- start 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864041020 con 0x7f486c15b4d0 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4864037040 con 0x7f486c15b4d0 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c156740 0x7f4c6c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c0c3ad0 con 0x7f4c6c0c4530 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c06dbb0 con 0x7f4c6c156740 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.345+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c0c3c50 con 0x7f4c6c1500d0 2024-04-26T22:00:15.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6affd640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c1e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7d0007830 con 0x7fc7e80ac490 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6bfff640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c156740 0x7f4c6c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6affd640 1 -- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c156740 msgr2=0x7f4c6c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6affd640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c156740 0x7f4c6c14a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6affd640 1 -- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 msgr2=0x7f4c6c0c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6affd640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 0x7f4c6c0c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4c6affd640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c1e8030 con 0x7f4c6c1500d0 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f494359e640 1 -- start start 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 0x7f4944073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 0x7f49440a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f494415b940 0x7f49440a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 0x7f4944073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 0x7f4944073f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40700/0 (socket says 172.21.15.135:40700) 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4941d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 0x7f49440a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f4941d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 0x7f49440a6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33670/0 (socket says 172.21.15.135:33670) 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494359e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f494415bea0 con 0x7f49440aa7c0 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f494415bd20 con 0x7f49440abdc0 2024-04-26T22:00:15.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494359e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f49440ac7b0 con 0x7f494415b940 2024-04-26T22:00:15.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c040020 con 0x7ff85406e870 2024-04-26T22:00:15.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8541f0db0 con 0x7ff85406e870 2024-04-26T22:00:15.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8541f1370 con 0x7ff85406e870 2024-04-26T22:00:15.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c03c610 con 0x7ff85406e870 2024-04-26T22:00:15.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c037040 con 0x7ff85406e870 2024-04-26T22:00:15.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7fc7e57fa640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7c806c9c0 0x7fc7c806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c1f47a0 con 0x7f486c15b4d0 2024-04-26T22:00:15.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c1f44a0 con 0x7f486c15b4d0 2024-04-26T22:00:15.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.346+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864036070 con 0x7f486c15b4d0 2024-04-26T22:00:15.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc7d008bcd0 con 0x7fc7e80ac490 2024-04-26T22:00:15.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f4c6b7fe640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 0x7f4c6c0c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:15.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7fc7e7fff640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7c806c9c0 0x7fc7c806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f4c6affd640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c1e8ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d5f0 tx=0x7f4c5c00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.347+0000 7f494259c640 1 -- 172.21.15.135:0/3591102255 learned_addr learned my addr 172.21.15.135:0/3591102255 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f4942d9d640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f494415b940 0x7f49440a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f494259c640 1 -- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f494415b940 msgr2=0x7f49440a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f494259c640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f494415b940 0x7f49440a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f494259c640 1 -- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 msgr2=0x7f49440a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f494259c640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 0x7f49440a6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f494259c640 1 -- 172.21.15.135:0/3591102255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49440a75d0 con 0x7f49440aa7c0 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f494259c640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 0x7f4944073f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f493800b420 tx=0x7f493800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f4941d9b640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 0x7f49440a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T22:00:15.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0125f0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c019070 con 0x7f4c6c1500d0 2024-04-26T22:00:15.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0129d0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa524d11640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa52006f680 0x7fa520073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa524d11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 0x7fa520151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff84c043020 con 0x7ff85406e870 2024-04-26T22:00:15.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7ff842ffd640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff83406c8f0 0x7ff83406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ff84c0ac4e0 con 0x7ff85406e870 2024-04-26T22:00:15.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa524d11640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 0x7fa5201597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa51dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 0x7fa520151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838005290 con 0x7ff85406e870 2024-04-26T22:00:15.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fc7e7fff640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7c806c9c0 0x7fc7c806ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8155b60 tx=0x7fc7d800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa51ed9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 0x7fa5201597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa51ed9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 0x7fa5201597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34132/0 (socket says 172.21.15.135:34132) 2024-04-26T22:00:15.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa51dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 0x7fa520151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33672/0 (socket says 172.21.15.135:33672) 2024-04-26T22:00:15.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6c14b7e0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6c14a6d0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa51ed9d640 1 -- 172.21.15.135:0/1885292415 learned_addr learned my addr 172.21.15.135:0/1885292415 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:15.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa52015bde0 con 0x7fa52006f680 2024-04-26T22:00:15.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa520070460 con 0x7fa5200c33f0 2024-04-26T22:00:15.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa5200c5b90 con 0x7fa520158610 2024-04-26T22:00:15.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e80506c0 con 0x7fc7e80ac490 2024-04-26T22:00:15.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.349+0000 7ff851d9b640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff83406c8f0 0x7ff83406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938011830 con 0x7f49440aa7c0 2024-04-26T22:00:15.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4938016070 con 0x7f49440aa7c0 2024-04-26T22:00:15.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49440730f0 con 0x7f49440aa7c0 2024-04-26T22:00:15.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441f4ef0 con 0x7f49440aa7c0 2024-04-26T22:00:15.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e84044020 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7ff851d9b640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff83406c8f0 0x7ff83406edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8540aa520 tx=0x7ff848009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.348+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c0506c0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c0506c0 con 0x7f486c15b4d0 2024-04-26T22:00:15.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fa51ed9d640 1 -- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 msgr2=0x7fa520151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fa51ed9d640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 0x7fa520151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fa51ed9d640 1 -- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa52006f680 msgr2=0x7fa520073870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:00:15.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.350+0000 7fa51e59c640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa52006f680 0x7fa520073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6c0506c0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7fa51ed9d640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa52006f680 0x7fa520073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7fa51ed9d640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa520152440 con 0x7fa520158610 2024-04-26T22:00:15.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7fa51e59c640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa52006f680 0x7fa520073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:15.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4864044020 con 0x7f486c15b4d0 2024-04-26T22:00:15.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7fa51dd9b640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 0x7fa520151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:15.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7fa51ed9d640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 0x7fa5201597c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa518001460 tx=0x7fa51800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f0e897fa640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e7006c9c0 0x7f0e7006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938011bf0 con 0x7f49440aa7c0 2024-04-26T22:00:15.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0e840aca30 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f4861ffb640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f485006c9c0 0x7f485006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.351+0000 7f0e90800640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e7006c9c0 0x7f0e7006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c5c012b70 con 0x7f4c6c1500d0 2024-04-26T22:00:15.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f48640ac930 con 0x7f486c15b4d0 2024-04-26T22:00:15.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f487118c640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f485006c9c0 0x7f485006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f0e90800640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e7006c9c0 0x7f0e7006ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c000f60 tx=0x7f0e7c0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f487118c640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f485006c9c0 0x7f485006ee80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f485c006fd0 tx=0x7f485c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f4c53fff640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c5806c970 0x7f4c5806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f4c6b7fe640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c5806c970 0x7f4c5806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f4c5c08bfe0 con 0x7f4c6c1500d0 2024-04-26T22:00:15.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.352+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4928005290 con 0x7f49440aa7c0 2024-04-26T22:00:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.353+0000 7f4c6b7fe640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c5806c970 0x7f4c5806ee30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006fd0 tx=0x7f4c60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.355+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff84c04a030 con 0x7ff85406e870 2024-04-26T22:00:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.356+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518017070 con 0x7fa520158610 2024-04-26T22:00:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.356+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa520074c20 con 0x7fa520158610 2024-04-26T22:00:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.356+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa520072d90 con 0x7fa520158610 2024-04-26T22:00:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.356+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e8404b030 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.357+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7d0059760 con 0x7fc7e80ac490 2024-04-26T22:00:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.357+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5200765e0 con 0x7fa520158610 2024-04-26T22:00:15.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.358+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f493801a3c0 con 0x7f49440aa7c0 2024-04-26T22:00:15.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.358+0000 7f4932ffd640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f492406c970 0x7f492406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.358+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f493808c110 con 0x7f49440aa7c0 2024-04-26T22:00:15.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.359+0000 7f4941d9b640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f492406c970 0x7f492406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.359+0000 7f4941d9b640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f492406c970 0x7f492406ee30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4934034000 tx=0x7f493402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.360+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51800ac30 con 0x7fa520158610 2024-04-26T22:00:15.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.360+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518021060 con 0x7fa520158610 2024-04-26T22:00:15.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.360+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f486404b030 con 0x7f486c15b4d0 2024-04-26T22:00:15.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.360+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c6c1e8030 con 0x7f4c6c1500d0 2024-04-26T22:00:15.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.361+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4938020020 con 0x7f49440aa7c0 2024-04-26T22:00:15.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.363+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa51801a070 con 0x7fa520158610 2024-04-26T22:00:15.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.363+0000 7fa50effd640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa50006c920 0x7fa50006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:15.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.363+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa51808c380 con 0x7fa520158610 2024-04-26T22:00:15.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.364+0000 7fa51e59c640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa50006c920 0x7fa50006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:15.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.365+0000 7fa51e59c640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa50006c920 0x7fa50006ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa514006fd0 tx=0x7fa514008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:15.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.366+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa520152440 con 0x7fa520158610 2024-04-26T22:00:15.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.698+0000 7f8ded7f0640 1 -- 172.21.15.135:0/3132310475 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8dc8005580 con 0x7f8de004c2a0 2024-04-26T22:00:15.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.699+0000 7f8dcf7fe640 1 -- 172.21.15.135:0/3132310475 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8dd807be00 con 0x7f8de004c2a0 2024-04-26T22:00:15.699 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.702+0000 7f8dcd7fa640 1 -- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8dd406c9f0 msgr2=0x7f8dd406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.702+0000 7f8dcd7fa640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8dd406c9f0 0x7f8dd406eeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00a730 tx=0x7f8ddc008040 comp rx=0 tx=0).stop 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.702+0000 7f8dcd7fa640 1 -- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 msgr2=0x7f8de00cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.702+0000 7f8dcd7fa640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 0x7f8de00cd000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800b480 tx=0x7f8dd800b940 comp rx=0 tx=0).stop 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 -- 172.21.15.135:0/3132310475 shutdown_connections 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8dd406c9f0 0x7f8dd406eeb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de00d25f0 0x7f8de00d6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8de004c2a0 0x7f8de00cd000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 --2- 172.21.15.135:0/3132310475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de000c020 0x7f8de0195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 -- 172.21.15.135:0/3132310475 >> 172.21.15.135:0/3132310475 conn(0x7f8de0049520 msgr2=0x7f8de0005bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 -- 172.21.15.135:0/3132310475 shutdown_connections 2024-04-26T22:00:15.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.703+0000 7f8dcd7fa640 1 -- 172.21.15.135:0/3132310475 wait complete. 2024-04-26T22:00:15.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-04-26T22:00:15.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.744+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4928005850 con 0x7f49440aa7c0 2024-04-26T22:00:15.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.745+0000 7f4932ffd640 1 -- 172.21.15.135:0/3591102255 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f493805a380 con 0x7f49440aa7c0 2024-04-26T22:00:15.745 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:15.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f492406c970 msgr2=0x7f492406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f492406c970 0x7f492406ee30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4934034000 tx=0x7f493402e000 comp rx=0 tx=0).stop 2024-04-26T22:00:15.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 msgr2=0x7f4944073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 0x7f4944073f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f493800b420 tx=0x7f493800b8e0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 shutdown_connections 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f492406c970 0x7f492406ee30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f494415b940 0x7f49440a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49440abdc0 0x7f49440a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 --2- 172.21.15.135:0/3591102255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49440aa7c0 0x7f4944073f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 >> 172.21.15.135:0/3591102255 conn(0x7f49440c80c0 msgr2=0x7f49440c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.747+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 shutdown_connections 2024-04-26T22:00:15.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.748+0000 7f494359e640 1 -- 172.21.15.135:0/3591102255 wait complete. 2024-04-26T22:00:15.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-04-26T22:00:15.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.778+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff838005580 con 0x7ff85406e870 2024-04-26T22:00:15.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.778+0000 7ff842ffd640 1 -- 172.21.15.135:0/2397469414 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff8541f0850 con 0x7ff85406e870 2024-04-26T22:00:15.778 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff83406c8f0 msgr2=0x7ff83406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff83406c8f0 0x7ff83406edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8540aa520 tx=0x7ff848009040 comp rx=0 tx=0).stop 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 msgr2=0x7ff8541513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 0x7ff8541513a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b290 tx=0x7ff84c004a70 comp rx=0 tx=0).stop 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 shutdown_connections 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff83406c8f0 0x7ff83406edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8540c4040 0x7ff854157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8540749a0 0x7ff85415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 --2- 172.21.15.135:0/2397469414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85406e870 0x7ff8541513a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 >> 172.21.15.135:0/2397469414 conn(0x7ff8540c81d0 msgr2=0x7ff8540c72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 shutdown_connections 2024-04-26T22:00:15.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.781+0000 7ff858c62640 1 -- 172.21.15.135:0/2397469414 wait complete. 2024-04-26T22:00:15.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.783+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc7e80aba60 con 0x7fc7e80ac490 2024-04-26T22:00:15.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.783+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0e8c0abba0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.784+0000 7fc7e57fa640 1 -- 172.21.15.135:0/2240303718 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7d005da60 con 0x7fc7e80ac490 2024-04-26T22:00:15.784 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:15.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.784+0000 7f0e897fa640 1 -- 172.21.15.135:0/1349622756 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e8407aba0 con 0x7f0e8c0abdc0 2024-04-26T22:00:15.784 INFO:teuthology.orchestra.run.smithi135.stdout:55834574854 2024-04-26T22:00:15.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.786+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f486c0c5740 con 0x7f486c15b4d0 2024-04-26T22:00:15.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.786+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4c6c0acc90 con 0x7f4c6c1500d0 2024-04-26T22:00:15.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.786+0000 7f4861ffb640 1 -- 172.21.15.135:0/4265178223 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f486407aaa0 con 0x7f486c15b4d0 2024-04-26T22:00:15.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.786+0000 7f4c53fff640 1 -- 172.21.15.135:0/815705327 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c5c05a250 con 0x7f4c6c1500d0 2024-04-26T22:00:15.787 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:15.787 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:15.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa520070010 con 0x7fa520158610 2024-04-26T22:00:15.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7c806c9c0 msgr2=0x7fc7c806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7c806c9c0 0x7fc7c806ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8155b60 tx=0x7fc7d800d040 comp rx=0 tx=0).stop 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 msgr2=0x7fc7e8156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 0x7fc7e8156ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000c820 tx=0x7fc7d000cce0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 shutdown_connections 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc7c806c9c0 0x7fc7c806ee80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7e815b950 0x7fc7e814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e80c3d40 0x7fc7e814e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 --2- 172.21.15.135:0/2240303718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e80ac490 0x7fc7e8156ff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 >> 172.21.15.135:0/2240303718 conn(0x7fc7e80c4bc0 msgr2=0x7fc7e80c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e7006c9c0 msgr2=0x7f0e7006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7f0e91802640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e7006c9c0 0x7f0e7006ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c000f60 tx=0x7f0e7c0079b0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0abdc0 msgr2=0x7f0e8c1e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7f0e91802640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0abdc0 0x7f0e8c1e7230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e84030000 tx=0x7f0e84004a40 comp rx=0 tx=0).stop 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.787+0000 7fa50effd640 1 -- 172.21.15.135:0/1885292415 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa51800ae20 con 0x7fa520158610 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 shutdown_connections 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e7006c9c0 0x7f0e7006ee80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e8c15b940 0x7f0e8c1e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e8c0abdc0 0x7f0e8c1e7230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 --2- 172.21.15.135:0/1349622756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e8c0aa7c0 0x7f0e8c0c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 >> 172.21.15.135:0/1349622756 conn(0x7f0e8c0c80c0 msgr2=0x7f0e8c0c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 shutdown_connections 2024-04-26T22:00:15.790 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:15.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7fc7edf4d640 1 -- 172.21.15.135:0/2240303718 wait complete. 2024-04-26T22:00:15.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.788+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 shutdown_connections 2024-04-26T22:00:15.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.789+0000 7f0e91802640 1 -- 172.21.15.135:0/1349622756 wait complete. 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c5806c970 msgr2=0x7f4c5806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c5806c970 0x7f4c5806ee30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006fd0 tx=0x7f4c60008040 comp rx=0 tx=0).stop 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 msgr2=0x7f4c6c1e8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c1e8ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d5f0 tx=0x7f4c5c00dab0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 shutdown_connections 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c5806c970 0x7f4c5806ee30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c156740 0x7f4c6c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c6c1500d0 0x7f4c6c1e8ab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 --2- 172.21.15.135:0/815705327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6c0c4530 0x7f4c6c0c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 >> 172.21.15.135:0/815705327 conn(0x7f4c6c0c4bc0 msgr2=0x7f4c6c06e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f485006c9c0 msgr2=0x7f485006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f48733f0640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f485006c9c0 0x7f485006ee80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f485c006fd0 tx=0x7f485c008040 comp rx=0 tx=0).stop 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 msgr2=0x7f486c1f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.790+0000 7f48733f0640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c1f3f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f486402ac00 tx=0x7f4864004a40 comp rx=0 tx=0).stop 2024-04-26T22:00:15.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 shutdown_connections 2024-04-26T22:00:15.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 shutdown_connections 2024-04-26T22:00:15.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f485006c9c0 0x7f485006ee80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f486c15b4d0 0x7f486c1f3f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c158730 0x7f486c0be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 --2- 172.21.15.135:0/4265178223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486c0756f0 0x7f486c1e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 >> 172.21.15.135:0/4265178223 conn(0x7f486c0c9600 msgr2=0x7f486c06f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f4c70df0640 1 -- 172.21.15.135:0/815705327 wait complete. 2024-04-26T22:00:15.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 shutdown_connections 2024-04-26T22:00:15.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7f48733f0640 1 -- 172.21.15.135:0/4265178223 wait complete. 2024-04-26T22:00:15.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa50006c920 msgr2=0x7fa50006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.791+0000 7fa524d11640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa50006c920 0x7fa50006ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa514006fd0 tx=0x7fa514008040 comp rx=0 tx=0).stop 2024-04-26T22:00:15.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 msgr2=0x7fa5201597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:15.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 0x7fa5201597c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa518001460 tx=0x7fa51800cf90 comp rx=0 tx=0).stop 2024-04-26T22:00:15.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 shutdown_connections 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa50006c920 0x7fa50006ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa520158610 0x7fa5201597c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5200c33f0 0x7fa520151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 --2- 172.21.15.135:0/1885292415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa52006f680 0x7fa520073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 >> 172.21.15.135:0/1885292415 conn(0x7fa5200c89f0 msgr2=0x7fa5200680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 shutdown_connections 2024-04-26T22:00:15.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:15.792+0000 7fa524d11640 1 -- 172.21.15.135:0/1885292415 wait complete. 2024-04-26T22:00:15.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-04-26T22:00:15.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-26T22:00:15.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-04-26T22:00:15.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-04-26T22:00:15.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-04-26T22:00:15.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-04-26T22:00:16.721 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T22:00:16.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T22:00:16.802 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T22:00:16.807 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T22:00:16.808 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T22:00:16.810 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T22:00:16.810 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T22:00:16.813 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T22:00:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.839+0000 7f92bb559640 1 Processor -- start 2024-04-26T22:00:16.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.839+0000 7f92bb559640 1 -- start start 2024-04-26T22:00:16.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92bb559640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92b40c3c70 0x7f92b4074370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92bb559640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 0x7f92b406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92bb559640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b4072e60 con 0x7f92b4150e90 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92bb559640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f92b40710d0 con 0x7f92b40c3c70 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92bb559640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f92b406fc50 con 0x7f92b4070c80 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92b40c3c70 0x7f92b4074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92b40c3c70 0x7f92b4074370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33674/0 (socket says 172.21.15.135:33674) 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 -- 172.21.15.135:0/313552414 learned_addr learned my addr 172.21.15.135:0/313552414 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b92f5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f92b4150e90 0x7f92b406f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57368/0 (socket says 172.21.15.135:57368) 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 -- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 msgr2=0x7f92b406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 --2- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 0x7f92b406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 -- 172.21.15.135:0/313552414 >> v1:172.21.15.139:6789/0 conn(0x7f92b4150e90 legacy=0x7f92b406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.840+0000 7f92b8af4640 1 -- 172.21.15.135:0/313552414 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4154580 con 0x7f92b40c3c70 2024-04-26T22:00:16.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.841+0000 7f92b8af4640 1 --2- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 0x7f92b4074370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009870 tx=0x7f92a402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79d73c245bffd546 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.842+0000 7f92abfff640 1 -- 172.21.15.135:0/313552414 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4037070 con 0x7f92b40c3c70 2024-04-26T22:00:16.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.842+0000 7f92abfff640 1 -- 172.21.15.135:0/313552414 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a4032440 con 0x7f92b40c3c70 2024-04-26T22:00:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.842+0000 7f92abfff640 1 -- 172.21.15.135:0/313552414 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4032810 con 0x7f92b40c3c70 2024-04-26T22:00:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.843+0000 7f92bb559640 1 -- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 msgr2=0x7f92b4074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.843+0000 7f92bb559640 1 --2- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 0x7f92b4074370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009870 tx=0x7f92a402a130 comp rx=0 tx=0).stop 2024-04-26T22:00:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.844+0000 7f92bb559640 1 -- 172.21.15.135:0/313552414 shutdown_connections 2024-04-26T22:00:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.844+0000 7f92bb559640 1 --2- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 0x7f92b406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.844+0000 7f92bb559640 1 --2- 172.21.15.135:0/313552414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 0x7f92b4074370 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.844+0000 7f92bb559640 1 -- 172.21.15.135:0/313552414 >> 172.21.15.135:0/313552414 conn(0x7f92b40c5d70 msgr2=0x7f92b40c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.845+0000 7f92bb559640 1 -- 172.21.15.135:0/313552414 shutdown_connections 2024-04-26T22:00:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.845+0000 7f92bb559640 1 -- 172.21.15.135:0/313552414 wait complete. 2024-04-26T22:00:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.847+0000 7f92bb559640 1 Processor -- start 2024-04-26T22:00:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.847+0000 7f92bb559640 1 -- start start 2024-04-26T22:00:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.848+0000 7f92bb559640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 0x7f92b41f0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.848+0000 7f92bb559640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 0x7f92b4143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.848+0000 7f92bb559640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 0x7f92b41572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.848+0000 7f92bb559640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4154230 con 0x7f92b4150e90 2024-04-26T22:00:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.848+0000 7f92bb559640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4072af0 con 0x7f92b40c3c70 2024-04-26T22:00:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.848+0000 7f92bb559640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f92b4151270 con 0x7f92b4070c80 2024-04-26T22:00:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.849+0000 7f92b9af6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 0x7f92b41572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.849+0000 7f92b9af6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 0x7f92b41572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40706/0 (socket says 172.21.15.135:40706) 2024-04-26T22:00:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.849+0000 7f92b9af6640 1 -- 172.21.15.135:0/2731316850 learned_addr learned my addr 172.21.15.135:0/2731316850 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92b9af6640 1 -- 172.21.15.135:0/2731316850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 msgr2=0x7f92b41f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92b9af6640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 0x7f92b41f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92b9af6640 1 -- 172.21.15.135:0/2731316850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 msgr2=0x7f92b4143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92b9af6640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 0x7f92b4143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92b9af6640 1 -- 172.21.15.135:0/2731316850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41ea4e0 con 0x7f92b4150e90 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92b9af6640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 0x7f92b41572e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b9a0 tx=0x7f92ac00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac019070 con 0x7f92b4150e90 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac00cbc0 con 0x7f92b4150e90 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.850+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac014440 con 0x7f92b4150e90 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.851+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41467e0 con 0x7f92b4150e90 2024-04-26T22:00:16.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.851+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b4146ac0 con 0x7f92b4150e90 2024-04-26T22:00:16.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.853+0000 7f92977fe640 1 -- 172.21.15.135:0/2731316850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b41f09f0 con 0x7f92b4150e90 2024-04-26T22:00:16.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.854+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92ac01c070 con 0x7f92b4150e90 2024-04-26T22:00:16.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.855+0000 7f92a9ffb640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f929806c9c0 0x7f929806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.855+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f92ac08bb80 con 0x7f92b4150e90 2024-04-26T22:00:16.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.856+0000 7f92b92f5640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f929806c9c0 0x7f929806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.857+0000 7f92b92f5640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f929806c9c0 0x7f929806ee80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92b0000f60 tx=0x7f92b0007a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.863+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92ac059cf0 con 0x7f92b4150e90 2024-04-26T22:00:16.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.913+0000 7f134ba46640 1 Processor -- start 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.913+0000 7f134ba46640 1 -- start start 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.914+0000 7f134ba46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344071450 0x7f13441589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.914+0000 7f134ba46640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1344155060 0x7f13441554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.914+0000 7f134ba46640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1344071060 con 0x7f1344151ff0 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.914+0000 7f134ba46640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1344154bd0 con 0x7f1344155060 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.914+0000 7f134ba46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13441523d0 con 0x7f1344071450 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.914+0000 7f1349fe3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1344155060 0x7f13441554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1349fe3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1344155060 0x7f13441554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40720/0 (socket says 172.21.15.135:40720) 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1349fe3640 1 -- 172.21.15.135:0/3424913180 learned_addr learned my addr 172.21.15.135:0/3424913180 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1348fe1640 1 --2- 172.21.15.135:0/3424913180 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344071450 0x7f13441589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1349fe3640 1 -- 172.21.15.135:0/3424913180 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344071450 msgr2=0x7f13441589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1349fe3640 1 --2- 172.21.15.135:0/3424913180 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344071450 0x7f13441589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1349fe3640 1 -- 172.21.15.135:0/3424913180 >> v1:172.21.15.135:6789/0 conn(0x7f1344151ff0 legacy=0x7f13440c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.915+0000 7f1349fe3640 1 -- 172.21.15.135:0/3424913180 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13440707c0 con 0x7f1344155060 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.917+0000 7f1349fe3640 1 --2- 172.21.15.135:0/3424913180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13441554f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f133c009e50 tx=0x7f133c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f11b8178c53e076 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.917+0000 7f133bfff640 1 -- 172.21.15.135:0/3424913180 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c036070 con 0x7f1344155060 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.917+0000 7f133bfff640 1 -- 172.21.15.135:0/3424913180 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c031600 con 0x7f1344155060 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.917+0000 7f133bfff640 1 -- 172.21.15.135:0/3424913180 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c0319d0 con 0x7f1344155060 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.918+0000 7f134ba46640 1 -- 172.21.15.135:0/3424913180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 msgr2=0x7f13441554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.918+0000 7f134ba46640 1 --2- 172.21.15.135:0/3424913180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13441554f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f133c009e50 tx=0x7f133c02a6c0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.918+0000 7f134ba46640 1 -- 172.21.15.135:0/3424913180 shutdown_connections 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.919+0000 7f134ba46640 1 --2- 172.21.15.135:0/3424913180 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344071450 0x7f13441589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.919+0000 7f134ba46640 1 --2- 172.21.15.135:0/3424913180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13441554f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.919+0000 7f134ba46640 1 -- 172.21.15.135:0/3424913180 >> 172.21.15.135:0/3424913180 conn(0x7f13440ca3d0 msgr2=0x7f13440ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.920+0000 7f134ba46640 1 -- 172.21.15.135:0/3424913180 shutdown_connections 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.920+0000 7f134ba46640 1 -- 172.21.15.135:0/3424913180 wait complete. 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.922+0000 7f134ba46640 1 Processor -- start 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.922+0000 7f134ba46640 1 -- start start 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.923+0000 7f134ba46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344071450 0x7f1344156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.923+0000 7f134ba46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344151ff0 0x7f134406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.923+0000 7f134ba46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13440c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.923+0000 7f1348fe1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344151ff0 0x7f134406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1348fe1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344151ff0 0x7f134406d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34168/0 (socket says 172.21.15.135:34168) 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13440c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13440c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40728/0 (socket says 172.21.15.135:40728) 2024-04-26T22:00:16.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1348fe1640 1 -- 172.21.15.135:0/3429745847 learned_addr learned my addr 172.21.15.135:0/3429745847 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13440c54e0 con 0x7f1344155060 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1344070d80 con 0x7f1344071450 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13440c5660 con 0x7f1344151ff0 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 -- 172.21.15.135:0/3429745847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344151ff0 msgr2=0x7f134406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344151ff0 0x7f134406d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 -- 172.21.15.135:0/3429745847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344071450 msgr2=0x7f1344156370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344071450 0x7f1344156370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.924+0000 7f1349fe3640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13440c9fb0 con 0x7f1344155060 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.925+0000 7f1349fe3640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13440c8bb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f133c030000 tx=0x7f133c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.926+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c041020 con 0x7f1344155060 2024-04-26T22:00:16.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.926+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13441462b0 con 0x7f1344155060 2024-04-26T22:00:16.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.927+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134406de60 con 0x7f1344155060 2024-04-26T22:00:16.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.929+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c008a30 con 0x7f1344155060 2024-04-26T22:00:16.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.930+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c036040 con 0x7f1344155060 2024-04-26T22:00:16.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.930+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f133c044020 con 0x7f1344155060 2024-04-26T22:00:16.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.930+0000 7f1339ffb640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f132806ca20 0x7f132806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.931+0000 7f13497e2640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f132806ca20 0x7f132806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.932+0000 7f13497e2640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f132806ca20 0x7f132806eee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1334000f60 tx=0x7f13340079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.932+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f133c0acee0 con 0x7f1344155060 2024-04-26T22:00:16.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.932+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134406cd20 con 0x7f1344155060 2024-04-26T22:00:16.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.941+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f133c07b110 con 0x7f1344155060 2024-04-26T22:00:16.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.969+0000 7fdaec693640 1 Processor -- start 2024-04-26T22:00:16.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.969+0000 7f02c2b80640 1 Processor -- start 2024-04-26T22:00:16.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.969+0000 7f02c2b80640 1 -- start start 2024-04-26T22:00:16.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f5386d78640 1 Processor -- start 2024-04-26T22:00:16.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f5386d78640 1 -- start start 2024-04-26T22:00:16.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.969+0000 7fdaec693640 1 -- start start 2024-04-26T22:00:16.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7fdaec693640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdae40c5080 0x7fdae40c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7fdaec693640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7fdaec693640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdae40710a0 con 0x7fdae406f830 2024-04-26T22:00:16.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7fdaec693640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdae40c40e0 con 0x7fdae40c5080 2024-04-26T22:00:16.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7fdaec693640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdae4070dc0 con 0x7fdae415b110 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f02c2b80640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f02b40d7ad0 0x7f02b4046150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f02c2b80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b40db800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f02c2b80640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b4007080 con 0x7f02b40dbd40 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f02c2b80640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f02b4007340 con 0x7f02b40d7ad0 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.970+0000 7f02c2b80640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f02b40da420 con 0x7f02b4046690 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c1b7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f02b40d7ad0 0x7f02b4046150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c1b7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f02b40d7ad0 0x7f02b4046150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33684/0 (socket says 172.21.15.135:33684) 2024-04-26T22:00:16.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3739616789 learned_addr learned my addr 172.21.15.135:0/3739616789 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7fdaea42f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdae40c5080 0x7fdae40c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5386d78640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5380156e50 0x7f5380152b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5386d78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c3f20 0x7f5380153070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5386d78640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53800c4400 con 0x7f53800c5720 2024-04-26T22:00:16.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5386d78640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5380156120 con 0x7f5380156e50 2024-04-26T22:00:16.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5386d78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f53801569c0 con 0x7f53800c3f20 2024-04-26T22:00:16.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c137d640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b40db800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5384b14640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f53800c5720 0x7f538014fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57392/0 (socket says 172.21.15.135:57392) 2024-04-26T22:00:16.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f5384b14640 1 -- 172.21.15.135:0/648785260 learned_addr learned my addr 172.21.15.135:0/648785260 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:16.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 msgr2=0x7f02b40db800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:16.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c1b7e640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b40db800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3739616789 >> v1:172.21.15.135:6789/0 conn(0x7f02b40dbd40 legacy=0x7f02b40dd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:16.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7fdaea42f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdae40c5080 0x7fdae40c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33700/0 (socket says 172.21.15.135:33700) 2024-04-26T22:00:16.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f5385315640 1 --2- 172.21.15.135:0/648785260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c3f20 0x7f5380153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.971+0000 7fdaea42f640 1 -- 172.21.15.135:0/2674009638 learned_addr learned my addr 172.21.15.135:0/2674009638 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f537ffff640 1 --2- 172.21.15.135:0/648785260 >> v2:172.21.15.139:3300/0 conn(0x7f5380156e50 0x7f5380152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7fdae942d640 1 -- 172.21.15.135:0/2674009638 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650050260 0 0) 0x7fdae40710a0 con 0x7fdae406f830 2024-04-26T22:00:16.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f5385315640 1 -- 172.21.15.135:0/648785260 >> v2:172.21.15.139:3300/0 conn(0x7f5380156e50 msgr2=0x7f5380152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:16.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f5385315640 1 --2- 172.21.15.135:0/648785260 >> v2:172.21.15.139:3300/0 conn(0x7f5380156e50 0x7f5380152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f5385315640 1 -- 172.21.15.135:0/648785260 >> v1:172.21.15.139:6789/0 conn(0x7f53800c5720 legacy=0x7f538014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:16.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f5385315640 1 -- 172.21.15.135:0/648785260 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5380156370 con 0x7f53800c3f20 2024-04-26T22:00:16.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3739616789 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b40da700 con 0x7f02b40d7ad0 2024-04-26T22:00:16.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7fdaea42f640 1 -- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 msgr2=0x7fdae406d320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:16.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.973+0000 7f02c137d640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b40db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:16.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7fdaea42f640 1 --2- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae406d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7fdaea42f640 1 -- 172.21.15.135:0/2674009638 >> v1:172.21.15.139:6789/0 conn(0x7fdae406f830 legacy=0x7fdae415ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.973+0000 7f02c1b7e640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40d7ad0 0x7f02b4046150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e599ed5122f2ca10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.972+0000 7fdaea42f640 1 -- 172.21.15.135:0/2674009638 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae415e790 con 0x7fdae40c5080 2024-04-26T22:00:16.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.973+0000 7f02c0b7c640 1 -- 172.21.15.135:0/3739616789 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8037070 con 0x7f02b40d7ad0 2024-04-26T22:00:16.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.973+0000 7f02c0b7c640 1 -- 172.21.15.135:0/3739616789 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8032440 con 0x7f02b40d7ad0 2024-04-26T22:00:16.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.973+0000 7f02c0b7c640 1 -- 172.21.15.135:0/3739616789 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8032810 con 0x7f02b40d7ad0 2024-04-26T22:00:16.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.973+0000 7f5385315640 1 --2- 172.21.15.135:0/648785260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c3f20 0x7f5380153070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5378009d90 tx=0x7f537802a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47863562da1e25ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7fdaea42f640 1 --2- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae40c42c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae002a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958fe192dc07d273 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7fdae942d640 1 -- 172.21.15.135:0/2674009638 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0037070 con 0x7fdae40c5080 2024-04-26T22:00:16.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7fdae942d640 1 -- 172.21.15.135:0/2674009638 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0032440 con 0x7fdae40c5080 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 -- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40d7ad0 msgr2=0x7f02b4046150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40d7ad0 0x7f02b4046150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7fdaec693640 1 -- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 msgr2=0x7fdae40c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 -- 172.21.15.135:0/3739616789 shutdown_connections 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b40db800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3739616789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40d7ad0 0x7f02b4046150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 -- 172.21.15.135:0/3739616789 >> 172.21.15.135:0/3739616789 conn(0x7f02b4005e90 msgr2=0x7f02b40062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 -- 172.21.15.135:0/3739616789 shutdown_connections 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f02c2b80640 1 -- 172.21.15.135:0/3739616789 wait complete. 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f537f7fe640 1 -- 172.21.15.135:0/648785260 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378007740 con 0x7f53800c3f20 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.974+0000 7f537f7fe640 1 -- 172.21.15.135:0/648785260 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53780078e0 con 0x7f53800c3f20 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7f5386d78640 1 -- 172.21.15.135:0/648785260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c3f20 msgr2=0x7f5380153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7f5386d78640 1 --2- 172.21.15.135:0/648785260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c3f20 0x7f5380153070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5378009d90 tx=0x7f537802a630 comp rx=0 tx=0).stop 2024-04-26T22:00:16.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7fdaec693640 1 --2- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae40c42c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae002a1f0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7fdaec693640 1 -- 172.21.15.135:0/2674009638 shutdown_connections 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7fdaec693640 1 --2- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7fdaec693640 1 --2- 172.21.15.135:0/2674009638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae40c42c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7fdaec693640 1 -- 172.21.15.135:0/2674009638 >> 172.21.15.135:0/2674009638 conn(0x7fdae40c5d70 msgr2=0x7fdae40c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7f5386d78640 1 -- 172.21.15.135:0/648785260 shutdown_connections 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7f5386d78640 1 --2- 172.21.15.135:0/648785260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c3f20 0x7f5380153070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7f5386d78640 1 --2- 172.21.15.135:0/648785260 >> v2:172.21.15.139:3300/0 conn(0x7f5380156e50 0x7f5380152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.975+0000 7f5386d78640 1 -- 172.21.15.135:0/648785260 >> 172.21.15.135:0/648785260 conn(0x7f53800c89b0 msgr2=0x7f53800c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:16.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7f5386d78640 1 -- 172.21.15.135:0/648785260 shutdown_connections 2024-04-26T22:00:16.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7f02c2b80640 1 Processor -- start 2024-04-26T22:00:16.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7f02c2b80640 1 -- start start 2024-04-26T22:00:16.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.976+0000 7f5386d78640 1 -- 172.21.15.135:0/648785260 wait complete. 2024-04-26T22:00:16.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.977+0000 7fdaec693640 1 -- 172.21.15.135:0/2674009638 shutdown_connections 2024-04-26T22:00:16.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.977+0000 7f5386d78640 1 Processor -- start 2024-04-26T22:00:16.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.977+0000 7f02c2b80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b4174100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.977+0000 7f02c2b80640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02b40d7ad0 0x7f02b40de250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.977+0000 7f5386d78640 1 -- start start 2024-04-26T22:00:16.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c2b80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40dbd40 0x7f02b4177d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c2b80640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f02b40dc1d0 con 0x7f02b40d7ad0 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c2b80640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f02b40dacc0 con 0x7f02b40dbd40 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c2b80640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f02b40d7eb0 con 0x7f02b4046690 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c1b7e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b4174100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c1b7e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b4174100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34206/0 (socket says 172.21.15.135:34206) 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3540374686 learned_addr learned my addr 172.21.15.135:0/3540374686 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c137d640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02b40d7ad0 0x7f02b40de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.977+0000 7fdaec693640 1 -- 172.21.15.135:0/2674009638 wait complete. 2024-04-26T22:00:16.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7fdaec693640 1 Processor -- start 2024-04-26T22:00:16.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40dbd40 msgr2=0x7f02b4177d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c1b7e640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40dbd40 0x7f02b4177d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.978+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02b40d7ad0 msgr2=0x7f02b40de250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02c1b7e640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02b40d7ad0 0x7f02b40de250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02c1b7e640 1 -- 172.21.15.135:0/3540374686 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b40dca70 con 0x7f02b4046690 2024-04-26T22:00:16.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02c137d640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02b40d7ad0 0x7f02b40de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:16.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02c1b7e640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b4174100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7fdaec693640 1 -- start start 2024-04-26T22:00:16.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8037040 con 0x7f02b4046690 2024-04-26T22:00:16.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8042070 con 0x7f02b4046690 2024-04-26T22:00:16.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8030a40 con 0x7f02b4046690 2024-04-26T22:00:16.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.979+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b40dc4d0 con 0x7f02b4046690 2024-04-26T22:00:16.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b4177840 con 0x7f02b4046690 2024-04-26T22:00:16.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7fdaec693640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdae406f830 0x7fdae40ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7fdaec693640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7fdaec693640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae40c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f5386d78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 0x7f5380155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7fdaec693640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdae415b5c0 con 0x7fdae406f830 2024-04-26T22:00:16.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7fdaec693640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdae40c3d90 con 0x7fdae40c5080 2024-04-26T22:00:16.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.980+0000 7fdaec693640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdae415b740 con 0x7fdae415b110 2024-04-26T22:00:16.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f5386d78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5380156e50 0x7f53800c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f5384b14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 0x7f5380155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f5384b14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 0x7f5380155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34208/0 (socket says 172.21.15.135:34208) 2024-04-26T22:00:16.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f5384b14640 1 -- 172.21.15.135:0/1373663243 learned_addr learned my addr 172.21.15.135:0/1373663243 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53800c71b0 0x7f53801e0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b4005390 con 0x7f02b4046690 2024-04-26T22:00:16.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.981+0000 7f537ffff640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5380156e50 0x7f53800c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7f5385315640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53800c71b0 0x7f53801e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53800c3aa0 con 0x7f53800c71b0 2024-04-26T22:00:16.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdae9c2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdae9c2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae414b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33738/0 (socket says 172.21.15.135:33738) 2024-04-26T22:00:16.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdae9c2e640 1 -- 172.21.15.135:0/3859436242 learned_addr learned my addr 172.21.15.135:0/3859436242 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:16.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaeac30640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae40c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53800ae1e0 con 0x7f5380156e50 2024-04-26T22:00:16.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaea42f640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdae406f830 0x7fdae40ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5380150440 con 0x7f53800c5720 2024-04-26T22:00:16.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaeac30640 1 -- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 msgr2=0x7fdae414b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaeac30640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae414b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaeac30640 1 -- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdae406f830 msgr2=0x7fdae40ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaeac30640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdae406f830 0x7fdae40ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaeac30640 1 -- 172.21.15.135:0/3859436242 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae4147bb0 con 0x7fdae415b110 2024-04-26T22:00:16.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02b8030be0 con 0x7f02b4046690 2024-04-26T22:00:16.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7fdaea42f640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdae406f830 0x7fdae40ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:16.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.982+0000 7f5384b14640 1 -- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5380156e50 msgr2=0x7f53800c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f5384b14640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5380156e50 0x7f53800c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f5384b14640 1 -- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53800c71b0 msgr2=0x7f53801e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:16.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f5384b14640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53800c71b0 0x7f53801e0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:16.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f5384b14640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53800abe60 con 0x7f53800c5720 2024-04-26T22:00:16.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f02b27fc640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02a006c8f0 0x7f02a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:16.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f02b80ac890 con 0x7f02b4046690 2024-04-26T22:00:16.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f5384b14640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 0x7f5380155310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f537000c460 tx=0x7f537000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7fdaeac30640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae40c85a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdad400b440 tx=0x7fdad400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.983+0000 7f02c137d640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02a006c8f0 0x7f02a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:16.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.984+0000 7f02c137d640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02a006c8f0 0x7f02a006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02bc051470 tx=0x7f02bc06bca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:16.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.987+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370005ce0 con 0x7f53800c5720 2024-04-26T22:00:16.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.987+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5370007070 con 0x7f53800c5720 2024-04-26T22:00:16.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.987+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02b807a330 con 0x7f02b4046690 2024-04-26T22:00:16.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.988+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370007460 con 0x7f53800c5720 2024-04-26T22:00:17.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.988+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4015020 con 0x7fdae415b110 2024-04-26T22:00:17.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.988+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae4148300 con 0x7fdae415b110 2024-04-26T22:00:17.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.988+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae40c7a40 con 0x7fdae415b110 2024-04-26T22:00:17.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.989+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad4011830 con 0x7fdae415b110 2024-04-26T22:00:17.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.989+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4011c20 con 0x7fdae415b110 2024-04-26T22:00:17.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.989+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53801526d0 con 0x7f53800c5720 2024-04-26T22:00:17.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.989+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5380151f30 con 0x7f53800c5720 2024-04-26T22:00:17.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.991+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdad4018020 con 0x7fdae415b110 2024-04-26T22:00:17.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.991+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5380076100 con 0x7f53800c5720 2024-04-26T22:00:17.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.991+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc005290 con 0x7fdae415b110 2024-04-26T22:00:17.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.992+0000 7fdadaffd640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdac806c9c0 0x7fdac806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.993+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fdad408bb50 con 0x7fdae415b110 2024-04-26T22:00:17.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.993+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5370007600 con 0x7f53800c5720 2024-04-26T22:00:17.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.993+0000 7f537d7fa640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f536c06c940 0x7f536c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.994+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f537008bc40 con 0x7f53800c5720 2024-04-26T22:00:17.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.993+0000 7fdaea42f640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdac806c9c0 0x7fdac806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.994+0000 7fdaea42f640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdac806c9c0 0x7fdac806ee80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdae0034000 tx=0x7fdae002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.994+0000 7f537ffff640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f536c06c940 0x7f536c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.994+0000 7f537ffff640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f536c06c940 0x7f536c06ee00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f53800c29d0 tx=0x7f537400d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.997+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdad401f020 con 0x7fdae415b110 2024-04-26T22:00:17.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.999+0000 7fab6e4a0640 1 Processor -- start 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:16.999+0000 7fab6e4a0640 1 -- start start 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6e4a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab68150e90 0x7fab68074370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6e4a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fab680c3c70 0x7fab68070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6e4a0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab68072e60 con 0x7fab6806f540 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6e4a0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fab68154580 con 0x7fab680c3c70 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6e4a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fab68072810 con 0x7fab68150e90 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab677fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fab680c3c70 0x7fab68070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6ca3d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fab6806f540 0x7fab6806d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53060/0 (socket says 172.21.15.135:53060) 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab6ca3d640 1 -- 172.21.15.135:0/1485491026 learned_addr learned my addr 172.21.15.135:0/1485491026 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:17.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7fab67fff640 1 --2- 172.21.15.135:0/1485491026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab68150e90 0x7fab68074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.001+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5370059760 con 0x7f53800c5720 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.002+0000 7fab66ffd640 1 -- 172.21.15.135:0/1485491026 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034442179 0 0) 0x7fab68072e60 con 0x7fab6806f540 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.002+0000 7fab66ffd640 1 -- 172.21.15.135:0/1485491026 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab50003520 con 0x7fab6806f540 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.002+0000 7fab677fe640 1 -- 172.21.15.135:0/1485491026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab68150e90 msgr2=0x7fab68074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.002+0000 7fab677fe640 1 --2- 172.21.15.135:0/1485491026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab68150e90 0x7fab68074370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.002+0000 7fab677fe640 1 -- 172.21.15.135:0/1485491026 >> v1:172.21.15.135:6789/0 conn(0x7fab6806f540 legacy=0x7fab6806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.002+0000 7fab677fe640 1 -- 172.21.15.135:0/1485491026 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab68072af0 con 0x7fab680c3c70 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.003+0000 7fab677fe640 1 --2- 172.21.15.135:0/1485491026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68070c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0097a0 tx=0x7fab5c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6922b0dea48aafe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.005+0000 7fcbcb12f640 1 Processor -- start 2024-04-26T22:00:17.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.005+0000 7fcbcb12f640 1 -- start start 2024-04-26T22:00:17.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.004+0000 7fab66ffd640 1 -- 172.21.15.135:0/1485491026 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c037070 con 0x7fab680c3c70 2024-04-26T22:00:17.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fcbcb12f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc046150 0x7fcbbc0f5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbcb12f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcbbc007330 0x7fcbbc00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbcb12f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbbc0468e0 con 0x7fcbbc006ed0 2024-04-26T22:00:17.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbcb12f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc006a40 con 0x7fcbbc007330 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbcb12f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc0f8ac0 con 0x7fcbbc046150 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbc992c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fcbbc006ed0 0x7fcbbc0f9420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53070/0 (socket says 172.21.15.135:53070) 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbc992c640 1 -- 172.21.15.135:0/1122906496 learned_addr learned my addr 172.21.15.135:0/1122906496 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863994587 0 0) 0x7fcbbc0468e0 con 0x7fcbbc006ed0 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.007+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbb0003520 con 0x7fcbbc006ed0 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2425201283 0 0) 0x7fcbb0003520 con 0x7fcbbc006ed0 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc046150 msgr2=0x7fcbbc0f5830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fcbc912b640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc046150 0x7fcbbc0f5830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 >> v2:172.21.15.139:3300/0 conn(0x7fcbbc007330 msgr2=0x7fcbbc00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fcbc912b640 1 --2- 172.21.15.135:0/1122906496 >> v2:172.21.15.139:3300/0 conn(0x7fcbbc007330 0x7fcbbc00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc0f8da0 con 0x7fcbbc006ed0 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fcbc0001670 con 0x7fcbbc006ed0 2024-04-26T22:00:17.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 >> v1:172.21.15.135:6789/0 conn(0x7fcbbc006ed0 legacy=0x7fcbbc0f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb0003a30 0x7fcbb0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbb0004800 0x7fcbb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0005290 0x7fcbb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc006a40 con 0x7fcbb0003a30 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc0f8ac0 con 0x7fcbb0005290 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb000c230 con 0x7fcbb0004800 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.004+0000 7fab66ffd640 1 -- 172.21.15.135:0/1485491026 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c02adb0 con 0x7fab680c3c70 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.005+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1485491026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 msgr2=0x7fab68070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.005+0000 7fab66ffd640 1 -- 172.21.15.135:0/1485491026 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c037070 con 0x7fab680c3c70 2024-04-26T22:00:17.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.005+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1485491026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68070c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0097a0 tx=0x7fab5c02a060 comp rx=0 tx=0).stop 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1485491026 shutdown_connections 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1485491026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68070c80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1485491026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab68150e90 0x7fab68074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1485491026 >> 172.21.15.135:0/1485491026 conn(0x7fab680c5d70 msgr2=0x7fab680c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1485491026 shutdown_connections 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.006+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1485491026 wait complete. 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fab6e4a0640 1 Processor -- start 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbc992c640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbb0004800 0x7fcbb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fcbca12d640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb0003a30 0x7fcbb0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.008+0000 7fab6e4a0640 1 -- start start 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.009+0000 7fab6e4a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 0x7fab68146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab6e4a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab6e4a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab68150e90 0x7fab680c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab6e4a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806dbb0 con 0x7fab680c3c70 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab6e4a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab6806da30 con 0x7fab68150e90 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab6e4a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fab68151270 con 0x7fab6806f540 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 0x7fab68146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 0x7fab68146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34254/0 (socket says 172.21.15.135:34254) 2024-04-26T22:00:17.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 -- 172.21.15.135:0/1112638308 learned_addr learned my addr 172.21.15.135:0/1112638308 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:17.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab6ca3d640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab68150e90 0x7fab680c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab677fe640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fcbca92e640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0005290 0x7fcbb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fcbca12d640 1 -- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbb0004800 msgr2=0x7fcbb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:17.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fcbca12d640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbb0004800 0x7fcbb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 -- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab68150e90 msgr2=0x7fab680c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fcbca12d640 1 -- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0005290 msgr2=0x7fcbb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab68150e90 0x7fab680c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 -- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 msgr2=0x7fab68145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab67fff640 1 -- 172.21.15.135:0/1112638308 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab68155910 con 0x7fab6806f540 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.010+0000 7fab677fe640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.011+0000 7fcbca12d640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0005290 0x7fcbb000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.011+0000 7fcbca12d640 1 -- 172.21.15.135:0/1122906496 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb0003520 con 0x7fcbb0003a30 2024-04-26T22:00:17.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fab67fff640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 0x7fab68146270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab5800e870 tx=0x7fab5800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fcbca92e640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbb0005290 0x7fcbb000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:17.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58017070 con 0x7fab6806f540 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fcbca12d640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb0003a30 0x7fcbb0003e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009960 tx=0x7fcbb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9957c8efbf354e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab68155bf0 con 0x7fab6806f540 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab68144b70 con 0x7fab6806f540 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5800c940 con 0x7fab6806f540 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.012+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5800cd30 con 0x7fab6806f540 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.013+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4037070 con 0x7fcbb0003a30 2024-04-26T22:00:17.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.013+0000 7fcbc912b640 1 -- 172.21.15.135:0/1122906496 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb402a970 con 0x7fcbb0003a30 2024-04-26T22:00:17.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.013+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab48005290 con 0x7fab6806f540 2024-04-26T22:00:17.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.014+0000 7fcbcb12f640 1 -- 172.21.15.135:0/1122906496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb0003a30 msgr2=0x7fcbb0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.015+0000 7fe4a49b5640 1 Processor -- start 2024-04-26T22:00:17.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.014+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/1122906496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb0003a30 0x7fcbb0003e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009960 tx=0x7fcbb0005830 comp rx=0 tx=0).stop 2024-04-26T22:00:17.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.014+0000 7fcbca92e640 1 -- 172.21.15.135:0/1122906496 reap_dead start 2024-04-26T22:00:17.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.014+0000 7fcbcb12f640 1 -- 172.21.15.135:0/1122906496 shutdown_connections 2024-04-26T22:00:17.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.014+0000 7fcbcb12f640 1 -- 172.21.15.135:0/1122906496 >> 172.21.15.135:0/1122906496 conn(0x7fcbbc005e90 msgr2=0x7fcbbc0062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.015+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab5801a070 con 0x7fab6806f540 2024-04-26T22:00:17.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.015+0000 7fab64ff9640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fab5406c9c0 0x7fab5406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.016+0000 7fab677fe640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fab5406c9c0 0x7fab5406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.017+0000 7fab677fe640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fab5406c9c0 0x7fab5406ee80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab5c034000 tx=0x7fab5c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.017+0000 7fe4a49b5640 1 -- start start 2024-04-26T22:00:17.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.017+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fab5808bfb0 con 0x7fab6806f540 2024-04-26T22:00:17.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.017+0000 7fcbcb12f640 1 -- 172.21.15.135:0/1122906496 shutdown_connections 2024-04-26T22:00:17.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.017+0000 7fcbcb12f640 1 -- 172.21.15.135:0/1122906496 wait complete. 2024-04-26T22:00:17.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe4a49b5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe498048340 0x7fe498048720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe4a49b5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe49804d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe4a49b5640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49804d6b0 con 0x7fe49804bfe0 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe4a49b5640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe49804cce0 con 0x7fe498048340 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe4a49b5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe4980dafd0 con 0x7fe49804b8c0 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe49ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe498048340 0x7fe498048720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe49ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe498048340 0x7fe498048720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40796/0 (socket says 172.21.15.135:40796) 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe49f59e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe49804bfe0 0x7fe49800be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53078/0 (socket says 172.21.15.135:53078) 2024-04-26T22:00:17.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.018+0000 7fe49f59e640 1 -- 172.21.15.135:0/583488343 learned_addr learned my addr 172.21.15.135:0/583488343 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.019+0000 7fcbcb12f640 1 Processor -- start 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.019+0000 7fcbcb12f640 1 -- start start 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbcb12f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 0x7fcbbc0de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbcb12f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc0f9cb0 0x7fcbbc02c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab58059ad0 con 0x7fab6806f540 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.019+0000 7fe49ed9d640 1 -- 172.21.15.135:0/583488343 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 msgr2=0x7fe49804d170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:17.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.019+0000 7fe49ed9d640 1 --2- 172.21.15.135:0/583488343 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe49804d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.019+0000 7fe49ed9d640 1 -- 172.21.15.135:0/583488343 >> v1:172.21.15.135:6789/0 conn(0x7fe49804bfe0 legacy=0x7fe49800be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.019+0000 7fe49ed9d640 1 -- 172.21.15.135:0/583488343 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4980db2b0 con 0x7fe498048340 2024-04-26T22:00:17.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbcb12f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbbc02b270 0x7fcbbc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fe49ed9d640 1 --2- 172.21.15.135:0/583488343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe498048720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe4900097a0 tx=0x7fe49002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbad424669020994 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbcb12f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc009610 con 0x7fcbbc02c2f0 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbcb12f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc0f9b30 con 0x7fcbbc02b270 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbcb12f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc009790 con 0x7fcbbc0f9cb0 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbca12d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 0x7fcbbc0de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbc992c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc0f9cb0 0x7fcbbc02c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbca12d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 0x7fcbbc0de5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40800/0 (socket says 172.21.15.135:40800) 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbca12d640 1 -- 172.21.15.135:0/3767954382 learned_addr learned my addr 172.21.15.135:0/3767954382 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.020+0000 7fcbca92e640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbbc02b270 0x7fcbbc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fe49dd9b640 1 -- 172.21.15.135:0/583488343 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490037070 con 0x7fe498048340 2024-04-26T22:00:17.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca92e640 1 -- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc0f9cb0 msgr2=0x7fcbbc02c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca92e640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc0f9cb0 0x7fcbbc02c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca92e640 1 -- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 msgr2=0x7fcbbc0de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca92e640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 0x7fcbbc0de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca92e640 1 -- 172.21.15.135:0/3767954382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc042170 con 0x7fcbbc02b270 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca12d640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 0x7fcbbc0de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbca92e640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbbc02b270 0x7fcbbc199810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4075190 tx=0x7fcbc4069f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4073070 con 0x7fcbbc02b270 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.022+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc406d5b0 con 0x7fcbbc02b270 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.022+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc406d960 con 0x7fcbbc02b270 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.022+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc042410 con 0x7fcbbc02b270 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.022+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc0fa8a0 con 0x7fcbbc02b270 2024-04-26T22:00:17.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fe49dd9b640 1 -- 172.21.15.135:0/583488343 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49002add0 con 0x7fe498048340 2024-04-26T22:00:17.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.021+0000 7fe49dd9b640 1 -- 172.21.15.135:0/583488343 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490037070 con 0x7fe498048340 2024-04-26T22:00:17.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.023+0000 7fe4a49b5640 1 -- 172.21.15.135:0/583488343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 msgr2=0x7fe498048720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.023+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/583488343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe498048720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe4900097a0 tx=0x7fe49002a080 comp rx=0 tx=0).stop 2024-04-26T22:00:17.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.024+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbc406db00 con 0x7fcbbc02b270 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.024+0000 7fcbbaffd640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcbb006e280 0x7fcbb0070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.024+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fcbc40f7380 con 0x7fcbbc02b270 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fcbca12d640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcbb006e280 0x7fcbb0070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc005390 con 0x7fcbbc02b270 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fe4a49b5640 1 -- 172.21.15.135:0/583488343 shutdown_connections 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/583488343 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe49804d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/583488343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe498048720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fe4a49b5640 1 -- 172.21.15.135:0/583488343 >> 172.21.15.135:0/583488343 conn(0x7fe49802f4b0 msgr2=0x7fe49802f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fe4a49b5640 1 -- 172.21.15.135:0/583488343 shutdown_connections 2024-04-26T22:00:17.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.025+0000 7fe4a49b5640 1 -- 172.21.15.135:0/583488343 wait complete. 2024-04-26T22:00:17.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.026+0000 7fcbca12d640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcbb006e280 0x7fcbb0070740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009620 tx=0x7fcbb402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.027+0000 7fe4a49b5640 1 Processor -- start 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.028+0000 7fe4a49b5640 1 -- start start 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe4a49b5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe4980d8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe49ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe4980d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe49ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe4980d8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40804/0 (socket says 172.21.15.135:40804) 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe4a49b5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe4980cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe49ed9d640 1 -- 172.21.15.135:0/4028527132 learned_addr learned my addr 172.21.15.135:0/4028527132 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49804bfe0 0x7fe498170390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe49804c620 con 0x7fe498048340 2024-04-26T22:00:17.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980d7dc0 con 0x7fe49804bfe0 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.029+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe498030300 con 0x7fe49804b8c0 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49f59e640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49804bfe0 0x7fe498170390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49e59c640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe4980cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49ed9d640 1 -- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 msgr2=0x7fe4980cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49ed9d640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe4980cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49ed9d640 1 -- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49804bfe0 msgr2=0x7fe498170390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49ed9d640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49804bfe0 0x7fe498170390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49ed9d640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4980da3f0 con 0x7fe498048340 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.030+0000 7fe49ed9d640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe4980d8880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4900097a0 tx=0x7fe4900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.031+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbc406a000 con 0x7fcbbc02b270 2024-04-26T22:00:17.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.031+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490037050 con 0x7fe498048340 2024-04-26T22:00:17.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.031+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4980da6d0 con 0x7fe498048340 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.033+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4980d4560 con 0x7fe498048340 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.033+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe490038040 con 0x7fe498048340 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.034+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49003e670 con 0x7fe498048340 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.034+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe49003e810 con 0x7fe498048340 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.035+0000 7fe4877fe640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe48c06c940 0x7fe48c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.035+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe4900ac500 con 0x7fe498048340 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.036+0000 7fe49e59c640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe48c06c940 0x7fe48c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.036+0000 7fe49e59c640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe48c06c940 0x7fe48c06ee00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0051470 tx=0x7fe4a006b9a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.037+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe480005290 con 0x7fe498048340 2024-04-26T22:00:17.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.046+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe49007a670 con 0x7fe498048340 2024-04-26T22:00:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.251+0000 7f92977fe640 1 -- 172.21.15.135:0/2731316850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f92b4141e90 con 0x7f92b4150e90 2024-04-26T22:00:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.252+0000 7f92a9ffb640 1 -- 172.21.15.135:0/2731316850 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92b4141e90 con 0x7f92b4150e90 2024-04-26T22:00:17.252 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-26T22:00:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f929806c9c0 msgr2=0x7f929806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f929806c9c0 0x7f929806ee80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92b0000f60 tx=0x7f92b0007a00 comp rx=0 tx=0).stop 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 msgr2=0x7f92b41572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 0x7f92b41572e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b9a0 tx=0x7f92ac00be60 comp rx=0 tx=0).stop 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 shutdown_connections 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f929806c9c0 0x7f929806ee80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92b4150e90 0x7f92b41572e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92b40c3c70 0x7f92b4143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.255+0000 7f92bb559640 1 --2- 172.21.15.135:0/2731316850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92b4070c80 0x7f92b41f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.256+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 >> 172.21.15.135:0/2731316850 conn(0x7f92b40c5d70 msgr2=0x7f92b406c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.256+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 shutdown_connections 2024-04-26T22:00:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.256+0000 7f92bb559640 1 -- 172.21.15.135:0/2731316850 wait complete. 2024-04-26T22:00:17.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-26T22:00:17.272 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.319+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f134414fab0 con 0x7f1344155060 2024-04-26T22:00:17.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.320+0000 7f1339ffb640 1 -- 172.21.15.135:0/3429745847 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f133c07aa10 con 0x7f1344155060 2024-04-26T22:00:17.320 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:17.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f132806ca20 msgr2=0x7f132806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f132806ca20 0x7f132806eee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1334000f60 tx=0x7f13340079b0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 msgr2=0x7f13440c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13440c8bb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f133c030000 tx=0x7f133c004a40 comp rx=0 tx=0).stop 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 shutdown_connections 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f132806ca20 0x7f132806eee0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1344155060 0x7f13440c8bb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1344151ff0 0x7f134406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.322+0000 7f134ba46640 1 --2- 172.21.15.135:0/3429745847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344071450 0x7f1344156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.323+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 >> 172.21.15.135:0/3429745847 conn(0x7f13440ca3d0 msgr2=0x7f134414fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.323+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 shutdown_connections 2024-04-26T22:00:17.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.323+0000 7f134ba46640 1 -- 172.21.15.135:0/3429745847 wait complete. 2024-04-26T22:00:17.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-04-26T22:00:17.334 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.370+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f53800507a0 con 0x7f53800c5720 2024-04-26T22:00:17.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.371+0000 7f537d7fa640 1 -- 172.21.15.135:0/1373663243 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f537005da60 con 0x7f53800c5720 2024-04-26T22:00:17.371 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f536c06c940 msgr2=0x7f536c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f536c06c940 0x7f536c06ee00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f53800c29d0 tx=0x7f537400d040 comp rx=0 tx=0).stop 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 msgr2=0x7f5380155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 0x7f5380155310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f537000c460 tx=0x7f537000c920 comp rx=0 tx=0).stop 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 shutdown_connections 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f536c06c940 0x7f536c06ee00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53800c71b0 0x7f53801e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5380156e50 0x7f53800c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 --2- 172.21.15.135:0/1373663243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53800c5720 0x7f5380155310 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 >> 172.21.15.135:0/1373663243 conn(0x7f53800c89b0 msgr2=0x7f538014f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 shutdown_connections 2024-04-26T22:00:17.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.374+0000 7f5386d78640 1 -- 172.21.15.135:0/1373663243 wait complete. 2024-04-26T22:00:17.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.377+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f02b4177a10 con 0x7f02b4046690 2024-04-26T22:00:17.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.378+0000 7f02b27fc640 1 -- 172.21.15.135:0/3540374686 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02b807e630 con 0x7f02b4046690 2024-04-26T22:00:17.377 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02a006c8f0 msgr2=0x7f02a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02a006c8f0 0x7f02a006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02bc051470 tx=0x7f02bc06bca0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 msgr2=0x7f02b4174100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b4174100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b80046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 shutdown_connections 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02a006c8f0 0x7f02a006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02b40dbd40 0x7f02b4177d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02b40d7ad0 0x7f02b40de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 --2- 172.21.15.135:0/3540374686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02b4046690 0x7f02b4174100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 >> 172.21.15.135:0/3540374686 conn(0x7f02b4005e90 msgr2=0x7f02b4009dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 shutdown_connections 2024-04-26T22:00:17.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.380+0000 7f02c2b80640 1 -- 172.21.15.135:0/3540374686 wait complete. 2024-04-26T22:00:17.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-26T22:00:17.391 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-26T22:00:17.392 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.399+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fab48005580 con 0x7fab6806f540 2024-04-26T22:00:17.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.399+0000 7fab64ff9640 1 -- 172.21.15.135:0/1112638308 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab5805ddd0 con 0x7fab6806f540 2024-04-26T22:00:17.399 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:17.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fab5406c9c0 msgr2=0x7fab5406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fab5406c9c0 0x7fab5406ee80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab5c034000 tx=0x7fab5c02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:17.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 msgr2=0x7fab68146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 0x7fab68146270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab5800e870 tx=0x7fab5800ed30 comp rx=0 tx=0).stop 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 shutdown_connections 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fab5406c9c0 0x7fab5406ee80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab68150e90 0x7fab680c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab680c3c70 0x7fab68145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 --2- 172.21.15.135:0/1112638308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6806f540 0x7fab68146270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 >> 172.21.15.135:0/1112638308 conn(0x7fab680c5d70 msgr2=0x7fab680c6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 shutdown_connections 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.401+0000 7fab6e4a0640 1 -- 172.21.15.135:0/1112638308 wait complete. 2024-04-26T22:00:17.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.403+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fdacc005580 con 0x7fdae415b110 2024-04-26T22:00:17.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.404+0000 7fdadaffd640 1 -- 172.21.15.135:0/3859436242 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdad4059cc0 con 0x7fdae415b110 2024-04-26T22:00:17.403 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdac806c9c0 msgr2=0x7fdac806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdac806c9c0 0x7fdac806ee80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdae0034000 tx=0x7fdae002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 msgr2=0x7fdae40c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae40c85a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdad400b440 tx=0x7fdad400b900 comp rx=0 tx=0).stop 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 shutdown_connections 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdac806c9c0 0x7fdac806ee80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae415b110 0x7fdae40c85a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae40c5080 0x7fdae414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 --2- 172.21.15.135:0/3859436242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdae406f830 0x7fdae40ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 >> 172.21.15.135:0/3859436242 conn(0x7fdae40c5d70 msgr2=0x7fdae406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 shutdown_connections 2024-04-26T22:00:17.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.407+0000 7fdaec693640 1 -- 172.21.15.135:0/3859436242 wait complete. 2024-04-26T22:00:17.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-26T22:00:17.413 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.416+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcbbc0f8790 con 0x7fcbbc02b270 2024-04-26T22:00:17.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.417+0000 7fcbbaffd640 1 -- 172.21.15.135:0/3767954382 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcbc40c5460 con 0x7fcbbc02b270 2024-04-26T22:00:17.417 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-26T22:00:17.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcbb006e280 msgr2=0x7fcbb0070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcbb006e280 0x7fcbb0070740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009620 tx=0x7fcbb402e000 comp rx=0 tx=0).stop 2024-04-26T22:00:17.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbbc02b270 msgr2=0x7fcbbc199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbbc02b270 0x7fcbbc199810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4075190 tx=0x7fcbc4069f90 comp rx=0 tx=0).stop 2024-04-26T22:00:17.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 shutdown_connections 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcbb006e280 0x7fcbb0070740 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbbc02b270 0x7fcbbc199810 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbbc0f9cb0 0x7fcbbc02c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 --2- 172.21.15.135:0/3767954382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbbc02c2f0 0x7fcbbc0de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 >> 172.21.15.135:0/3767954382 conn(0x7fcbbc005e90 msgr2=0x7fcbbc0ef360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 shutdown_connections 2024-04-26T22:00:17.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.419+0000 7fcbcb12f640 1 -- 172.21.15.135:0/3767954382 wait complete. 2024-04-26T22:00:17.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-04-26T22:00:17.422 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-26T22:00:17.430 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.436+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe480005580 con 0x7fe498048340 2024-04-26T22:00:17.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.436+0000 7fe4877fe640 1 -- 172.21.15.135:0/4028527132 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe490079f70 con 0x7fe498048340 2024-04-26T22:00:17.436 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe48c06c940 msgr2=0x7fe48c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe48c06c940 0x7fe48c06ee00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0051470 tx=0x7fe4a006b9a0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 msgr2=0x7fe4980d8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe4980d8880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4900097a0 tx=0x7fe4900046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 shutdown_connections 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe48c06c940 0x7fe48c06ee00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49804bfe0 0x7fe498170390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe49804b8c0 0x7fe4980cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 --2- 172.21.15.135:0/4028527132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe498048340 0x7fe4980d8880 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.438+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 >> 172.21.15.135:0/4028527132 conn(0x7fe49802f4b0 msgr2=0x7fe4980d4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.439+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 shutdown_connections 2024-04-26T22:00:17.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.439+0000 7fe4a49b5640 1 -- 172.21.15.135:0/4028527132 wait complete. 2024-04-26T22:00:17.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-04-26T22:00:17.449 DEBUG:teuthology.parallel:result is None 2024-04-26T22:00:17.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T22:00:17.449 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:17.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.587+0000 7f9584862640 1 Processor -- start 2024-04-26T22:00:17.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.587+0000 7f9584862640 1 -- start start 2024-04-26T22:00:17.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f9584862640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f95800aecf0 0x7f95800c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f9584862640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9580156f60 con 0x7f95800c3d40 2024-04-26T22:00:17.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f9584862640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9580157240 con 0x7f95800c4820 2024-04-26T22:00:17.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f9584862640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9580157520 con 0x7f95800aecf0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957e59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f95800aecf0 0x7f95800c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957e59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f95800aecf0 0x7f95800c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40818/0 (socket says 172.21.15.135:40818) 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957e59c640 1 -- 172.21.15.135:0/1221003586 learned_addr learned my addr 172.21.15.135:0/1221003586 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957ed9d640 1 --1- 172.21.15.135:0/1221003586 >> v1:172.21.15.135:6789/0 conn(0x7f95800c3d40 0x7f9580074180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53080/0 (socket says 172.21.15.135:53080) 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957e59c640 1 -- 172.21.15.135:0/1221003586 >> v1:172.21.15.139:6789/0 conn(0x7f95800c4820 legacy=0x7f9580073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957e59c640 1 -- 172.21.15.135:0/1221003586 >> v1:172.21.15.135:6789/0 conn(0x7f95800c3d40 legacy=0x7f9580074180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957e59c640 1 -- 172.21.15.135:0/1221003586 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9580157800 con 0x7f95800aecf0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.588+0000 7f957dd9b640 1 -- 172.21.15.135:0/1221003586 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 289027336 0 0) 0x7f9580156f60 con 0x7f95800c3d40 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f957e59c640 1 --2- 172.21.15.135:0/1221003586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800aecf0 0x7f95800c3800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9570009870 tx=0x7f957002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af321c1f48fce70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f957dd9b640 1 -- 172.21.15.135:0/1221003586 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037070 con 0x7f95800aecf0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f957dd9b640 1 -- 172.21.15.135:0/1221003586 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570032440 con 0x7f95800aecf0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f957dd9b640 1 -- 172.21.15.135:0/1221003586 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570032810 con 0x7f95800aecf0 2024-04-26T22:00:17.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 -- 172.21.15.135:0/1221003586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800aecf0 msgr2=0x7f95800c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 --2- 172.21.15.135:0/1221003586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800aecf0 0x7f95800c3800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9570009870 tx=0x7f957002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 -- 172.21.15.135:0/1221003586 shutdown_connections 2024-04-26T22:00:17.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 --2- 172.21.15.135:0/1221003586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800aecf0 0x7f95800c3800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 -- 172.21.15.135:0/1221003586 >> 172.21.15.135:0/1221003586 conn(0x7f95800c6ed0 msgr2=0x7f95800c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 -- 172.21.15.135:0/1221003586 shutdown_connections 2024-04-26T22:00:17.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.589+0000 7f9584862640 1 -- 172.21.15.135:0/1221003586 wait complete. 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 Processor -- start 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 -- start start 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95800aecf0 0x7f958014d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 0x7f95801592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95800c4820 0x7f95800c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95800af0d0 con 0x7f95800c3d40 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9580156c10 con 0x7f95800c4820 2024-04-26T22:00:17.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9584862640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f95800af250 con 0x7f95800aecf0 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9577fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 0x7f95801592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9577fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 0x7f95801592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40824/0 (socket says 172.21.15.135:40824) 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.591+0000 7f9577fff640 1 -- 172.21.15.135:0/906982108 learned_addr learned my addr 172.21.15.135:0/906982108 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f957e59c640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95800aecf0 0x7f958014d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f957ed9d640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95800c4820 0x7f95800c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9577fff640 1 -- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95800aecf0 msgr2=0x7f958014d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9577fff640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95800aecf0 0x7f958014d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9577fff640 1 -- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95800c4820 msgr2=0x7f95800c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9577fff640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95800c4820 0x7f95800c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9577fff640 1 -- 172.21.15.135:0/906982108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9580158250 con 0x7f95800c3d40 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f957e59c640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95800aecf0 0x7f958014d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f957ed9d640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95800c4820 0x7f95800c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9577fff640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 0x7f95801592f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f956800c400 tx=0x7f956800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568017070 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95680052d0 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568003830 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9580158530 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.592+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9580158af0 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.593+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95800507a0 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.593+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f956801a070 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.594+0000 7f9576ffd640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f956006c8f0 0x7f956006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.594+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f956808b0d0 con 0x7f95800c3d40 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.594+0000 7f957e59c640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f956006c8f0 0x7f956006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.594+0000 7f957e59c640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f956006c8f0 0x7f956006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9570034000 tx=0x7f957002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.598+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9568059260 con 0x7f95800c3d40 2024-04-26T22:00:17.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.771+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95800c4600 con 0x7f956006c8f0 2024-04-26T22:00:17.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.781+0000 7f9576ffd640 1 -- 172.21.15.135:0/906982108 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490306 (secure 0 0 0) 0x7f95700b9b60 con 0x7f956006c8f0 2024-04-26T22:00:17.787 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:17.790 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f956006c8f0 msgr2=0x7f956006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f956006c8f0 0x7f956006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9570034000 tx=0x7f957002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 msgr2=0x7f95801592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 0x7f95801592f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f956800c400 tx=0x7f956800c8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 shutdown_connections 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f956006c8f0 0x7f956006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95800c4820 0x7f95800c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95800c3d40 0x7f95801592f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 --2- 172.21.15.135:0/906982108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95800aecf0 0x7f958014d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 >> 172.21.15.135:0/906982108 conn(0x7f95800c6ed0 msgr2=0x7f95800ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 shutdown_connections 2024-04-26T22:00:17.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.786+0000 7f9584862640 1 -- 172.21.15.135:0/906982108 wait complete. 2024-04-26T22:00:17.797 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-26T22:00:17.073288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002435"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809657+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:07.809657+0000","last_peered":"2024-04-26T22:00:07.809657+0000","last_clean":"2024-04-26T22:00:07.809657+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:07.809657+0000","last_undegraded":"2024-04-26T22:00:07.809657+0000","last_fullsized":"2024-04-26T22:00:07.809657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:17.804 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:17.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.936+0000 7f4b7605b640 1 Processor -- start 2024-04-26T22:00:17.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.936+0000 7f4b7605b640 1 -- start start 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b7605b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4b700c3060 0x7f4b700720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b7605b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b70154ba0 con 0x7f4b700c46e0 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b7605b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b70154e80 con 0x7f4b700c40e0 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b7605b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4b70155160 con 0x7f4b700c3060 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4b700c46e0 0x7f4b700c4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53096/0 (socket says 172.21.15.135:53096) 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6ffff640 1 -- 172.21.15.135:0/2565293349 learned_addr learned my addr 172.21.15.135:0/2565293349 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3895753112 0 0) 0x7f4b70154ba0 con 0x7f4b700c46e0 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b58003520 con 0x7f4b700c46e0 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6f7fe640 1 --2- 172.21.15.135:0/2565293349 >> v2:172.21.15.139:3300/0 conn(0x7f4b700c3060 0x7f4b700720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 891790777 0 0) 0x7f4b58003520 con 0x7f4b700c46e0 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 >> v2:172.21.15.139:3300/0 conn(0x7f4b700c3060 msgr2=0x7f4b700720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:17.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 --2- 172.21.15.135:0/2565293349 >> v2:172.21.15.139:3300/0 conn(0x7f4b700c3060 0x7f4b700720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 >> v1:172.21.15.139:6789/0 conn(0x7f4b700c40e0 legacy=0x7f4b70151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70155440 con 0x7f4b700c46e0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f4b680019f0 con 0x7f4b700c46e0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 >> v1:172.21.15.135:6789/0 conn(0x7f4b700c46e0 legacy=0x7f4b700c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580039b0 0x7f4b58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b580051c0 0x7f4b5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70155160 con 0x7f4b580051c0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5800c160 con 0x7f4b580039b0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.937+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5800c360 con 0x7f4b58004800 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b580051c0 0x7f4b5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6f7fe640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580039b0 0x7f4b58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6effd640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 -- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b58004800 msgr2=0x7f4b58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 -- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580039b0 msgr2=0x7f4b58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580039b0 0x7f4b58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 -- 172.21.15.135:0/2565293349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b58003520 con 0x7f4b580051c0 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6f7fe640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580039b0 0x7f4b58003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 -- 172.21.15.135:0/2565293349 reap_dead start 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.938+0000 7f4b6ffff640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b580051c0 0x7f4b5800b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800a850 tx=0x7f4b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c481af0caa13592b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60009690 con 0x7f4b580051c0 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6802f070 con 0x7f4b580051c0 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b6e7fc640 1 -- 172.21.15.135:0/2565293349 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800a4a0 con 0x7f4b580051c0 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 -- 172.21.15.135:0/2565293349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b580051c0 msgr2=0x7f4b5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b580051c0 0x7f4b5800b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800a850 tx=0x7f4b58005830 comp rx=0 tx=0).stop 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 -- 172.21.15.135:0/2565293349 shutdown_connections 2024-04-26T22:00:17.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 --2- 172.21.15.135:0/2565293349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b580051c0 0x7f4b5800b9c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 -- 172.21.15.135:0/2565293349 >> 172.21.15.135:0/2565293349 conn(0x7f4b7006c310 msgr2=0x7f4b7006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:17.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 -- 172.21.15.135:0/2565293349 shutdown_connections 2024-04-26T22:00:17.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.939+0000 7f4b7605b640 1 -- 172.21.15.135:0/2565293349 wait complete. 2024-04-26T22:00:17.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.940+0000 7f4b7605b640 1 Processor -- start 2024-04-26T22:00:17.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.940+0000 7f4b7605b640 1 -- start start 2024-04-26T22:00:17.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b7605b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580051c0 0x7f4b70150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b7605b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 0x7f4b70144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b7605b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b7014c330 0x7f4b701445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b7605b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700c3440 con 0x7f4b7014e660 2024-04-26T22:00:17.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b7605b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700c3aa0 con 0x7f4b580051c0 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b7605b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4b700c35c0 con 0x7f4b7014c330 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 0x7f4b70144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 0x7f4b70144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40842/0 (socket says 172.21.15.135:40842) 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 -- 172.21.15.135:0/3164316147 learned_addr learned my addr 172.21.15.135:0/3164316147 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6ffff640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b7014c330 0x7f4b701445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6f7fe640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580051c0 0x7f4b70150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 -- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b7014c330 msgr2=0x7f4b701445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b7014c330 0x7f4b701445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 -- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580051c0 msgr2=0x7f4b70150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580051c0 0x7f4b70150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.941+0000 7f4b6effd640 1 -- 172.21.15.135:0/3164316147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70144d50 con 0x7f4b7014e660 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.942+0000 7f4b6effd640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 0x7f4b70144090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400f650 tx=0x7f4b6400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.942+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640050d0 con 0x7f4b7014e660 2024-04-26T22:00:17.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.942+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b64018070 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.942+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b70154260 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.942+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64005490 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.942+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b701510b0 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.943+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700c5540 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.943+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b6400d6d0 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.944+0000 7f4b57fff640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4b5c06c920 0x7f4b5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.944+0000 7f4b6f7fe640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4b5c06c920 0x7f4b5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.944+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f4b6408ba80 con 0x7f4b7014e660 2024-04-26T22:00:17.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.944+0000 7f4b6f7fe640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4b5c06c920 0x7f4b5c06ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009960 tx=0x7f4b60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:17.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:17.948+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b64059c10 con 0x7f4b7014e660 2024-04-26T22:00:18.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.122+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b700507a0 con 0x7f4b5c06c920 2024-04-26T22:00:18.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.132+0000 7f4b57fff640 1 -- 172.21.15.135:0/3164316147 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490306 (secure 0 0 0) 0x7f4b700507a0 con 0x7f4b5c06c920 2024-04-26T22:00:18.137 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:18.140 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4b5c06c920 msgr2=0x7f4b5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4b5c06c920 0x7f4b5c06ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009960 tx=0x7f4b60008040 comp rx=0 tx=0).stop 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 msgr2=0x7f4b70144090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 0x7f4b70144090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400f650 tx=0x7f4b6400fb10 comp rx=0 tx=0).stop 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 shutdown_connections 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4b5c06c920 0x7f4b5c06ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b7014c330 0x7f4b701445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b7014e660 0x7f4b70144090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 --2- 172.21.15.135:0/3164316147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b580051c0 0x7f4b70150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 >> 172.21.15.135:0/3164316147 conn(0x7f4b7006c310 msgr2=0x7f4b701519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 shutdown_connections 2024-04-26T22:00:18.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.136+0000 7f4b7605b640 1 -- 172.21.15.135:0/3164316147 wait complete. 2024-04-26T22:00:18.147 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-26T22:00:17.073288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002435"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809657+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:07.809657+0000","last_peered":"2024-04-26T22:00:07.809657+0000","last_clean":"2024-04-26T22:00:07.809657+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:07.809657+0000","last_undegraded":"2024-04-26T22:00:07.809657+0000","last_fullsized":"2024-04-26T22:00:07.809657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:18.154 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T22:00:18.154 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T22:00:18.154 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T22:00:18.155 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T22:00:18.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.283+0000 7f7497f89640 1 Processor -- start 2024-04-26T22:00:18.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.283+0000 7f7497f89640 1 -- start start 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7497f89640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f74900c5050 0x7f74901576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7497f89640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490156c10 con 0x7f749013feb0 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7497f89640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74900c5570 con 0x7f74900c2b80 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7497f89640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f749015a8b0 con 0x7f74900c5050 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7495d25640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f749013feb0 0x7f749015a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:53104/0 (socket says 172.21.15.135:53104) 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7495d25640 1 -- 172.21.15.135:0/305006422 learned_addr learned my addr 172.21.15.135:0/305006422 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7495524640 1 --2- 172.21.15.135:0/305006422 >> v2:172.21.15.135:3300/0 conn(0x7f74900c5050 0x7f74901576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7495524640 1 -- 172.21.15.135:0/305006422 >> v1:172.21.15.139:6789/0 conn(0x7f74900c2b80 legacy=0x7f749006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7495524640 1 -- 172.21.15.135:0/305006422 >> v1:172.21.15.135:6789/0 conn(0x7f749013feb0 legacy=0x7f749015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.284+0000 7f7495524640 1 -- 172.21.15.135:0/305006422 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74900c5850 con 0x7f74900c5050 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.285+0000 7f7495524640 1 --2- 172.21.15.135:0/305006422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74900c5050 0x7f74901576d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74800097a0 tx=0x7f748002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b455882fe5a626 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.285+0000 7f7494d23640 1 -- 172.21.15.135:0/305006422 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480037070 con 0x7f74900c5050 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.285+0000 7f7494d23640 1 -- 172.21.15.135:0/305006422 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7480032440 con 0x7f74900c5050 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.285+0000 7f7494d23640 1 -- 172.21.15.135:0/305006422 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748002ae60 con 0x7f74900c5050 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 -- 172.21.15.135:0/305006422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74900c5050 msgr2=0x7f74901576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 --2- 172.21.15.135:0/305006422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74900c5050 0x7f74901576d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74800097a0 tx=0x7f748002a110 comp rx=0 tx=0).stop 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 -- 172.21.15.135:0/305006422 shutdown_connections 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 --2- 172.21.15.135:0/305006422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74900c5050 0x7f74901576d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 -- 172.21.15.135:0/305006422 >> 172.21.15.135:0/305006422 conn(0x7f74900c5ce0 msgr2=0x7f74900c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 -- 172.21.15.135:0/305006422 shutdown_connections 2024-04-26T22:00:18.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.286+0000 7f7497f89640 1 -- 172.21.15.135:0/305006422 wait complete. 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 Processor -- start 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 -- start start 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74900c2b80 0x7f7490159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74900c5050 0x7f74900ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 0x7f74900c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f749006da30 con 0x7f749013feb0 2024-04-26T22:00:18.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7490157240 con 0x7f74900c2b80 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.287+0000 7f7497f89640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f749006dbb0 con 0x7f74900c5050 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 0x7f74900c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 0x7f74900c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40852/0 (socket says 172.21.15.135:40852) 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 -- 172.21.15.135:0/3125030507 learned_addr learned my addr 172.21.15.135:0/3125030507 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7495d25640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74900c2b80 0x7f7490159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7495524640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74900c5050 0x7f74900ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 -- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74900c5050 msgr2=0x7f74900ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74900c5050 0x7f74900ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 -- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74900c2b80 msgr2=0x7f7490159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74900c2b80 0x7f7490159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 -- 172.21.15.135:0/3125030507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74900ac9f0 con 0x7f749013feb0 2024-04-26T22:00:18.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7495d25640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74900c2b80 0x7f7490159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7496526640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 0x7f74900c75a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f748800c6b0 tx=0x7f748800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488003b70 con 0x7f749013feb0 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7488003d30 con 0x7f749013feb0 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74900acfb0 con 0x7f749013feb0 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488005310 con 0x7f749013feb0 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.288+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7490159da0 con 0x7f749013feb0 2024-04-26T22:00:18.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.289+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74901593d0 con 0x7f749013feb0 2024-04-26T22:00:18.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.290+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74880054b0 con 0x7f749013feb0 2024-04-26T22:00:18.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.290+0000 7f74867fc640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747406c8f0 0x7f747406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.290+0000 7f7495d25640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747406c8f0 0x7f747406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.290+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f748808bce0 con 0x7f749013feb0 2024-04-26T22:00:18.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.290+0000 7f7495d25640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747406c8f0 0x7f747406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f748c005880 tx=0x7f748c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.294+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7488059de0 con 0x7f749013feb0 2024-04-26T22:00:18.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.508+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f74900507a0 con 0x7f749013feb0 2024-04-26T22:00:18.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.508+0000 7f74867fc640 1 -- 172.21.15.135:0/3125030507 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f74880596e0 con 0x7f749013feb0 2024-04-26T22:00:18.508 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:18.508 INFO:teuthology.orchestra.run.smithi135.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747406c8f0 msgr2=0x7f747406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747406c8f0 0x7f747406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f748c005880 tx=0x7f748c009040 comp rx=0 tx=0).stop 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 msgr2=0x7f74900c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 0x7f74900c75a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f748800c6b0 tx=0x7f748800cb70 comp rx=0 tx=0).stop 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 shutdown_connections 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747406c8f0 0x7f747406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f749013feb0 0x7f74900c75a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74900c5050 0x7f74900ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 --2- 172.21.15.135:0/3125030507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74900c2b80 0x7f7490159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 >> 172.21.15.135:0/3125030507 conn(0x7f74900c5ce0 msgr2=0x7f749006cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 shutdown_connections 2024-04-26T22:00:18.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.510+0000 7f7497f89640 1 -- 172.21.15.135:0/3125030507 wait complete. 2024-04-26T22:00:18.520 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T22:00:18.521 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:18.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.618+0000 7fd64bc2a640 1 Processor -- start 2024-04-26T22:00:18.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.618+0000 7fd64bc2a640 1 -- start start 2024-04-26T22:00:18.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd64bc2a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440c6e20 0x7fd6440c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd64bc2a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd64415a530 con 0x7fd6440740e0 2024-04-26T22:00:18.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd64bc2a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd64415a810 con 0x7fd6440ca3a0 2024-04-26T22:00:18.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd64bc2a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd644072930 con 0x7fd6440c6e20 2024-04-26T22:00:18.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6491c5640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd6440740e0 0x7fd644156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46266/0 (socket says 172.21.15.135:46266) 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6491c5640 1 -- 172.21.15.135:0/3255889632 learned_addr learned my addr 172.21.15.135:0/3255889632 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6499c6640 1 --2- 172.21.15.135:0/3255889632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440c6e20 0x7fd6440c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3666805724 0 0) 0x7fd64415a530 con 0x7fd6440740e0 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd630003520 con 0x7fd6440740e0 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600601093 0 0) 0x7fd64415a810 con 0x7fd6440ca3a0 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd64415a530 con 0x7fd6440ca3a0 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6499c6640 1 -- 172.21.15.135:0/3255889632 >> v1:172.21.15.139:6789/0 conn(0x7fd6440ca3a0 legacy=0x7fd644161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6499c6640 1 -- 172.21.15.135:0/3255889632 >> v1:172.21.15.135:6789/0 conn(0x7fd6440740e0 legacy=0x7fd644156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.619+0000 7fd6499c6640 1 -- 172.21.15.135:0/3255889632 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64415aaf0 con 0x7fd6440c6e20 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.620+0000 7fd6499c6640 1 --2- 172.21.15.135:0/3255889632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440c6e20 0x7fd6440c7200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6400097d0 tx=0x7fd64002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135fd59423fece4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.620+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640037070 con 0x7fd6440c6e20 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.620+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd640032440 con 0x7fd6440c6e20 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.620+0000 7fd6489c4640 1 -- 172.21.15.135:0/3255889632 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640032810 con 0x7fd6440c6e20 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.620+0000 7fd64bc2a640 1 -- 172.21.15.135:0/3255889632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440c6e20 msgr2=0x7fd6440c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.620+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/3255889632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440c6e20 0x7fd6440c7200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6400097d0 tx=0x7fd64002a180 comp rx=0 tx=0).stop 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.621+0000 7fd64bc2a640 1 -- 172.21.15.135:0/3255889632 shutdown_connections 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.621+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/3255889632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440c6e20 0x7fd6440c7200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.621+0000 7fd64bc2a640 1 -- 172.21.15.135:0/3255889632 >> 172.21.15.135:0/3255889632 conn(0x7fd6440ae0b0 msgr2=0x7fd6440ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.621+0000 7fd64bc2a640 1 -- 172.21.15.135:0/3255889632 shutdown_connections 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.621+0000 7fd64bc2a640 1 -- 172.21.15.135:0/3255889632 wait complete. 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 Processor -- start 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 -- start start 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440740e0 0x7fd644073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 0x7fd644149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440ca3a0 0x7fd64414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd644154540 con 0x7fd6440c6e20 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441543c0 con 0x7fd6440740e0 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd64bc2a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd64415a0c0 con 0x7fd6440ca3a0 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd6491c5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 0x7fd644149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd6491c5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 0x7fd644149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57896/0 (socket says 172.21.15.135:57896) 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.622+0000 7fd6491c5640 1 -- 172.21.15.135:0/2645467725 learned_addr learned my addr 172.21.15.135:0/2645467725 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd64a1c7640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440ca3a0 0x7fd64414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6499c6640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440740e0 0x7fd644073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6491c5640 1 -- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440ca3a0 msgr2=0x7fd64414a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6491c5640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440ca3a0 0x7fd64414a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6491c5640 1 -- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440740e0 msgr2=0x7fd644073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6491c5640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440740e0 0x7fd644073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6491c5640 1 -- 172.21.15.135:0/2645467725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6440ad280 con 0x7fd6440c6e20 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd6491c5640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 0x7fd644149ca0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c410 tx=0x7fd63400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634017070 con 0x7fd6440c6e20 2024-04-26T22:00:18.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd634005ce0 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6340072d0 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64414a720 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.623+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd644158ae0 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.624+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6440766c0 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.624+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd63401a070 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.625+0000 7fd63a7fc640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd62806c8f0 0x7fd62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.625+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd63408b0c0 con 0x7fd6440c6e20 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.625+0000 7fd6499c6640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd62806c8f0 0x7fd62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.625+0000 7fd6499c6640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd62806c8f0 0x7fd62806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd640034000 tx=0x7fd64002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.629+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd634058c00 con 0x7fd6440c6e20 2024-04-26T22:00:18.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.805+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6440507a0 con 0x7fd6440c6e20 2024-04-26T22:00:18.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.807+0000 7fd63a7fc640 1 -- 172.21.15.135:0/2645467725 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7fd63405cf00 con 0x7fd6440c6e20 2024-04-26T22:00:18.807 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:18.807 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.808+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd62806c8f0 msgr2=0x7fd62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.808+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd62806c8f0 0x7fd62806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd640034000 tx=0x7fd64002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.808+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 msgr2=0x7fd644149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.808+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 0x7fd644149ca0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c410 tx=0x7fd63400c8d0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 shutdown_connections 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd62806c8f0 0x7fd62806edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6440ca3a0 0x7fd64414a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6440c6e20 0x7fd644149ca0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 --2- 172.21.15.135:0/2645467725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440740e0 0x7fd644073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 >> 172.21.15.135:0/2645467725 conn(0x7fd6440ae0b0 msgr2=0x7fd6441514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 shutdown_connections 2024-04-26T22:00:18.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.809+0000 7fd64bc2a640 1 -- 172.21.15.135:0/2645467725 wait complete. 2024-04-26T22:00:18.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:18.820 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:18.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.921+0000 7fa3d94dc640 1 Processor -- start 2024-04-26T22:00:18.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.921+0000 7fa3d94dc640 1 -- start start 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d94dc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa3d415a280 0x7fa3d40c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d94dc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa3d4158180 0x7fa3d41585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d94dc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d40c8a30 con 0x7fa3d413ffb0 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d94dc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d4072be0 con 0x7fa3d4158180 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d94dc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d40c9640 con 0x7fa3d415a280 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa3d4158180 0x7fa3d41585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa3d4158180 0x7fa3d41585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57900/0 (socket says 172.21.15.135:57900) 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d37fe640 1 -- 172.21.15.135:0/475406131 learned_addr learned my addr 172.21.15.135:0/475406131 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.922+0000 7fa3d27fc640 1 --2- 172.21.15.135:0/475406131 >> v2:172.21.15.139:3300/0 conn(0x7fa3d415a280 0x7fa3d40c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d37fe640 1 -- 172.21.15.135:0/475406131 >> v2:172.21.15.139:3300/0 conn(0x7fa3d415a280 msgr2=0x7fa3d40c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d37fe640 1 --2- 172.21.15.135:0/475406131 >> v2:172.21.15.139:3300/0 conn(0x7fa3d415a280 0x7fa3d40c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d37fe640 1 -- 172.21.15.135:0/475406131 >> v1:172.21.15.139:6789/0 conn(0x7fa3d413ffb0 legacy=0x7fa3d4072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d37fe640 1 -- 172.21.15.135:0/475406131 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d40c8d10 con 0x7fa3d4158180 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d37fe640 1 --2- 172.21.15.135:0/475406131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d4158180 0x7fa3d41585d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406d2f0 tx=0x7fa3cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1084c89abdd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d1ffb640 1 -- 172.21.15.135:0/475406131 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc036070 con 0x7fa3d4158180 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d1ffb640 1 -- 172.21.15.135:0/475406131 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc031600 con 0x7fa3d4158180 2024-04-26T22:00:18.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.923+0000 7fa3d1ffb640 1 -- 172.21.15.135:0/475406131 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0319d0 con 0x7fa3d4158180 2024-04-26T22:00:18.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 -- 172.21.15.135:0/475406131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d4158180 msgr2=0x7fa3d41585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/475406131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d4158180 0x7fa3d41585d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406d2f0 tx=0x7fa3cc02a6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 -- 172.21.15.135:0/475406131 shutdown_connections 2024-04-26T22:00:18.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/475406131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d4158180 0x7fa3d41585d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/475406131 >> v2:172.21.15.139:3300/0 conn(0x7fa3d415a280 0x7fa3d40c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 -- 172.21.15.135:0/475406131 >> 172.21.15.135:0/475406131 conn(0x7fa3d406dc10 msgr2=0x7fa3d406c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:18.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 -- 172.21.15.135:0/475406131 shutdown_connections 2024-04-26T22:00:18.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.924+0000 7fa3d94dc640 1 -- 172.21.15.135:0/475406131 wait complete. 2024-04-26T22:00:18.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.925+0000 7fa3d94dc640 1 Processor -- start 2024-04-26T22:00:18.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.925+0000 7fa3d94dc640 1 -- start start 2024-04-26T22:00:18.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d94dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 0x7fa3d41e2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d94dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3d4158180 0x7fa3d40ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d94dc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3d415a280 0x7fa3d41e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d94dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d40c92d0 con 0x7fa3d413ffb0 2024-04-26T22:00:18.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d94dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d40c87e0 con 0x7fa3d4158180 2024-04-26T22:00:18.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d94dc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d4073700 con 0x7fa3d415a280 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 0x7fa3d41e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 0x7fa3d41e2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57904/0 (socket says 172.21.15.135:57904) 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 -- 172.21.15.135:0/1576264697 learned_addr learned my addr 172.21.15.135:0/1576264697 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d37fe640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3d415a280 0x7fa3d41e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d27fc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3d4158180 0x7fa3d40ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 -- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3d415a280 msgr2=0x7fa3d41e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3d415a280 0x7fa3d41e3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 -- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3d4158180 msgr2=0x7fa3d40ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3d4158180 0x7fa3d40ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.926+0000 7fa3d2ffd640 1 -- 172.21.15.135:0/1576264697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d40c6a20 con 0x7fa3d413ffb0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.927+0000 7fa3d2ffd640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 0x7fa3d41e2520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c400 tx=0x7fa3c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.927+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40052d0 con 0x7fa3d413ffb0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.927+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c4005490 con 0x7fa3d413ffb0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.927+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4007260 con 0x7fa3d413ffb0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.927+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d41e0540 con 0x7fa3d413ffb0 2024-04-26T22:00:18.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.927+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d41e09f0 con 0x7fa3d413ffb0 2024-04-26T22:00:18.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.928+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d4076650 con 0x7fa3d413ffb0 2024-04-26T22:00:18.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.928+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3c4007400 con 0x7fa3d413ffb0 2024-04-26T22:00:18.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.928+0000 7fa3bb7fe640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa3c006c8f0 0x7fa3c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:18.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.929+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fa3c408b360 con 0x7fa3d413ffb0 2024-04-26T22:00:18.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.929+0000 7fa3d27fc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa3c006c8f0 0x7fa3c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:18.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.929+0000 7fa3d27fc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa3c006c8f0 0x7fa3c006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:18.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:18.933+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3c4058e10 con 0x7fa3d413ffb0 2024-04-26T22:00:19.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.108+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3d40507a0 con 0x7fa3d413ffb0 2024-04-26T22:00:19.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.109+0000 7fa3bb7fe640 1 -- 172.21.15.135:0/1576264697 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fa3c405d110 con 0x7fa3d413ffb0 2024-04-26T22:00:19.109 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:19.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa3c006c8f0 msgr2=0x7fa3c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa3c006c8f0 0x7fa3c006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8008040 comp rx=0 tx=0).stop 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 msgr2=0x7fa3d41e2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 0x7fa3d41e2520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c400 tx=0x7fa3c400c8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 shutdown_connections 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa3c006c8f0 0x7fa3c006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3d415a280 0x7fa3d41e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3d4158180 0x7fa3d40ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 --2- 172.21.15.135:0/1576264697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3d413ffb0 0x7fa3d41e2520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 >> 172.21.15.135:0/1576264697 conn(0x7fa3d406dc10 msgr2=0x7fa3d40cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 shutdown_connections 2024-04-26T22:00:19.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.111+0000 7fa3d94dc640 1 -- 172.21.15.135:0/1576264697 wait complete. 2024-04-26T22:00:19.122 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:19.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.222+0000 7f44128bd640 1 Processor -- start 2024-04-26T22:00:19.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.222+0000 7f44128bd640 1 -- start start 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f44128bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0acc30 0x7f440c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f44128bd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f440c0c4250 0x7f440c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f44128bd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f440c0ad010 con 0x7f440c0c3cd0 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f44128bd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f440c0c8ba0 con 0x7f440c0c4250 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f44128bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f440c156e50 con 0x7f440c0acc30 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f440c0c4250 0x7f440c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f440c0c4250 0x7f440c0c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57910/0 (socket says 172.21.15.135:57910) 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 -- 172.21.15.135:0/2286033457 learned_addr learned my addr 172.21.15.135:0/2286033457 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440bfff640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0acc30 0x7f440c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 -- 172.21.15.135:0/2286033457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0acc30 msgr2=0x7f440c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0acc30 0x7f440c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 -- 172.21.15.135:0/2286033457 >> v1:172.21.15.139:6789/0 conn(0x7f440c0c3cd0 legacy=0x7f440c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:19.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440b7fe640 1 -- 172.21.15.135:0/2286033457 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c15b130 con 0x7f440c0c4250 2024-04-26T22:00:19.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.223+0000 7f440bfff640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0acc30 0x7f440c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:19.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f440b7fe640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c0c3790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009870 tx=0x7f43fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942b699f0b9b487d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f440affd640 1 -- 172.21.15.135:0/2286033457 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc037070 con 0x7f440c0c4250 2024-04-26T22:00:19.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f440affd640 1 -- 172.21.15.135:0/2286033457 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc032440 con 0x7f440c0c4250 2024-04-26T22:00:19.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f440affd640 1 -- 172.21.15.135:0/2286033457 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc032810 con 0x7f440c0c4250 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 -- 172.21.15.135:0/2286033457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 msgr2=0x7f440c0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c0c3790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009870 tx=0x7f43fc02a180 comp rx=0 tx=0).stop 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 -- 172.21.15.135:0/2286033457 shutdown_connections 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c0c3790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 --2- 172.21.15.135:0/2286033457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0acc30 0x7f440c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 -- 172.21.15.135:0/2286033457 >> 172.21.15.135:0/2286033457 conn(0x7f440c0c6e60 msgr2=0x7f440c0c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 -- 172.21.15.135:0/2286033457 shutdown_connections 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.224+0000 7f44128bd640 1 -- 172.21.15.135:0/2286033457 wait complete. 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.225+0000 7f44128bd640 1 Processor -- start 2024-04-26T22:00:19.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 -- start start 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440c0acc30 0x7f440c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0c3cd0 0x7f440c1ec630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c14b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0c88c0 con 0x7f440c0c4250 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0c85e0 con 0x7f440c0acc30 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f44128bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f440c0abec0 con 0x7f440c0c3cd0 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f4410e5a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c14b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f4410e5a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c14b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57912/0 (socket says 172.21.15.135:57912) 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.226+0000 7f4410e5a640 1 -- 172.21.15.135:0/2836992357 learned_addr learned my addr 172.21.15.135:0/2836992357 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f440b7fe640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0c3cd0 0x7f440c1ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f440bfff640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440c0acc30 0x7f440c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4410e5a640 1 -- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0c3cd0 msgr2=0x7f440c1ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4410e5a640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0c3cd0 0x7f440c1ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4410e5a640 1 -- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440c0acc30 msgr2=0x7f440c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4410e5a640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440c0acc30 0x7f440c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4410e5a640 1 -- 172.21.15.135:0/2836992357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c156790 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f440bfff640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440c0acc30 0x7f440c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4410e5a640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c14b2b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f440400c7b0 tx=0x7f440400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404004b60 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4404004d00 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440400aa90 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c158120 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.227+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c1519e0 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.228+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c153dd0 con 0x7f440c0c4250 2024-04-26T22:00:19.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.228+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f440400ac30 con 0x7f440c0c4250 2024-04-26T22:00:19.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.229+0000 7f4408ff9640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f43ec06c8f0 0x7f43ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.229+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f440408c020 con 0x7f440c0c4250 2024-04-26T22:00:19.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.229+0000 7f440bfff640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f43ec06c8f0 0x7f43ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.229+0000 7f440bfff640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f43ec06c8f0 0x7f43ec06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f43f400aa30 tx=0x7f43f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.233+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f440c153dd0 con 0x7f440c0c4250 2024-04-26T22:00:19.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.408+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f440c157bc0 con 0x7f440c0c4250 2024-04-26T22:00:19.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.408+0000 7f4408ff9640 1 -- 172.21.15.135:0/2836992357 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f4404020070 con 0x7f440c0c4250 2024-04-26T22:00:19.408 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:19.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f43ec06c8f0 msgr2=0x7f43ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f43ec06c8f0 0x7f43ec06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f43f400aa30 tx=0x7f43f4008040 comp rx=0 tx=0).stop 2024-04-26T22:00:19.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 msgr2=0x7f440c14b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c14b2b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f440400c7b0 tx=0x7f440400cc70 comp rx=0 tx=0).stop 2024-04-26T22:00:19.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 shutdown_connections 2024-04-26T22:00:19.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f43ec06c8f0 0x7f43ec06edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f440c0c4250 0x7f440c14b2b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f440c0c3cd0 0x7f440c1ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 --2- 172.21.15.135:0/2836992357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440c0acc30 0x7f440c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 >> 172.21.15.135:0/2836992357 conn(0x7f440c0c6e60 msgr2=0x7f440c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:19.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 shutdown_connections 2024-04-26T22:00:19.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.410+0000 7f44128bd640 1 -- 172.21.15.135:0/2836992357 wait complete. 2024-04-26T22:00:19.421 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:19.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.519+0000 7f5873259640 1 Processor -- start 2024-04-26T22:00:19.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.519+0000 7f5873259640 1 -- start start 2024-04-26T22:00:19.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5873259640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f586c13ffb0 0x7f586c0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5873259640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586c0c8a30 con 0x7f586c158180 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5873259640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586c0c8d10 con 0x7f586c15a280 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5873259640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f586c072be0 con 0x7f586c13ffb0 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5870ff5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f586c13ffb0 0x7f586c0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5870ff5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f586c13ffb0 0x7f586c0c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57916/0 (socket says 172.21.15.135:57916) 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f5870ff5640 1 -- 172.21.15.135:0/3486379308 learned_addr learned my addr 172.21.15.135:0/3486379308 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.520+0000 7f58717f6640 1 --1- 172.21.15.135:0/3486379308 >> v1:172.21.15.135:6789/0 conn(0x7f586c158180 0x7f586c161610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46280/0 (socket says 172.21.15.135:46280) 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f5870ff5640 1 -- 172.21.15.135:0/3486379308 >> v1:172.21.15.139:6789/0 conn(0x7f586c15a280 legacy=0x7f586c072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f5870ff5640 1 -- 172.21.15.135:0/3486379308 >> v1:172.21.15.135:6789/0 conn(0x7f586c158180 legacy=0x7f586c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f5870ff5640 1 -- 172.21.15.135:0/3486379308 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c0c8ff0 con 0x7f586c13ffb0 2024-04-26T22:00:19.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f5870ff5640 1 --2- 172.21.15.135:0/3486379308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c13ffb0 0x7f586c0c82a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f585c009870 tx=0x7f585c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e3e8ca8202439b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f586b7fe640 1 -- 172.21.15.135:0/3486379308 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c037070 con 0x7f586c13ffb0 2024-04-26T22:00:19.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f586b7fe640 1 -- 172.21.15.135:0/3486379308 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c032440 con 0x7f586c13ffb0 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.521+0000 7f586b7fe640 1 -- 172.21.15.135:0/3486379308 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c032810 con 0x7f586c13ffb0 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 -- 172.21.15.135:0/3486379308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c13ffb0 msgr2=0x7f586c0c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 --2- 172.21.15.135:0/3486379308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c13ffb0 0x7f586c0c82a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f585c009870 tx=0x7f585c02a210 comp rx=0 tx=0).stop 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 -- 172.21.15.135:0/3486379308 shutdown_connections 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 --2- 172.21.15.135:0/3486379308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c13ffb0 0x7f586c0c82a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 -- 172.21.15.135:0/3486379308 >> 172.21.15.135:0/3486379308 conn(0x7f586c06dc10 msgr2=0x7f586c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 -- 172.21.15.135:0/3486379308 shutdown_connections 2024-04-26T22:00:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.522+0000 7f5873259640 1 -- 172.21.15.135:0/3486379308 wait complete. 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.523+0000 7f5873259640 1 Processor -- start 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.523+0000 7f5873259640 1 -- start start 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5873259640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f586c13ffb0 0x7f586c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5873259640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 0x7f586c1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5873259640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f586c15a280 0x7f586c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5873259640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f586c0c87e0 con 0x7f586c158180 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5873259640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f586c0c9640 con 0x7f586c13ffb0 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5873259640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f586c073700 con 0x7f586c15a280 2024-04-26T22:00:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 0x7f586c1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 0x7f586c1351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57930/0 (socket says 172.21.15.135:57930) 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 -- 172.21.15.135:0/1582614870 learned_addr learned my addr 172.21.15.135:0/1582614870 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5870ff5640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f586c13ffb0 0x7f586c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f58717f6640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f586c15a280 0x7f586c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 -- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f586c15a280 msgr2=0x7f586c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f586c15a280 0x7f586c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 -- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f586c13ffb0 msgr2=0x7f586c153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f586c13ffb0 0x7f586c153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 -- 172.21.15.135:0/1582614870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c134920 con 0x7f586c158180 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f5870ff5640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f586c13ffb0 0x7f586c153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f58717f6640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f586c15a280 0x7f586c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.524+0000 7f586bfff640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 0x7f586c1351f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f586000b420 tx=0x7f586000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.525+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860011830 con 0x7f586c158180 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.525+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5860016070 con 0x7f586c158180 2024-04-26T22:00:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.525+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c139720 con 0x7f586c158180 2024-04-26T22:00:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.525+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860011c10 con 0x7f586c158180 2024-04-26T22:00:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.525+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c134be0 con 0x7f586c158180 2024-04-26T22:00:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.526+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c076650 con 0x7f586c158180 2024-04-26T22:00:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.526+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f586001b460 con 0x7f586c158180 2024-04-26T22:00:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.526+0000 7f58697fa640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f585806c8f0 0x7f585806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.526+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f586008b8b0 con 0x7f586c158180 2024-04-26T22:00:19.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.526+0000 7f5870ff5640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f585806c8f0 0x7f585806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.527+0000 7f5870ff5640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f585806c8f0 0x7f585806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f585c034000 tx=0x7f585c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.531+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5860020020 con 0x7f586c158180 2024-04-26T22:00:19.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.703+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f586c15a060 con 0x7f586c158180 2024-04-26T22:00:19.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.704+0000 7f58697fa640 1 -- 172.21.15.135:0/1582614870 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f5860059a40 con 0x7f586c158180 2024-04-26T22:00:19.704 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f585806c8f0 msgr2=0x7f585806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f585806c8f0 0x7f585806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f585c034000 tx=0x7f585c02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 msgr2=0x7f586c1351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 0x7f586c1351f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f586000b420 tx=0x7f586000b8e0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 shutdown_connections 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f585806c8f0 0x7f585806edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f586c15a280 0x7f586c152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f586c158180 0x7f586c1351f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 --2- 172.21.15.135:0/1582614870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f586c13ffb0 0x7f586c153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 >> 172.21.15.135:0/1582614870 conn(0x7f586c06dc10 msgr2=0x7f586c0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:19.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 shutdown_connections 2024-04-26T22:00:19.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.706+0000 7f5873259640 1 -- 172.21.15.135:0/1582614870 wait complete. 2024-04-26T22:00:19.716 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.816+0000 7f603fc25640 1 Processor -- start 2024-04-26T22:00:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.816+0000 7f603fc25640 1 -- start start 2024-04-26T22:00:19.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603fc25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 0x7f60380c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603fc25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f60380ca3b0 0x7f6038156b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603fc25640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60380cc770 con 0x7f60380bfb40 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603fc25640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6038157780 con 0x7f60380ca3b0 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603fc25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f603815abd0 con 0x7f60381540b0 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603d9c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 0x7f60380c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603d9c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 0x7f60380c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:36094/0 (socket says 172.21.15.135:36094) 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603d9c1640 1 -- 172.21.15.135:0/3276780973 learned_addr learned my addr 172.21.15.135:0/3276780973 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.817+0000 7f603d1c0640 1 --2- 172.21.15.135:0/3276780973 >> v2:172.21.15.139:3300/0 conn(0x7f60380ca3b0 0x7f6038156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.818+0000 7f603d9c1640 1 -- 172.21.15.135:0/3276780973 >> v2:172.21.15.139:3300/0 conn(0x7f60380ca3b0 msgr2=0x7f6038156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.818+0000 7f603d9c1640 1 --2- 172.21.15.135:0/3276780973 >> v2:172.21.15.139:3300/0 conn(0x7f60380ca3b0 0x7f6038156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.818+0000 7f603d9c1640 1 -- 172.21.15.135:0/3276780973 >> v1:172.21.15.139:6789/0 conn(0x7f60380bfb40 legacy=0x7f6038157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.818+0000 7f603d9c1640 1 -- 172.21.15.135:0/3276780973 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603815aeb0 con 0x7f60381540b0 2024-04-26T22:00:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.818+0000 7f603d9c1640 1 --2- 172.21.15.135:0/3276780973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 0x7f60380c6e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6030009e50 tx=0x7f603002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea363d377504daec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.818+0000 7f603c9bf640 1 -- 172.21.15.135:0/3276780973 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030036070 con 0x7f60381540b0 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603c9bf640 1 -- 172.21.15.135:0/3276780973 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030031600 con 0x7f60381540b0 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603c9bf640 1 -- 172.21.15.135:0/3276780973 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300319d0 con 0x7f60381540b0 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 -- 172.21.15.135:0/3276780973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 msgr2=0x7f60380c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 --2- 172.21.15.135:0/3276780973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 0x7f60380c6e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6030009e50 tx=0x7f603002a5f0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 -- 172.21.15.135:0/3276780973 shutdown_connections 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 --2- 172.21.15.135:0/3276780973 >> v2:172.21.15.139:3300/0 conn(0x7f60380ca3b0 0x7f6038156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 --2- 172.21.15.135:0/3276780973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60381540b0 0x7f60380c6e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 -- 172.21.15.135:0/3276780973 >> 172.21.15.135:0/3276780973 conn(0x7f603806c4f0 msgr2=0x7f603806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 -- 172.21.15.135:0/3276780973 shutdown_connections 2024-04-26T22:00:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.819+0000 7f603fc25640 1 -- 172.21.15.135:0/3276780973 wait complete. 2024-04-26T22:00:19.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.820+0000 7f603fc25640 1 Processor -- start 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.820+0000 7f603fc25640 1 -- start start 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603fc25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60380bfb40 0x7f60380c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603fc25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60380ca3b0 0x7f60380ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603fc25640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 0x7f60381f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603fc25640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6038154490 con 0x7f60381540b0 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603fc25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60380cccd0 con 0x7f60380bfb40 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603fc25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6038072930 con 0x7f60380ca3b0 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 0x7f60381f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 0x7f60381f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57936/0 (socket says 172.21.15.135:57936) 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 -- 172.21.15.135:0/3338394177 learned_addr learned my addr 172.21.15.135:0/3338394177 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603d1c0640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60380ca3b0 0x7f60380ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603d9c1640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60380bfb40 0x7f60380c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 -- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60380ca3b0 msgr2=0x7f60380ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60380ca3b0 0x7f60380ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 -- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60380bfb40 msgr2=0x7f60380c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60380bfb40 0x7f60380c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.821+0000 7f603e1c2640 1 -- 172.21.15.135:0/3338394177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60380c3260 con 0x7f60381540b0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.822+0000 7f603e1c2640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 0x7f60381f04f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f602800c3f0 tx=0x7f602800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.822+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280052d0 con 0x7f60381540b0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.822+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6028005490 con 0x7f60381540b0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.822+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028007260 con 0x7f60381540b0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.822+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60380ad900 con 0x7f60381540b0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.822+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60381e4200 con 0x7f60381540b0 2024-04-26T22:00:19.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.823+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60380507a0 con 0x7f60381540b0 2024-04-26T22:00:19.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.823+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6028007400 con 0x7f60381540b0 2024-04-26T22:00:19.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.824+0000 7f60367fc640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f602406c8f0 0x7f602406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:19.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.824+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f602808b200 con 0x7f60381540b0 2024-04-26T22:00:19.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.824+0000 7f603d9c1640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f602406c8f0 0x7f602406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:19.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.824+0000 7f603d9c1640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f602406c8f0 0x7f602406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6030033000 tx=0x7f6030009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:19.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:19.828+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f60380cc770 con 0x7f60381540b0 2024-04-26T22:00:20.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.004+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f6038073590 con 0x7f60381540b0 2024-04-26T22:00:20.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.005+0000 7f60367fc640 1 -- 172.21.15.135:0/3338394177 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f6028058cb0 con 0x7f60381540b0 2024-04-26T22:00:20.005 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f602406c8f0 msgr2=0x7f602406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f602406c8f0 0x7f602406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6030033000 tx=0x7f6030009000 comp rx=0 tx=0).stop 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 msgr2=0x7f60381f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 0x7f60381f04f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f602800c3f0 tx=0x7f602800c8b0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 shutdown_connections 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f602406c8f0 0x7f602406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60381540b0 0x7f60381f04f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60380ca3b0 0x7f60380ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 --2- 172.21.15.135:0/3338394177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60380bfb40 0x7f60380c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 >> 172.21.15.135:0/3338394177 conn(0x7f603806c4f0 msgr2=0x7f6038152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 shutdown_connections 2024-04-26T22:00:20.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.007+0000 7f603fc25640 1 -- 172.21.15.135:0/3338394177 wait complete. 2024-04-26T22:00:20.017 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:20.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.118+0000 7fece5386640 1 Processor -- start 2024-04-26T22:00:20.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.118+0000 7fece5386640 1 -- start start 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fece5386640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fece5386640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fece00c5520 0x7fece00c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fece5386640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece00c7a40 con 0x7fece00adeb0 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fece5386640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fece0155550 con 0x7fece00c5520 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fece5386640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fece0155a80 con 0x7fece00c4cd0 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fece00c5520 0x7fece00c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdeffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fece00adeb0 0x7fece0151fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46284/0 (socket says 172.21.15.135:46284) 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdeffd640 1 -- 172.21.15.135:0/410343749 learned_addr learned my addr 172.21.15.135:0/410343749 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecde7fc640 1 --2- 172.21.15.135:0/410343749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdf7fe640 1 -- 172.21.15.135:0/410343749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 msgr2=0x7fece006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdf7fe640 1 --2- 172.21.15.135:0/410343749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece006ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdf7fe640 1 -- 172.21.15.135:0/410343749 >> v1:172.21.15.135:6789/0 conn(0x7fece00adeb0 legacy=0x7fece0151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecddffb640 1 -- 172.21.15.135:0/410343749 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1802517367 0 0) 0x7fece00c7a40 con 0x7fece00adeb0 2024-04-26T22:00:20.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.119+0000 7fecdf7fe640 1 -- 172.21.15.135:0/410343749 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece01557a0 con 0x7fece00c5520 2024-04-26T22:00:20.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.120+0000 7fecdf7fe640 1 --2- 172.21.15.135:0/410343749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00c5520 0x7fece00c59b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fecd80097a0 tx=0x7fecd802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c2214103229727 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.120+0000 7fecddffb640 1 -- 172.21.15.135:0/410343749 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8037070 con 0x7fece00c5520 2024-04-26T22:00:20.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.120+0000 7fecddffb640 1 -- 172.21.15.135:0/410343749 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd802adb0 con 0x7fece00c5520 2024-04-26T22:00:20.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.120+0000 7fecddffb640 1 -- 172.21.15.135:0/410343749 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8032660 con 0x7fece00c5520 2024-04-26T22:00:20.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.120+0000 7fece5386640 1 -- 172.21.15.135:0/410343749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00c5520 msgr2=0x7fece00c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.120+0000 7fece5386640 1 --2- 172.21.15.135:0/410343749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00c5520 0x7fece00c59b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fecd80097a0 tx=0x7fecd802a060 comp rx=0 tx=0).stop 2024-04-26T22:00:20.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.121+0000 7fece5386640 1 -- 172.21.15.135:0/410343749 shutdown_connections 2024-04-26T22:00:20.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.121+0000 7fece5386640 1 --2- 172.21.15.135:0/410343749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece006ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.121+0000 7fece5386640 1 --2- 172.21.15.135:0/410343749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00c5520 0x7fece00c59b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.121+0000 7fece5386640 1 -- 172.21.15.135:0/410343749 >> 172.21.15.135:0/410343749 conn(0x7fece00aa630 msgr2=0x7fece00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.121+0000 7fece5386640 1 -- 172.21.15.135:0/410343749 shutdown_connections 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.121+0000 7fece5386640 1 -- 172.21.15.135:0/410343749 wait complete. 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 Processor -- start 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 -- start start 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 0x7fece01584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece00c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fece00c5520 0x7fece00c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fece00c7660 con 0x7fece00adeb0 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fece00a8380 con 0x7fece00c5520 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fece5386640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fece01526b0 con 0x7fece00c4cd0 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.122+0000 7fecdeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 0x7fece01584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 0x7fece01584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57944/0 (socket says 172.21.15.135:57944) 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 -- 172.21.15.135:0/2877162116 learned_addr learned my addr 172.21.15.135:0/2877162116 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 -- 172.21.15.135:0/2877162116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 msgr2=0x7fece00c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece00c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 -- 172.21.15.135:0/2877162116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fece00c5520 msgr2=0x7fece00c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fece00c5520 0x7fece00c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 -- 172.21.15.135:0/2877162116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece00c2590 con 0x7fece00adeb0 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecdeffd640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 0x7fece01584d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c7f0 tx=0x7fecd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0018070 con 0x7fece00adeb0 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd000a640 con 0x7fece00adeb0 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd000aa30 con 0x7fece00adeb0 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece0156e00 con 0x7fece00adeb0 2024-04-26T22:00:20.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.123+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece01573c0 con 0x7fece00adeb0 2024-04-26T22:00:20.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.124+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece00765e0 con 0x7fece00adeb0 2024-04-26T22:00:20.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.124+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd001b070 con 0x7fece00adeb0 2024-04-26T22:00:20.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.125+0000 7fecc77fe640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.125+0000 7fecde7fc640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.125+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fecd008b410 con 0x7fece00adeb0 2024-04-26T22:00:20.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.125+0000 7fecde7fc640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fecd4006fd0 tx=0x7fecd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.129+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecd00595a0 con 0x7fece00adeb0 2024-04-26T22:00:20.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.304+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fece00507a0 con 0x7fece00adeb0 2024-04-26T22:00:20.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.304+0000 7fecc77fe640 1 -- 172.21.15.135:0/2877162116 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fecd0058ea0 con 0x7fece00adeb0 2024-04-26T22:00:20.304 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:20.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feccc06c8f0 msgr2=0x7feccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fecd4006fd0 tx=0x7fecd4008040 comp rx=0 tx=0).stop 2024-04-26T22:00:20.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 msgr2=0x7fece01584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 0x7fece01584d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c7f0 tx=0x7fecd000ccb0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 shutdown_connections 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fece00c5520 0x7fece00c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fece00c4cd0 0x7fece00c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 --2- 172.21.15.135:0/2877162116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fece00adeb0 0x7fece01584d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 >> 172.21.15.135:0/2877162116 conn(0x7fece00aa630 msgr2=0x7fece0154980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 shutdown_connections 2024-04-26T22:00:20.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.306+0000 7fece5386640 1 -- 172.21.15.135:0/2877162116 wait complete. 2024-04-26T22:00:20.317 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:20.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.413+0000 7ff69f7af640 1 Processor -- start 2024-04-26T22:00:20.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.414+0000 7ff69f7af640 1 -- start start 2024-04-26T22:00:20.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.414+0000 7ff69f7af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981510c0 0x7ff698072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.414+0000 7ff69f7af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff6980c4160 0x7ff69815e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.414+0000 7ff69f7af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6980c4b50 con 0x7ff6980734d0 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.414+0000 7ff69f7af640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff6980c50d0 con 0x7ff6980c4160 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.414+0000 7ff69f7af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff6981514a0 con 0x7ff6981510c0 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69dd4c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff6980c4160 0x7ff69815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69d54b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981510c0 0x7ff698072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69cd4a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff6980734d0 0x7ff69806d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:41934/0 (socket says 172.21.15.135:41934) 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69cd4a640 1 -- 172.21.15.135:0/3609651985 learned_addr learned my addr 172.21.15.135:0/3609651985 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69d54b640 1 -- 172.21.15.135:0/3609651985 >> v2:172.21.15.139:3300/0 conn(0x7ff6980c4160 msgr2=0x7ff69815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:20.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69d54b640 1 --2- 172.21.15.135:0/3609651985 >> v2:172.21.15.139:3300/0 conn(0x7ff6980c4160 0x7ff69815e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69d54b640 1 -- 172.21.15.135:0/3609651985 >> v1:172.21.15.139:6789/0 conn(0x7ff6980734d0 legacy=0x7ff69806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.415+0000 7ff69d54b640 1 -- 172.21.15.135:0/3609651985 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff698154920 con 0x7ff6981510c0 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69d54b640 1 --2- 172.21.15.135:0/3609651985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981510c0 0x7ff698072f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6980c5250 tx=0x7ff69402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec7f49057f45d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff68ffff640 1 -- 172.21.15.135:0/3609651985 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694037070 con 0x7ff6981510c0 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff68ffff640 1 -- 172.21.15.135:0/3609651985 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff69402adb0 con 0x7ff6981510c0 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff68ffff640 1 -- 172.21.15.135:0/3609651985 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694032660 con 0x7ff6981510c0 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69f7af640 1 -- 172.21.15.135:0/3609651985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981510c0 msgr2=0x7ff698072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3609651985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981510c0 0x7ff698072f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6980c5250 tx=0x7ff69402a060 comp rx=0 tx=0).stop 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69f7af640 1 -- 172.21.15.135:0/3609651985 shutdown_connections 2024-04-26T22:00:20.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3609651985 >> v2:172.21.15.139:3300/0 conn(0x7ff6980c4160 0x7ff69815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3609651985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981510c0 0x7ff698072f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.416+0000 7ff69f7af640 1 -- 172.21.15.135:0/3609651985 >> 172.21.15.135:0/3609651985 conn(0x7ff6980c5ce0 msgr2=0x7ff6980c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.417+0000 7ff69f7af640 1 -- 172.21.15.135:0/3609651985 shutdown_connections 2024-04-26T22:00:20.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.417+0000 7ff69f7af640 1 -- 172.21.15.135:0/3609651985 wait complete. 2024-04-26T22:00:20.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 Processor -- start 2024-04-26T22:00:20.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 -- start start 2024-04-26T22:00:20.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6980c4160 0x7ff6980c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 0x7ff6981ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981ee6c0 0x7ff6981f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6980c5920 con 0x7ff6981510c0 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6980c5400 con 0x7ff6980c4160 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69f7af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff69806f500 con 0x7ff6981ee6c0 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69cd4a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 0x7ff6981ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69cd4a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 0x7ff6981ee180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57960/0 (socket says 172.21.15.135:57960) 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69cd4a640 1 -- 172.21.15.135:0/3921751186 learned_addr learned my addr 172.21.15.135:0/3921751186 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.418+0000 7ff69dd4c640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981ee6c0 0x7ff6981f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69d54b640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6980c4160 0x7ff6980c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69cd4a640 1 -- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981ee6c0 msgr2=0x7ff6981f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69cd4a640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981ee6c0 0x7ff6981f0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69cd4a640 1 -- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6980c4160 msgr2=0x7ff6980c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69cd4a640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6980c4160 0x7ff6980c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69cd4a640 1 -- 172.21.15.135:0/3921751186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff698144c10 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69cd4a640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 0x7ff6981ee180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff68800c400 tx=0x7ff68800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688017070 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff688005ce0 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff698144ef0 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6880072f0 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.419+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6980c8e30 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.420+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff698147970 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.420+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff68801a070 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.421+0000 7ff68dffb640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff67c06c8f0 0x7ff67c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.421+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ff68808b200 con 0x7ff6981510c0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.421+0000 7ff69d54b640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff67c06c8f0 0x7ff67c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.421+0000 7ff69d54b640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff67c06c8f0 0x7ff67c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff694034000 tx=0x7ff69402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.425+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff688059390 con 0x7ff6981510c0 2024-04-26T22:00:20.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.634+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6980507a0 con 0x7ff6981510c0 2024-04-26T22:00:20.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.635+0000 7ff68dffb640 1 -- 172.21.15.135:0/3921751186 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7ff688058c90 con 0x7ff6981510c0 2024-04-26T22:00:20.635 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:20.635 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:20.635 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff67c06c8f0 msgr2=0x7ff67c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff67c06c8f0 0x7ff67c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff694034000 tx=0x7ff69402e000 comp rx=0 tx=0).stop 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 msgr2=0x7ff6981ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 0x7ff6981ee180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff68800c400 tx=0x7ff68800c8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 shutdown_connections 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff67c06c8f0 0x7ff67c06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6981ee6c0 0x7ff6981f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6981510c0 0x7ff6981ee180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 --2- 172.21.15.135:0/3921751186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6980c4160 0x7ff6980c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 >> 172.21.15.135:0/3921751186 conn(0x7ff6980c5ce0 msgr2=0x7ff6980756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 shutdown_connections 2024-04-26T22:00:20.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.637+0000 7ff69f7af640 1 -- 172.21.15.135:0/3921751186 wait complete. 2024-04-26T22:00:20.649 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:20.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.744+0000 7f5671c08640 1 Processor -- start 2024-04-26T22:00:20.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5671c08640 1 -- start start 2024-04-26T22:00:20.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5671c08640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5671c08640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c0c8df0 con 0x7f566c0ca5a0 2024-04-26T22:00:20.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5671c08640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c154540 con 0x7f566c154160 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5671c08640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f566c15a6e0 con 0x7f566c156bb0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5663fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f566c0ca5a0 0x7f566c0c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46286/0 (socket says 172.21.15.135:46286) 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.745+0000 7f5663fff640 1 -- 172.21.15.135:0/935442443 learned_addr learned my addr 172.21.15.135:0/935442443 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566bfff640 1 --2- 172.21.15.135:0/935442443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845851367 0 0) 0x7f5654001110 con 0x7f566c0ca5a0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5658003520 con 0x7f566c0ca5a0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576239907 0 0) 0x7f566c154540 con 0x7f566c154160 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5654001110 con 0x7f566c154160 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566bfff640 1 -- 172.21.15.135:0/935442443 >> v1:172.21.15.139:6789/0 conn(0x7f566c154160 legacy=0x7f566c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566bfff640 1 -- 172.21.15.135:0/935442443 >> v1:172.21.15.135:6789/0 conn(0x7f566c0ca5a0 legacy=0x7f566c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3713032283 0 0) 0x7f5658003520 con 0x7f566c0ca5a0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.746+0000 7f566bfff640 1 -- 172.21.15.135:0/935442443 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c15a910 con 0x7f566c156bb0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f566bfff640 1 --2- 172.21.15.135:0/935442443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c15a1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5664009dc0 tx=0x7f566402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb6a1292144ecbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654001840 con 0x7f566c156bb0 2024-04-26T22:00:20.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664007540 con 0x7f566c156bb0 2024-04-26T22:00:20.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f566affd640 1 -- 172.21.15.135:0/935442443 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640078f0 con 0x7f566c156bb0 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 -- 172.21.15.135:0/935442443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 msgr2=0x7f566c15a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 --2- 172.21.15.135:0/935442443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c15a1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5664009dc0 tx=0x7f566402a660 comp rx=0 tx=0).stop 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 -- 172.21.15.135:0/935442443 shutdown_connections 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 --2- 172.21.15.135:0/935442443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c15a1a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 -- 172.21.15.135:0/935442443 >> 172.21.15.135:0/935442443 conn(0x7f566c0ae010 msgr2=0x7f566c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 -- 172.21.15.135:0/935442443 shutdown_connections 2024-04-26T22:00:20.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.747+0000 7f5671c08640 1 -- 172.21.15.135:0/935442443 wait complete. 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.748+0000 7f5671c08640 1 Processor -- start 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.748+0000 7f5671c08640 1 -- start start 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5671c08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f566c0ca5a0 0x7f566c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5671c08640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 0x7f566c14ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5671c08640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5671c08640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f566c15aed0 con 0x7f566c154160 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5671c08640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f566c0c9100 con 0x7f566c0ca5a0 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5671c08640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f566c0c7510 con 0x7f566c156bb0 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 0x7f566c14ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 0x7f566c14ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57972/0 (socket says 172.21.15.135:57972) 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 -- 172.21.15.135:0/2729683728 learned_addr learned my addr 172.21.15.135:0/2729683728 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f566bfff640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 -- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 msgr2=0x7f566c14cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c14cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f566b7fe640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f566c0ca5a0 0x7f566c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 -- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f566c0ca5a0 msgr2=0x7f566c155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f566c0ca5a0 0x7f566c155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f5663fff640 1 -- 172.21.15.135:0/2729683728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c0c40e0 con 0x7f566c154160 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.749+0000 7f566bfff640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c14cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.750+0000 7f5663fff640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 0x7f566c14ca80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5654000fd0 tx=0x7f565400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.750+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654017070 con 0x7f566c154160 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.750+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5654005d00 con 0x7f566c154160 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.750+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c0c43c0 con 0x7f566c154160 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.750+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56540072b0 con 0x7f566c154160 2024-04-26T22:00:20.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.750+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c156350 con 0x7f566c154160 2024-04-26T22:00:20.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.751+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c0c3e50 con 0x7f566c154160 2024-04-26T22:00:20.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.751+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f565401a070 con 0x7f566c154160 2024-04-26T22:00:20.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.752+0000 7f5668ff9640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f564c06c8f0 0x7f564c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:20.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.752+0000 7f566b7fe640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f564c06c8f0 0x7f564c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:20.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.752+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f565408b490 con 0x7f566c154160 2024-04-26T22:00:20.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.752+0000 7f566b7fe640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f564c06c8f0 0x7f564c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f565c0010e0 tx=0x7f565c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:20.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.756+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f566c0c8df0 con 0x7f566c154160 2024-04-26T22:00:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.932+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f566c076650 con 0x7f566c154160 2024-04-26T22:00:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.933+0000 7f5668ff9640 1 -- 172.21.15.135:0/2729683728 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f5654059620 con 0x7f566c154160 2024-04-26T22:00:20.933 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:20.934 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f564c06c8f0 msgr2=0x7f564c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f564c06c8f0 0x7f564c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f565c0010e0 tx=0x7f565c009040 comp rx=0 tx=0).stop 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 msgr2=0x7f566c14ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 0x7f566c14ca80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5654000fd0 tx=0x7f565400cf50 comp rx=0 tx=0).stop 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 shutdown_connections 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f564c06c8f0 0x7f564c06edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f566c156bb0 0x7f566c14cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f566c154160 0x7f566c14ca80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 --2- 172.21.15.135:0/2729683728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f566c0ca5a0 0x7f566c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 >> 172.21.15.135:0/2729683728 conn(0x7f566c0ae010 msgr2=0x7f566c0c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 shutdown_connections 2024-04-26T22:00:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:20.936+0000 7f5671c08640 1 -- 172.21.15.135:0/2729683728 wait complete. 2024-04-26T22:00:20.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:20.947 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:21.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.047+0000 7f7de2df7640 1 Processor -- start 2024-04-26T22:00:21.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.047+0000 7f7de2df7640 1 -- start start 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de2df7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ddc154160 0x7f7ddc156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de2df7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ddc15a1a0 0x7f7ddc0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de2df7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ddc0c8df0 con 0x7f7ddc0ca5a0 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de2df7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7ddc0c9100 con 0x7f7ddc15a1a0 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de2df7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7ddc154540 con 0x7f7ddc154160 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de1394640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ddc15a1a0 0x7f7ddc0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de1394640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ddc15a1a0 0x7f7ddc0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57974/0 (socket says 172.21.15.135:57974) 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7ddbfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7ddc0ca5a0 0x7f7ddc0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46290/0 (socket says 172.21.15.135:46290) 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.048+0000 7f7de1394640 1 -- 172.21.15.135:0/1697155717 learned_addr learned my addr 172.21.15.135:0/1697155717 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7de1394640 1 -- 172.21.15.135:0/1697155717 >> v2:172.21.15.139:3300/0 conn(0x7f7ddc154160 msgr2=0x7f7ddc156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7de1394640 1 --2- 172.21.15.135:0/1697155717 >> v2:172.21.15.139:3300/0 conn(0x7f7ddc154160 0x7f7ddc156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7de1394640 1 -- 172.21.15.135:0/1697155717 >> v1:172.21.15.135:6789/0 conn(0x7f7ddc0ca5a0 legacy=0x7f7ddc0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7ddb7fe640 1 -- 172.21.15.135:0/1697155717 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077619682 0 0) 0x7f7dd0001110 con 0x7f7ddc0ca5a0 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7de1394640 1 -- 172.21.15.135:0/1697155717 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc15a910 con 0x7f7ddc15a1a0 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7de1394640 1 --2- 172.21.15.135:0/1697155717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0c9590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0c9280 tx=0x7f7dd402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b09da705ba32098 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7ddb7fe640 1 -- 172.21.15.135:0/1697155717 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4036070 con 0x7f7ddc15a1a0 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7ddb7fe640 1 -- 172.21.15.135:0/1697155717 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd4031600 con 0x7f7ddc15a1a0 2024-04-26T22:00:21.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.049+0000 7f7ddb7fe640 1 -- 172.21.15.135:0/1697155717 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40319d0 con 0x7f7ddc15a1a0 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 -- 172.21.15.135:0/1697155717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc15a1a0 msgr2=0x7f7ddc0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1697155717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0c9590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0c9280 tx=0x7f7dd402a730 comp rx=0 tx=0).stop 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 -- 172.21.15.135:0/1697155717 shutdown_connections 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1697155717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0c9590 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1697155717 >> v2:172.21.15.139:3300/0 conn(0x7f7ddc154160 0x7f7ddc156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 -- 172.21.15.135:0/1697155717 >> 172.21.15.135:0/1697155717 conn(0x7f7ddc0ae010 msgr2=0x7f7ddc0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 -- 172.21.15.135:0/1697155717 shutdown_connections 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.050+0000 7f7de2df7640 1 -- 172.21.15.135:0/1697155717 wait complete. 2024-04-26T22:00:21.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.051+0000 7f7de2df7640 1 Processor -- start 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.051+0000 7f7de2df7640 1 -- start start 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de2df7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ddc0ca5a0 0x7f7ddc149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de2df7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 0x7f7ddc0c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de2df7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de2df7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc15a5b0 con 0x7f7ddc154160 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de2df7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc15aed0 con 0x7f7ddc15a1a0 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de2df7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7ddc0c7510 con 0x7f7ddc0ca5a0 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 0x7f7ddc0c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 0x7f7ddc0c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57984/0 (socket says 172.21.15.135:57984) 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 -- 172.21.15.135:0/1820458367 learned_addr learned my addr 172.21.15.135:0/1820458367 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de0b93640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ddc0ca5a0 0x7f7ddc149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7de1394640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 -- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ddc0ca5a0 msgr2=0x7f7ddc149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ddc0ca5a0 0x7f7ddc149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 -- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ddc15a1a0 msgr2=0x7f7ddc0ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 -- 172.21.15.135:0/1820458367 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc154bc0 con 0x7f7ddc154160 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.052+0000 7f7ddbfff640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 0x7f7ddc0c5cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000c7b0 tx=0x7f7dd000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.053+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0018070 con 0x7f7ddc154160 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.053+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd00072d0 con 0x7f7ddc154160 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.053+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc151d10 con 0x7f7ddc154160 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.053+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00047e0 con 0x7f7ddc154160 2024-04-26T22:00:21.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.053+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc154e60 con 0x7f7ddc154160 2024-04-26T22:00:21.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.054+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc1550a0 con 0x7f7ddc154160 2024-04-26T22:00:21.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.054+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dd001b070 con 0x7f7ddc154160 2024-04-26T22:00:21.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.054+0000 7f7dd97fa640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7dc806c8f0 0x7f7dc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.055+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f7dd008b1a0 con 0x7f7ddc154160 2024-04-26T22:00:21.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.055+0000 7f7de0b93640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7dc806c8f0 0x7f7dc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.055+0000 7f7de0b93640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7dc806c8f0 0x7f7dc806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009890 tx=0x7f7dcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.059+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ddc0c8df0 con 0x7f7ddc154160 2024-04-26T22:00:21.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.234+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ddc076650 con 0x7f7ddc154160 2024-04-26T22:00:21.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.235+0000 7f7dd97fa640 1 -- 172.21.15.135:0/1820458367 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f7dd00592a0 con 0x7f7ddc154160 2024-04-26T22:00:21.235 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:21.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.236+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7dc806c8f0 msgr2=0x7f7dc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.236+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7dc806c8f0 0x7f7dc806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009890 tx=0x7f7dcc008040 comp rx=0 tx=0).stop 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.236+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 msgr2=0x7f7ddc0c5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.236+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 0x7f7ddc0c5cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000c7b0 tx=0x7f7dd000cc70 comp rx=0 tx=0).stop 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 shutdown_connections 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7dc806c8f0 0x7f7dc806edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ddc15a1a0 0x7f7ddc0ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ddc154160 0x7f7ddc0c5cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 --2- 172.21.15.135:0/1820458367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ddc0ca5a0 0x7f7ddc149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 >> 172.21.15.135:0/1820458367 conn(0x7f7ddc0ae010 msgr2=0x7f7ddc0c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 shutdown_connections 2024-04-26T22:00:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.237+0000 7f7de2df7640 1 -- 172.21.15.135:0/1820458367 wait complete. 2024-04-26T22:00:21.247 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:21.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.347+0000 7f6aa24bd640 1 Processor -- start 2024-04-26T22:00:21.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.347+0000 7f6aa24bd640 1 -- start start 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6aa24bd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6a9c154160 0x7f6a9c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6aa24bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6aa24bd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9c0c8df0 con 0x7f6a9c0ca5a0 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6aa24bd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9c0c9100 con 0x7f6a9c154160 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6aa24bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c154540 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6a9c154160 0x7f6a9c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9b7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f6a9c0ca5a0 0x7f6a9c0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46302/0 (socket says 172.21.15.135:46302) 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6a9c154160 0x7f6a9c156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57992/0 (socket says 172.21.15.135:57992) 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9b7fe640 1 -- 172.21.15.135:0/3017355864 learned_addr learned my addr 172.21.15.135:0/3017355864 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:21.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6aa0a5a640 1 --2- 172.21.15.135:0/3017355864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9bfff640 1 -- 172.21.15.135:0/3017355864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c15a1a0 msgr2=0x7f6a9c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9bfff640 1 --2- 172.21.15.135:0/3017355864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.348+0000 7f6a9bfff640 1 -- 172.21.15.135:0/3017355864 >> v1:172.21.15.135:6789/0 conn(0x7f6a9c0ca5a0 legacy=0x7f6a9c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.349+0000 7f6a9affd640 1 -- 172.21.15.135:0/3017355864 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2597363725 0 0) 0x7f6a90001110 con 0x7f6a9c0ca5a0 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.349+0000 7f6a9bfff640 1 -- 172.21.15.135:0/3017355864 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c15a910 con 0x7f6a9c154160 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.349+0000 7f6a9bfff640 1 --2- 172.21.15.135:0/3017355864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c154160 0x7f6a9c156bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0097a0 tx=0x7f6a8c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5a9311e9742ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.349+0000 7f6a9affd640 1 -- 172.21.15.135:0/3017355864 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c037070 con 0x7f6a9c154160 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.349+0000 7f6a9affd640 1 -- 172.21.15.135:0/3017355864 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8c032440 con 0x7f6a9c154160 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.349+0000 7f6a9affd640 1 -- 172.21.15.135:0/3017355864 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c032810 con 0x7f6a9c154160 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 -- 172.21.15.135:0/3017355864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c154160 msgr2=0x7f6a9c156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/3017355864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c154160 0x7f6a9c156bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0097a0 tx=0x7f6a8c02a180 comp rx=0 tx=0).stop 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 -- 172.21.15.135:0/3017355864 shutdown_connections 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/3017355864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/3017355864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c154160 0x7f6a9c156bb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 -- 172.21.15.135:0/3017355864 >> 172.21.15.135:0/3017355864 conn(0x7f6a9c0ae010 msgr2=0x7f6a9c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 -- 172.21.15.135:0/3017355864 shutdown_connections 2024-04-26T22:00:21.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.350+0000 7f6aa24bd640 1 -- 172.21.15.135:0/3017355864 wait complete. 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.351+0000 7f6aa24bd640 1 Processor -- start 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.351+0000 7f6aa24bd640 1 -- start start 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa24bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c0ca5a0 0x7f6a9c0c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa24bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a9c154160 0x7f6a9c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa24bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa24bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c15a5f0 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa24bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c15aed0 con 0x7f6a9c154160 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa24bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c0c7510 con 0x7f6a9c0ca5a0 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57994/0 (socket says 172.21.15.135:57994) 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 -- 172.21.15.135:0/608988179 learned_addr learned my addr 172.21.15.135:0/608988179 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6a9bfff640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c0ca5a0 0x7f6a9c0c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6a9b7fe640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a9c154160 0x7f6a9c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 -- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c0ca5a0 msgr2=0x7f6a9c0c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c0ca5a0 0x7f6a9c0c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 -- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a9c154160 msgr2=0x7f6a9c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a9c154160 0x7f6a9c0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 -- 172.21.15.135:0/608988179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c0c3160 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.352+0000 7f6aa0a5a640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c3a90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400cc30 tx=0x7f6a9400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.353+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400ce40 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.353+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a94003990 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.353+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c1564e0 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.353+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94003d40 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.353+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c154d00 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.354+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c076650 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.354+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a94003ee0 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.354+0000 7f6a98ff9640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6a8806c920 0x7f6a8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.355+0000 7f6a9bfff640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6a8806c920 0x7f6a8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.355+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f6a9408c220 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.355+0000 7f6a9bfff640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6a8806c920 0x7f6a8806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c034000 tx=0x7f6a8c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.359+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a94014030 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.536+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f6a9c0ca380 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.537+0000 7f6a98ff9640 1 -- 172.21.15.135:0/608988179 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f6a94059cd0 con 0x7f6a9c15a1a0 2024-04-26T22:00:21.537 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6a8806c920 msgr2=0x7f6a8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6a8806c920 0x7f6a8806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c034000 tx=0x7f6a8c02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 msgr2=0x7f6a9c0c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c3a90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400cc30 tx=0x7f6a9400a5c0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 shutdown_connections 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6a8806c920 0x7f6a8806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a9c15a1a0 0x7f6a9c0c3a90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a9c154160 0x7f6a9c0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 --2- 172.21.15.135:0/608988179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a9c0ca5a0 0x7f6a9c0c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 >> 172.21.15.135:0/608988179 conn(0x7f6a9c0ae010 msgr2=0x7f6a9c0c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 shutdown_connections 2024-04-26T22:00:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.539+0000 7f6aa24bd640 1 -- 172.21.15.135:0/608988179 wait complete. 2024-04-26T22:00:21.550 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:21.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.650+0000 7ffbdc6fc640 1 Processor -- start 2024-04-26T22:00:21.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.650+0000 7ffbdc6fc640 1 -- start start 2024-04-26T22:00:21.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbdc6fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbd4070120 0x7ffbd40aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbdc6fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd40aef70 0x7ffbd40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbdc6fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd4154e10 con 0x7ffbd406d590 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbdc6fc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd40af400 con 0x7ffbd4070120 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbdc6fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd41559e0 con 0x7ffbd40aef70 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbd4070120 0x7ffbd40aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbd4070120 0x7ffbd40aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58004/0 (socket says 172.21.15.135:58004) 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 -- 172.21.15.135:0/3200595817 learned_addr learned my addr 172.21.15.135:0/3200595817 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbdac99640 1 --2- 172.21.15.135:0/3200595817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd40aef70 0x7ffbd40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 -- 172.21.15.135:0/3200595817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd40aef70 msgr2=0x7ffbd40c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 --2- 172.21.15.135:0/3200595817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd40aef70 0x7ffbd40c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 -- 172.21.15.135:0/3200595817 >> v1:172.21.15.139:6789/0 conn(0x7ffbd406d590 legacy=0x7ffbd4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.651+0000 7ffbd9c97640 1 -- 172.21.15.135:0/3200595817 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41550b0 con 0x7ffbd4070120 2024-04-26T22:00:21.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.652+0000 7ffbd9c97640 1 --2- 172.21.15.135:0/3200595817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd4070120 0x7ffbd40aea30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009870 tx=0x7ffbc402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c2a46d9c6ae764 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.652+0000 7ffbd9496640 1 -- 172.21.15.135:0/3200595817 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4037070 con 0x7ffbd4070120 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.652+0000 7ffbd9496640 1 -- 172.21.15.135:0/3200595817 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc4032440 con 0x7ffbd4070120 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.652+0000 7ffbd9496640 1 -- 172.21.15.135:0/3200595817 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4032810 con 0x7ffbd4070120 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/3200595817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd4070120 msgr2=0x7ffbd40aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/3200595817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd4070120 0x7ffbd40aea30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009870 tx=0x7ffbc402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/3200595817 shutdown_connections 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/3200595817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd40aef70 0x7ffbd40c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/3200595817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd4070120 0x7ffbd40aea30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/3200595817 >> 172.21.15.135:0/3200595817 conn(0x7ffbd40c5ce0 msgr2=0x7ffbd40c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/3200595817 shutdown_connections 2024-04-26T22:00:21.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.653+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/3200595817 wait complete. 2024-04-26T22:00:21.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.654+0000 7ffbdc6fc640 1 Processor -- start 2024-04-26T22:00:21.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.654+0000 7ffbdc6fc640 1 -- start start 2024-04-26T22:00:21.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdc6fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd406d590 0x7ffbd414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdc6fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd4070120 0x7ffbd41e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdc6fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 0x7ffbd4146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdc6fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd406d970 con 0x7ffbd40aef70 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdc6fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4155670 con 0x7ffbd406d590 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdc6fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd406daf0 con 0x7ffbd4070120 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 0x7ffbd4146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 0x7ffbd4146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58018/0 (socket says 172.21.15.135:58018) 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 -- 172.21.15.135:0/2547518574 learned_addr learned my addr 172.21.15.135:0/2547518574 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbda498640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd406d590 0x7ffbd414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbd9c97640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd4070120 0x7ffbd41e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 -- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd4070120 msgr2=0x7ffbd41e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd4070120 0x7ffbd41e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 -- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd406d590 msgr2=0x7ffbd414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd406d590 0x7ffbd414f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 -- 172.21.15.135:0/2547518574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd4150980 con 0x7ffbd40aef70 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.655+0000 7ffbdac99640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 0x7ffbd4146a80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00efc0 tx=0x7ffbcc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.656+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc018020 con 0x7ffbd40aef70 2024-04-26T22:00:21.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.656+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc014480 con 0x7ffbd40aef70 2024-04-26T22:00:21.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.656+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd4150f40 con 0x7ffbd40aef70 2024-04-26T22:00:21.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.656+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc014870 con 0x7ffbd40aef70 2024-04-26T22:00:21.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.656+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd4151500 con 0x7ffbd40aef70 2024-04-26T22:00:21.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.657+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd40507a0 con 0x7ffbd40aef70 2024-04-26T22:00:21.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.657+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbcc005d10 con 0x7ffbd40aef70 2024-04-26T22:00:21.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.658+0000 7ffbcaffd640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbb806c8f0 0x7ffbb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.658+0000 7ffbda498640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbb806c8f0 0x7ffbb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.658+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ffbcc08cb30 con 0x7ffbd40aef70 2024-04-26T22:00:21.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.658+0000 7ffbda498640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbb806c8f0 0x7ffbb806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00016e0 tx=0x7ffbd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.662+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffbcc022030 con 0x7ffbd40aef70 2024-04-26T22:00:21.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.837+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7ffbd406ff00 con 0x7ffbd40aef70 2024-04-26T22:00:21.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.837+0000 7ffbcaffd640 1 -- 172.21.15.135:0/2547518574 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7ffbcc010040 con 0x7ffbd40aef70 2024-04-26T22:00:21.837 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:21.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbb806c8f0 msgr2=0x7ffbb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbb806c8f0 0x7ffbb806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00016e0 tx=0x7ffbd0009040 comp rx=0 tx=0).stop 2024-04-26T22:00:21.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 msgr2=0x7ffbd4146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 0x7ffbd4146a80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00efc0 tx=0x7ffbcc00c490 comp rx=0 tx=0).stop 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 shutdown_connections 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbb806c8f0 0x7ffbb806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd40aef70 0x7ffbd4146a80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbd4070120 0x7ffbd41e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 --2- 172.21.15.135:0/2547518574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd406d590 0x7ffbd414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 >> 172.21.15.135:0/2547518574 conn(0x7ffbd40c5ce0 msgr2=0x7ffbd40c41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 shutdown_connections 2024-04-26T22:00:21.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.839+0000 7ffbdc6fc640 1 -- 172.21.15.135:0/2547518574 wait complete. 2024-04-26T22:00:21.850 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:21.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.950+0000 7f6ff3607640 1 Processor -- start 2024-04-26T22:00:21.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.950+0000 7f6ff3607640 1 -- start start 2024-04-26T22:00:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff3607640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6fec06d590 0x7f6fec158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff3607640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6fec070120 0x7f6fec0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff3607640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fec0af3b0 con 0x7f6fec0aef70 2024-04-26T22:00:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff3607640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec1559e0 con 0x7f6fec070120 2024-04-26T22:00:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff3607640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec154e10 con 0x7f6fec06d590 2024-04-26T22:00:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff0ba2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6fec070120 0x7f6fec0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff0ba2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6fec070120 0x7f6fec0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58022/0 (socket says 172.21.15.135:58022) 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/2187397781 learned_addr learned my addr 172.21.15.135:0/2187397781 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.951+0000 7f6ff1ba4640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f6fec0aef70 0x7f6fec0c3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46310/0 (socket says 172.21.15.135:46310) 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff13a3640 1 --2- 172.21.15.135:0/2187397781 >> v2:172.21.15.139:3300/0 conn(0x7f6fec06d590 0x7f6fec158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/2187397781 >> v2:172.21.15.139:3300/0 conn(0x7f6fec06d590 msgr2=0x7f6fec158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6fe3fff640 1 -- 172.21.15.135:0/2187397781 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175246442 0 0) 0x7f6fe4001110 con 0x7f6fec0aef70 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff0ba2640 1 --2- 172.21.15.135:0/2187397781 >> v2:172.21.15.139:3300/0 conn(0x7f6fec06d590 0x7f6fec158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/2187397781 >> v1:172.21.15.135:6789/0 conn(0x7f6fec0aef70 legacy=0x7f6fec0c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/2187397781 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec159240 con 0x7f6fec070120 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff13a3640 1 --2- 172.21.15.135:0/2187397781 >> v2:172.21.15.139:3300/0 conn(0x7f6fec06d590 0x7f6fec158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6ff0ba2640 1 --2- 172.21.15.135:0/2187397781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0aea30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0c6b90 tx=0x7f6fdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6145d7e85badcd15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6fe3fff640 1 -- 172.21.15.135:0/2187397781 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc037070 con 0x7f6fec070120 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6fe3fff640 1 -- 172.21.15.135:0/2187397781 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc032440 con 0x7f6fec070120 2024-04-26T22:00:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.952+0000 7f6fe3fff640 1 -- 172.21.15.135:0/2187397781 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc032810 con 0x7f6fec070120 2024-04-26T22:00:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 -- 172.21.15.135:0/2187397781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 msgr2=0x7f6fec0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 --2- 172.21.15.135:0/2187397781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0aea30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0c6b90 tx=0x7f6fdc02a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 -- 172.21.15.135:0/2187397781 shutdown_connections 2024-04-26T22:00:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 --2- 172.21.15.135:0/2187397781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0aea30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 --2- 172.21.15.135:0/2187397781 >> v2:172.21.15.139:3300/0 conn(0x7f6fec06d590 0x7f6fec158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 -- 172.21.15.135:0/2187397781 >> 172.21.15.135:0/2187397781 conn(0x7f6fec0c5ce0 msgr2=0x7f6fec0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 -- 172.21.15.135:0/2187397781 shutdown_connections 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.953+0000 7f6ff3607640 1 -- 172.21.15.135:0/2187397781 wait complete. 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.954+0000 7f6ff3607640 1 Processor -- start 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.954+0000 7f6ff3607640 1 -- start start 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fec06d590 0x7f6fec0c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fec0aef70 0x7f6fec157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec155670 con 0x7f6fec070120 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec155390 con 0x7f6fec06d590 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6fec06d970 con 0x7f6fec0aef70 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0c1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58026/0 (socket says 172.21.15.135:58026) 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/958073657 learned_addr learned my addr 172.21.15.135:0/958073657 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:21.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff13a3640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fec06d590 0x7f6fec0c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff1ba4640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fec0aef70 0x7f6fec157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fec0aef70 msgr2=0x7f6fec157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fec0aef70 0x7f6fec157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fec06d590 msgr2=0x7f6fec0c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fec06d590 0x7f6fec0c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 -- 172.21.15.135:0/958073657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec1572e0 con 0x7f6fec070120 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff0ba2640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0c1900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000c00 tx=0x7f6fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc037040 con 0x7f6fec070120 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc042070 con 0x7f6fec070120 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc030bb0 con 0x7f6fec070120 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.955+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec158730 con 0x7f6fec070120 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.956+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1eb180 con 0x7f6fec070120 2024-04-26T22:00:21.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.957+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec0507a0 con 0x7f6fec070120 2024-04-26T22:00:21.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.957+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fdc047460 con 0x7f6fec070120 2024-04-26T22:00:21.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.957+0000 7f6fe1ffb640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6fd006c920 0x7f6fd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:21.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.958+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f6fdc0ac950 con 0x7f6fec070120 2024-04-26T22:00:21.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.958+0000 7f6ff13a3640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6fd006c920 0x7f6fd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:21.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.958+0000 7f6ff13a3640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6fd006c920 0x7f6fd006ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800d620 tx=0x7f6fe8007810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:21.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:21.962+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fdc07a410 con 0x7f6fec070120 2024-04-26T22:00:22.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.136+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f6fec06ff00 con 0x7f6fec070120 2024-04-26T22:00:22.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.137+0000 7f6fe1ffb640 1 -- 172.21.15.135:0/958073657 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f6fdc07e710 con 0x7f6fec070120 2024-04-26T22:00:22.139 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:22.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6fd006c920 msgr2=0x7f6fd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6fd006c920 0x7f6fd006ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800d620 tx=0x7f6fe8007810 comp rx=0 tx=0).stop 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 msgr2=0x7f6fec0c1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0c1900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000c00 tx=0x7f6fdc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 shutdown_connections 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6fd006c920 0x7f6fd006ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fec0aef70 0x7f6fec157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fec070120 0x7f6fec0c1900 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 --2- 172.21.15.135:0/958073657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fec06d590 0x7f6fec0c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 >> 172.21.15.135:0/958073657 conn(0x7f6fec0c5ce0 msgr2=0x7f6fec154010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 shutdown_connections 2024-04-26T22:00:22.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.139+0000 7f6ff3607640 1 -- 172.21.15.135:0/958073657 wait complete. 2024-04-26T22:00:22.149 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.250+0000 7f0e586c8640 1 Processor -- start 2024-04-26T22:00:22.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.250+0000 7f0e586c8640 1 -- start start 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.250+0000 7f0e586c8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e500ca5a0 0x7f0e50156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.250+0000 7f0e586c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5015a1a0 0x7f0e500c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e586c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e500c8df0 con 0x7f0e50154160 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e586c8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0e500c9100 con 0x7f0e500ca5a0 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e586c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e50154540 con 0x7f0e5015a1a0 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e56c65640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5015a1a0 0x7f0e500c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e55c63640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e500ca5a0 0x7f0e50156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e56c65640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5015a1a0 0x7f0e500c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:36218/0 (socket says 172.21.15.135:36218) 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e55c63640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e500ca5a0 0x7f0e50156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:47494/0 (socket says 172.21.15.135:47494) 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e56c65640 1 -- 172.21.15.135:0/4199850611 learned_addr learned my addr 172.21.15.135:0/4199850611 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e55c63640 1 -- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5015a1a0 msgr2=0x7f0e500c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:22.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e55c63640 1 --2- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5015a1a0 0x7f0e500c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e55c63640 1 -- 172.21.15.135:0/4199850611 >> v1:172.21.15.139:6789/0 conn(0x7f0e50154160 legacy=0x7f0e500c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.251+0000 7f0e55c63640 1 -- 172.21.15.135:0/4199850611 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5015a910 con 0x7f0e500ca5a0 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.252+0000 7f0e55c63640 1 --2- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e50156bb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e400097a0 tx=0x7f0e4002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a9b0e998980ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.252+0000 7f0e55462640 1 -- 172.21.15.135:0/4199850611 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40037070 con 0x7f0e500ca5a0 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.252+0000 7f0e55462640 1 -- 172.21.15.135:0/4199850611 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e40032440 con 0x7f0e500ca5a0 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.252+0000 7f0e55462640 1 -- 172.21.15.135:0/4199850611 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40032810 con 0x7f0e500ca5a0 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.252+0000 7f0e586c8640 1 -- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 msgr2=0x7f0e50156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.252+0000 7f0e586c8640 1 --2- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e50156bb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e400097a0 tx=0x7f0e4002a180 comp rx=0 tx=0).stop 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.253+0000 7f0e586c8640 1 -- 172.21.15.135:0/4199850611 shutdown_connections 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.253+0000 7f0e586c8640 1 --2- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5015a1a0 0x7f0e500c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.253+0000 7f0e586c8640 1 --2- 172.21.15.135:0/4199850611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e50156bb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.253+0000 7f0e586c8640 1 -- 172.21.15.135:0/4199850611 >> 172.21.15.135:0/4199850611 conn(0x7f0e500ae010 msgr2=0x7f0e500ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.253+0000 7f0e586c8640 1 -- 172.21.15.135:0/4199850611 shutdown_connections 2024-04-26T22:00:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.253+0000 7f0e586c8640 1 -- 172.21.15.135:0/4199850611 wait complete. 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 Processor -- start 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 -- start start 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e501f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e50154160 0x7f0e50149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 0x7f0e501471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5015a5f0 con 0x7f0e5015a1a0 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5015aed0 con 0x7f0e500ca5a0 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e586c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e500c7510 con 0x7f0e50154160 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.254+0000 7f0e56c65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 0x7f0e501471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 0x7f0e501471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58040/0 (socket says 172.21.15.135:58040) 2024-04-26T22:00:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 -- 172.21.15.135:0/2256617872 learned_addr learned my addr 172.21.15.135:0/2256617872 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56464640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e501f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e55c63640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e50154160 0x7f0e50149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 -- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e50154160 msgr2=0x7f0e50149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e50154160 0x7f0e50149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 -- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 msgr2=0x7f0e501f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e501f08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 -- 172.21.15.135:0/2256617872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e501edee0 con 0x7f0e5015a1a0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e56c65640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 0x7f0e501471f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009cf0 tx=0x7f0e4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4800cd20 con 0x7f0e5015a1a0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e48003990 con 0x7f0e5015a1a0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48003d80 con 0x7f0e5015a1a0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e501ee1c0 con 0x7f0e5015a1a0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.255+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501ee780 con 0x7f0e5015a1a0 2024-04-26T22:00:22.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.256+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e50076650 con 0x7f0e5015a1a0 2024-04-26T22:00:22.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.257+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e48012050 con 0x7f0e5015a1a0 2024-04-26T22:00:22.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.257+0000 7f0e46ffd640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e3406c920 0x7f0e3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.257+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0e4808c160 con 0x7f0e5015a1a0 2024-04-26T22:00:22.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.257+0000 7f0e56464640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e3406c920 0x7f0e3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.257+0000 7f0e56464640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e3406c920 0x7f0e3406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c001780 tx=0x7f0e4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.261+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e48010040 con 0x7f0e5015a1a0 2024-04-26T22:00:22.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.436+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f0e500507a0 con 0x7f0e5015a1a0 2024-04-26T22:00:22.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.437+0000 7f0e46ffd640 1 -- 172.21.15.135:0/2256617872 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f0e48059ca0 con 0x7f0e5015a1a0 2024-04-26T22:00:22.437 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e3406c920 msgr2=0x7f0e3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e3406c920 0x7f0e3406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c001780 tx=0x7f0e4c009040 comp rx=0 tx=0).stop 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 msgr2=0x7f0e501471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 0x7f0e501471f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009cf0 tx=0x7f0e4800c490 comp rx=0 tx=0).stop 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 shutdown_connections 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0e3406c920 0x7f0e3406ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e5015a1a0 0x7f0e501471f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e50154160 0x7f0e50149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 --2- 172.21.15.135:0/2256617872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e500ca5a0 0x7f0e501f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 >> 172.21.15.135:0/2256617872 conn(0x7f0e500ae010 msgr2=0x7f0e5014c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 shutdown_connections 2024-04-26T22:00:22.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.439+0000 7f0e586c8640 1 -- 172.21.15.135:0/2256617872 wait complete. 2024-04-26T22:00:22.449 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:22.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.546+0000 7fe6127c1640 1 Processor -- start 2024-04-26T22:00:22.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.546+0000 7fe6127c1640 1 -- start start 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe6127c1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe60c1521f0 0x7fe60c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe6127c1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe60c070410 con 0x7fe60c06d860 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe6127c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe60c06f8b0 con 0x7fe60c0c6960 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe6127c1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe60c06f660 con 0x7fe60c1521f0 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe60c1521f0 0x7fe60c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe60c1521f0 0x7fe60c06d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58042/0 (socket says 172.21.15.135:58042) 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60bfff640 1 -- 172.21.15.135:0/2290599604 learned_addr learned my addr 172.21.15.135:0/2290599604 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe610d5e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe60c06d860 0x7fe60c15f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46326/0 (socket says 172.21.15.135:46326) 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60bfff640 1 -- 172.21.15.135:0/2290599604 >> v1:172.21.15.139:6789/0 conn(0x7fe60c0c6960 legacy=0x7fe60c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:22.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60bfff640 1 -- 172.21.15.135:0/2290599604 >> v1:172.21.15.135:6789/0 conn(0x7fe60c06d860 legacy=0x7fe60c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60affd640 1 -- 172.21.15.135:0/2290599604 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156916217 0 0) 0x7fe60c070410 con 0x7fe60c06d860 2024-04-26T22:00:22.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.547+0000 7fe60bfff640 1 -- 172.21.15.135:0/2290599604 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c06fb90 con 0x7fe60c1521f0 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe60bfff640 1 --2- 172.21.15.135:0/2290599604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c06d320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757c873eaf3615f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe60affd640 1 -- 172.21.15.135:0/2290599604 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600037070 con 0x7fe60c1521f0 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe60affd640 1 -- 172.21.15.135:0/2290599604 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe600032440 con 0x7fe60c1521f0 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe60affd640 1 -- 172.21.15.135:0/2290599604 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600032810 con 0x7fe60c1521f0 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 -- 172.21.15.135:0/2290599604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 msgr2=0x7fe60c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 --2- 172.21.15.135:0/2290599604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c06d320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002a210 comp rx=0 tx=0).stop 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 -- 172.21.15.135:0/2290599604 shutdown_connections 2024-04-26T22:00:22.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 --2- 172.21.15.135:0/2290599604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c06d320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 -- 172.21.15.135:0/2290599604 >> 172.21.15.135:0/2290599604 conn(0x7fe60c0c5ce0 msgr2=0x7fe60c0c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 -- 172.21.15.135:0/2290599604 shutdown_connections 2024-04-26T22:00:22.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.548+0000 7fe6127c1640 1 -- 172.21.15.135:0/2290599604 wait complete. 2024-04-26T22:00:22.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.549+0000 7fe6127c1640 1 Processor -- start 2024-04-26T22:00:22.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 -- start start 2024-04-26T22:00:22.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe60c06d860 0x7fe60c0c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c0c6960 0x7fe60c1f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c1f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0701b0 con 0x7fe60c1521f0 2024-04-26T22:00:22.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c070030 con 0x7fe60c0c6960 2024-04-26T22:00:22.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe6127c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe60c070c20 con 0x7fe60c06d860 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe610d5e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c1f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe610d5e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c1f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58046/0 (socket says 172.21.15.135:58046) 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.550+0000 7fe610d5e640 1 -- 172.21.15.135:0/3292993432 learned_addr learned my addr 172.21.15.135:0/3292993432 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe60b7fe640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c0c6960 0x7fe60c1f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe610d5e640 1 -- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe60c06d860 msgr2=0x7fe60c0c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe60bfff640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe60c06d860 0x7fe60c0c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe610d5e640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe60c06d860 0x7fe60c0c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe610d5e640 1 -- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c0c6960 msgr2=0x7fe60c1f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe610d5e640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c0c6960 0x7fe60c1f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe610d5e640 1 -- 172.21.15.135:0/3292993432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c149840 con 0x7fe60c1521f0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe60b7fe640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c0c6960 0x7fe60c1f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe610d5e640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c1f0660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c7d0 tx=0x7fe60400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604018070 con 0x7fe60c1521f0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60400a640 con 0x7fe60c1521f0 2024-04-26T22:00:22.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c073bb0 con 0x7fe60c1521f0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60400aa70 con 0x7fe60c1521f0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.551+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c074170 con 0x7fe60c1521f0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.552+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c074370 con 0x7fe60c1521f0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.552+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe60401b070 con 0x7fe60c1521f0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.553+0000 7fe608ff9640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5f806c920 0x7fe5f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.553+0000 7fe60bfff640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5f806c920 0x7fe5f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.553+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fe60408b480 con 0x7fe60c1521f0 2024-04-26T22:00:22.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.553+0000 7fe60bfff640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5f806c920 0x7fe5f806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe600034000 tx=0x7fe60002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.557+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe60c074370 con 0x7fe60c1521f0 2024-04-26T22:00:22.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.772+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe60c0507a0 con 0x7fe60c1521f0 2024-04-26T22:00:22.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.773+0000 7fe608ff9640 1 -- 172.21.15.135:0/3292993432 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7fe60405c970 con 0x7fe60c1521f0 2024-04-26T22:00:22.773 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:22.773 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:22.773 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5f806c920 msgr2=0x7fe5f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5f806c920 0x7fe5f806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe600034000 tx=0x7fe60002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 msgr2=0x7fe60c1f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c1f0660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c7d0 tx=0x7fe60400cc90 comp rx=0 tx=0).stop 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 shutdown_connections 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5f806c920 0x7fe5f806ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60c1521f0 0x7fe60c1f0660 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60c0c6960 0x7fe60c1f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 --2- 172.21.15.135:0/3292993432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe60c06d860 0x7fe60c0c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 >> 172.21.15.135:0/3292993432 conn(0x7fe60c0c5ce0 msgr2=0x7fe60c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 shutdown_connections 2024-04-26T22:00:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.775+0000 7fe6127c1640 1 -- 172.21.15.135:0/3292993432 wait complete. 2024-04-26T22:00:22.786 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:22.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.884+0000 7f68e88da640 1 Processor -- start 2024-04-26T22:00:22.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.884+0000 7f68e88da640 1 -- start start 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e88da640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f68e40c6e20 0x7f68e40c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e88da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68e40ca3a0 0x7f68e4157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e88da640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e415a530 con 0x7f68e40740e0 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e88da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f68e4072930 con 0x7f68e40ca3a0 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e88da640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f68e415a810 con 0x7f68e40c6e20 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68e40ca3a0 0x7f68e4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68e40ca3a0 0x7f68e4157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58050/0 (socket says 172.21.15.135:58050) 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 -- 172.21.15.135:0/4194605683 learned_addr learned my addr 172.21.15.135:0/4194605683 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e259c640 1 --2- 172.21.15.135:0/4194605683 >> v2:172.21.15.139:3300/0 conn(0x7f68e40c6e20 0x7f68e40c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 -- 172.21.15.135:0/4194605683 >> v2:172.21.15.139:3300/0 conn(0x7f68e40c6e20 msgr2=0x7f68e40c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 --2- 172.21.15.135:0/4194605683 >> v2:172.21.15.139:3300/0 conn(0x7f68e40c6e20 0x7f68e40c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 -- 172.21.15.135:0/4194605683 >> v1:172.21.15.139:6789/0 conn(0x7f68e40740e0 legacy=0x7f68e4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.885+0000 7f68e2d9d640 1 -- 172.21.15.135:0/4194605683 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e415aaf0 con 0x7f68e40ca3a0 2024-04-26T22:00:22.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.886+0000 7f68e2d9d640 1 --2- 172.21.15.135:0/4194605683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e4157630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f68e40af0b0 tx=0x7f68d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f91e48ca82d5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.888+0000 7f68e159a640 1 -- 172.21.15.135:0/4194605683 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4036070 con 0x7f68e40ca3a0 2024-04-26T22:00:22.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.888+0000 7f68e159a640 1 -- 172.21.15.135:0/4194605683 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d4031600 con 0x7f68e40ca3a0 2024-04-26T22:00:22.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.888+0000 7f68e159a640 1 -- 172.21.15.135:0/4194605683 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40319d0 con 0x7f68e40ca3a0 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 -- 172.21.15.135:0/4194605683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 msgr2=0x7f68e4157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 --2- 172.21.15.135:0/4194605683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e4157630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f68e40af0b0 tx=0x7f68d402a6a0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 -- 172.21.15.135:0/4194605683 shutdown_connections 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 --2- 172.21.15.135:0/4194605683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e4157630 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 --2- 172.21.15.135:0/4194605683 >> v2:172.21.15.139:3300/0 conn(0x7f68e40c6e20 0x7f68e40c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 -- 172.21.15.135:0/4194605683 >> 172.21.15.135:0/4194605683 conn(0x7f68e40ae0b0 msgr2=0x7f68e40ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 -- 172.21.15.135:0/4194605683 shutdown_connections 2024-04-26T22:00:22.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.889+0000 7f68e88da640 1 -- 172.21.15.135:0/4194605683 wait complete. 2024-04-26T22:00:22.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.890+0000 7f68e88da640 1 Processor -- start 2024-04-26T22:00:22.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.890+0000 7f68e88da640 1 -- start start 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68e40740e0 0x7f68e4146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68e40c6e20 0x7f68e40c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e41e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4154540 con 0x7f68e40ca3a0 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41543c0 con 0x7f68e40c6e20 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68e415a0c0 con 0x7f68e40740e0 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e41e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e41e1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58064/0 (socket says 172.21.15.135:58064) 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 -- 172.21.15.135:0/633712018 learned_addr learned my addr 172.21.15.135:0/633712018 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e259c640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68e40740e0 0x7f68e4146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e1d9b640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68e40c6e20 0x7f68e40c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 -- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68e40740e0 msgr2=0x7f68e4146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68e40740e0 0x7f68e4146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 -- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68e40c6e20 msgr2=0x7f68e40c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68e40c6e20 0x7f68e40c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 -- 172.21.15.135:0/633712018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e41efd30 con 0x7f68e40ca3a0 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e2d9d640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e41e1cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009f40 tx=0x7f68d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4036050 con 0x7f68e40ca3a0 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d4042070 con 0x7f68e40ca3a0 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e41f0010 con 0x7f68e40ca3a0 2024-04-26T22:00:22.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4009510 con 0x7f68e40ca3a0 2024-04-26T22:00:22.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.891+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41f05d0 con 0x7f68e40ca3a0 2024-04-26T22:00:22.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.892+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e40766c0 con 0x7f68e40ca3a0 2024-04-26T22:00:22.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.893+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68d4007bf0 con 0x7f68e40ca3a0 2024-04-26T22:00:22.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.893+0000 7f68d2ffd640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:22.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.893+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f68d40acbb0 con 0x7f68e40ca3a0 2024-04-26T22:00:22.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.893+0000 7f68e259c640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:22.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.894+0000 7f68e259c640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68c406c8f0 0x7f68c406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009b40 tx=0x7f68dc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:22.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:22.898+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68d407a5d0 con 0x7f68e40ca3a0 2024-04-26T22:00:23.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.073+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68e40cb9a0 con 0x7f68e40ca3a0 2024-04-26T22:00:23.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.075+0000 7f68d2ffd640 1 -- 172.21.15.135:0/633712018 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f68d407e8d0 con 0x7f68e40ca3a0 2024-04-26T22:00:23.075 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:23.075 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68c406c8f0 msgr2=0x7f68c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68c406c8f0 0x7f68c406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009b40 tx=0x7f68dc009190 comp rx=0 tx=0).stop 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 msgr2=0x7f68e41e1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e41e1cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009f40 tx=0x7f68d4004a40 comp rx=0 tx=0).stop 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 shutdown_connections 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68e40ca3a0 0x7f68e41e1cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68e40c6e20 0x7f68e40c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 --2- 172.21.15.135:0/633712018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68e40740e0 0x7f68e4146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 >> 172.21.15.135:0/633712018 conn(0x7f68e40ae0b0 msgr2=0x7f68e40ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 shutdown_connections 2024-04-26T22:00:23.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.077+0000 7f68e88da640 1 -- 172.21.15.135:0/633712018 wait complete. 2024-04-26T22:00:23.088 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:23.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.185+0000 7f480683e640 1 Processor -- start 2024-04-26T22:00:23.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.185+0000 7f480683e640 1 -- start start 2024-04-26T22:00:23.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.185+0000 7f480683e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48000acc30 0x7f480015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.185+0000 7f480683e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000c4250 0x7f48000c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f480683e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f48000c3cd0 0x7f48001572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f480683e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f48000c8ba0 con 0x7f48000acc30 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f480683e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4800156e50 con 0x7f48000c3cd0 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f480683e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f48000ad010 con 0x7f48000c4250 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48000acc30 0x7f480015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48000acc30 0x7f480015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58070/0 (socket says 172.21.15.135:58070) 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 -- 172.21.15.135:0/2831790947 learned_addr learned my addr 172.21.15.135:0/2831790947 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47ff7fe640 1 --2- 172.21.15.135:0/2831790947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000c4250 0x7f48000c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f4804ddb640 1 --2- 172.21.15.135:0/2831790947 >> v2:172.21.15.139:3300/0 conn(0x7f48000c3cd0 0x7f48001572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 -- 172.21.15.135:0/2831790947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000c4250 msgr2=0x7f48000c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 --2- 172.21.15.135:0/2831790947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000c4250 0x7f48000c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 -- 172.21.15.135:0/2831790947 >> v2:172.21.15.139:3300/0 conn(0x7f48000c3cd0 msgr2=0x7f48001572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 --2- 172.21.15.135:0/2831790947 >> v2:172.21.15.139:3300/0 conn(0x7f48000c3cd0 0x7f48001572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.186+0000 7f47fffff640 1 -- 172.21.15.135:0/2831790947 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480015b130 con 0x7f48000acc30 2024-04-26T22:00:23.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f47fffff640 1 --2- 172.21.15.135:0/2831790947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000acc30 0x7f480015abf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4800074f70 tx=0x7f47f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e942aa1c8aee5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f47feffd640 1 -- 172.21.15.135:0/2831790947 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0037070 con 0x7f48000acc30 2024-04-26T22:00:23.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f47feffd640 1 -- 172.21.15.135:0/2831790947 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f0032440 con 0x7f48000acc30 2024-04-26T22:00:23.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f47feffd640 1 -- 172.21.15.135:0/2831790947 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0032810 con 0x7f48000acc30 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 -- 172.21.15.135:0/2831790947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000acc30 msgr2=0x7f480015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 --2- 172.21.15.135:0/2831790947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000acc30 0x7f480015abf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4800074f70 tx=0x7f47f002a140 comp rx=0 tx=0).stop 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 -- 172.21.15.135:0/2831790947 shutdown_connections 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 --2- 172.21.15.135:0/2831790947 >> v2:172.21.15.139:3300/0 conn(0x7f48000c3cd0 0x7f48001572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 --2- 172.21.15.135:0/2831790947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000c4250 0x7f48000c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 --2- 172.21.15.135:0/2831790947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000acc30 0x7f480015abf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 -- 172.21.15.135:0/2831790947 >> 172.21.15.135:0/2831790947 conn(0x7f48000c6e60 msgr2=0x7f48000c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 -- 172.21.15.135:0/2831790947 shutdown_connections 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.187+0000 7f480683e640 1 -- 172.21.15.135:0/2831790947 wait complete. 2024-04-26T22:00:23.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.188+0000 7f480683e640 1 Processor -- start 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 -- start start 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000acc30 0x7f480006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48000c3cd0 0x7f48000ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 0x7f4800158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48000c88c0 con 0x7f48000c4250 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48000c85e0 con 0x7f48000c3cd0 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f480683e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f48000abec0 con 0x7f48000acc30 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f4804ddb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 0x7f4800158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f4804ddb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 0x7f4800158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58076/0 (socket says 172.21.15.135:58076) 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.189+0000 7f4804ddb640 1 -- 172.21.15.135:0/1353123643 learned_addr learned my addr 172.21.15.135:0/1353123643 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f47fffff640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000acc30 0x7f480006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f47ff7fe640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48000c3cd0 0x7f48000ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f4804ddb640 1 -- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000acc30 msgr2=0x7f480006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f4804ddb640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000acc30 0x7f480006c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f4804ddb640 1 -- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48000c3cd0 msgr2=0x7f48000ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f4804ddb640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48000c3cd0 0x7f48000ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f4804ddb640 1 -- 172.21.15.135:0/1353123643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48000ada70 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f4804ddb640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 0x7f4800158260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47f8007e60 tx=0x7f47f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8014620 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f8019070 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48000ae550 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8014a20 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.190+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48000aeb10 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.191+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48000507a0 con 0x7f48000c4250 2024-04-26T22:00:23.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.191+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47f8014bc0 con 0x7f48000c4250 2024-04-26T22:00:23.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.192+0000 7f47fcff9640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47ec06c8f0 0x7f47ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.192+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f47f808c7b0 con 0x7f48000c4250 2024-04-26T22:00:23.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.192+0000 7f47fffff640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47ec06c8f0 0x7f47ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.192+0000 7f47fffff640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47ec06c8f0 0x7f47ec06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47f0034000 tx=0x7f47f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.196+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47f805a820 con 0x7f48000c4250 2024-04-26T22:00:23.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.407+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48000c46f0 con 0x7f48000c4250 2024-04-26T22:00:23.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.408+0000 7f47fcff9640 1 -- 172.21.15.135:0/1353123643 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f47f805a120 con 0x7f48000c4250 2024-04-26T22:00:23.407 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:23.408 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:23.408 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47ec06c8f0 msgr2=0x7f47ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47ec06c8f0 0x7f47ec06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47f0034000 tx=0x7f47f002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 msgr2=0x7f4800158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 0x7f4800158260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47f8007e60 tx=0x7f47f800e4b0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 shutdown_connections 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47ec06c8f0 0x7f47ec06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48000c4250 0x7f4800158260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48000c3cd0 0x7f48000ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 --2- 172.21.15.135:0/1353123643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48000acc30 0x7f480006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 >> 172.21.15.135:0/1353123643 conn(0x7f48000c6e60 msgr2=0x7f480006e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 shutdown_connections 2024-04-26T22:00:23.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.410+0000 7f480683e640 1 -- 172.21.15.135:0/1353123643 wait complete. 2024-04-26T22:00:23.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:59:56.985345+0000', 'modified': '2024-04-26T22:00:00.110574+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6837/1410523687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6835', 'nonce': 1410523687}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 1410523687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'a', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4278, 'qdb_cluster': [4278]} 2024-04-26T22:00:23.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:00:23.823 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T22:00:23.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.924+0000 7f66ad92b640 1 Processor -- start 2024-04-26T22:00:23.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.924+0000 7f66ad92b640 1 -- start start 2024-04-26T22:00:23.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66ad92b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66a80c3c70 0x7f66a80c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66ad92b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a815a230 con 0x7f66a8157010 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66ad92b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a815a510 con 0x7f66a813fe20 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66ad92b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f66a80c39b0 con 0x7f66a80c3c70 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66a67fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66a80c3c70 0x7f66a80c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66a67fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66a80c3c70 0x7f66a80c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58092/0 (socket says 172.21.15.135:58092) 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66a67fc640 1 -- 172.21.15.135:0/1555464753 learned_addr learned my addr 172.21.15.135:0/1555464753 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.925+0000 7f66a77fe640 1 --1- 172.21.15.135:0/1555464753 >> v1:172.21.15.135:6789/0 conn(0x7f66a8157010 0x7f66a8157440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46328/0 (socket says 172.21.15.135:46328) 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a67fc640 1 -- 172.21.15.135:0/1555464753 >> v1:172.21.15.139:6789/0 conn(0x7f66a813fe20 legacy=0x7f66a806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a67fc640 1 -- 172.21.15.135:0/1555464753 >> v1:172.21.15.135:6789/0 conn(0x7f66a8157010 legacy=0x7f66a8157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a67fc640 1 -- 172.21.15.135:0/1555464753 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a815a7f0 con 0x7f66a80c3c70 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a67fc640 1 --2- 172.21.15.135:0/1555464753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a80c3c70 0x7f66a80c5430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f669c009870 tx=0x7f669c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acbff8622ba0d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a5ffb640 1 -- 172.21.15.135:0/1555464753 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c037070 con 0x7f66a80c3c70 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a5ffb640 1 -- 172.21.15.135:0/1555464753 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f669c02ae50 con 0x7f66a80c3c70 2024-04-26T22:00:23.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.926+0000 7f66a5ffb640 1 -- 172.21.15.135:0/1555464753 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c032860 con 0x7f66a80c3c70 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 -- 172.21.15.135:0/1555464753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a80c3c70 msgr2=0x7f66a80c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1555464753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a80c3c70 0x7f66a80c5430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f669c009870 tx=0x7f669c02a300 comp rx=0 tx=0).stop 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 -- 172.21.15.135:0/1555464753 shutdown_connections 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1555464753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a80c3c70 0x7f66a80c5430 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 -- 172.21.15.135:0/1555464753 >> 172.21.15.135:0/1555464753 conn(0x7f66a80c5d70 msgr2=0x7f66a80c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 -- 172.21.15.135:0/1555464753 shutdown_connections 2024-04-26T22:00:23.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.927+0000 7f66ad92b640 1 -- 172.21.15.135:0/1555464753 wait complete. 2024-04-26T22:00:23.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.928+0000 7f66ad92b640 1 Processor -- start 2024-04-26T22:00:23.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.928+0000 7f66ad92b640 1 -- start start 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66a80c3c70 0x7f66a806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66a813fe20 0x7f66a81efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 0x7f66a81f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a815adb0 con 0x7f66a8157010 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66a815b120 con 0x7f66a813fe20 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f66a8157b50 con 0x7f66a80c3c70 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 0x7f66a81f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 0x7f66a81f0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58106/0 (socket says 172.21.15.135:58106) 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 -- 172.21.15.135:0/1211922793 learned_addr learned my addr 172.21.15.135:0/1211922793 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66a80c3c70 0x7f66a806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a67fc640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66a813fe20 0x7f66a81efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 -- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66a80c3c70 msgr2=0x7f66a806fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66a80c3c70 0x7f66a806fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 -- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66a813fe20 msgr2=0x7f66a81efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66a813fe20 0x7f66a81efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 -- 172.21.15.135:0/1211922793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a813c040 con 0x7f66a8157010 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66a77fe640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 0x7f66a81f0210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66a0001eb0 tx=0x7f66a000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f668ffff640 1 -- 172.21.15.135:0/1211922793 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00074e0 con 0x7f66a8157010 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f668ffff640 1 -- 172.21.15.135:0/1211922793 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0007680 con 0x7f66a8157010 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a80746d0 con 0x7f66a8157010 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f668ffff640 1 -- 172.21.15.135:0/1211922793 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00049e0 con 0x7f66a8157010 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.929+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a813e530 con 0x7f66a8157010 2024-04-26T22:00:23.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.931+0000 7f668ffff640 1 -- 172.21.15.135:0/1211922793 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66a001a020 con 0x7f66a8157010 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.931+0000 7f668ffff640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f669406c920 0x7f669406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.932+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f669406c920 0x7f669406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.932+0000 7f668ffff640 1 -- 172.21.15.135:0/1211922793 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f66a008b410 con 0x7f66a8157010 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.932+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f669406c920 0x7f669406ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6698001f10 tx=0x7f6698009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.932+0000 7f66b0649740 1 Processor -- start 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.932+0000 7f66b0649740 1 -- start start 2024-04-26T22:00:23.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66b0649740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e3400 0x5559a95dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95c6760 0x5559a95e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66b0649740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5559a9509740 con 0x5559a95e9ac0 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5559a94a53a0 con 0x5559a95c6760 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x5559a9505700 con 0x5559a95e3400 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95e9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58116/0 (socket says 172.21.15.135:58116) 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.933+0000 7f66a6ffd640 1 -- 172.21.15.135:0/1407279928 learned_addr learned my addr 172.21.15.135:0/1407279928 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a6ffd640 1 -- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e3400 msgr2=0x5559a95dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a77fe640 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95c6760 0x5559a95e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e3400 0x5559a95dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a6ffd640 1 -- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95c6760 msgr2=0x5559a95e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95c6760 0x5559a95e04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a6ffd640 1 -- 172.21.15.135:0/1407279928 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a94c1750 con 0x5559a95e9ac0 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95e9ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6698007f40 tx=0x7f6698012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398440a3cd4f59b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f668d7fa640 1 -- 172.21.15.135:0/1407279928 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698026070 con 0x5559a95e9ac0 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f668d7fa640 1 -- 172.21.15.135:0/1407279928 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66980077c0 con 0x5559a95e9ac0 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f668d7fa640 1 -- 172.21.15.135:0/1407279928 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980133f0 con 0x5559a95e9ac0 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66b0649740 1 -- 172.21.15.135:0/1407279928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 msgr2=0x5559a95e9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.934+0000 7f66b0649740 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95e9ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6698007f40 tx=0x7f6698012000 comp rx=0 tx=0).stop 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 -- 172.21.15.135:0/1407279928 shutdown_connections 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95c6760 0x5559a95e04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e3400 0x5559a95dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 --2- 172.21.15.135:0/1407279928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95e9ea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 -- 172.21.15.135:0/1407279928 >> 172.21.15.135:0/1407279928 conn(0x5559a95a1930 msgr2=0x5559a95c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 -- 172.21.15.135:0/1407279928 shutdown_connections 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 -- 172.21.15.135:0/1407279928 wait complete. 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 -- start start 2024-04-26T22:00:23.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.935+0000 7f66b0649740 1 Processor -- start 2024-04-26T22:00:23.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e9ac0 0x5559a95c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 0x5559a95c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5559a95a0b30 con 0x5559a95c1420 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5559a9490fe0 con 0x5559a95e3400 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x5559a95ea3e0 con 0x5559a95e9ac0 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66b0649740 10 client.0 _ll_register_callbacks cb 0x5559a94a7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 0x5559a95c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 0x5559a95c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58124/0 (socket says 172.21.15.135:58124) 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 -- 172.21.15.135:0/3534149909 learned_addr learned my addr 172.21.15.135:0/3534149909 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a67fc640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e9ac0 0x5559a95c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 -- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e9ac0 msgr2=0x5559a95c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e9ac0 0x5559a95c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 -- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 msgr2=0x5559a95c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 -- 172.21.15.135:0/3534149909 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a921b2d0 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66a77fe640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 0x5559a95c1880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66a0058c30 tx=0x7f66a0059070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e3ba8b644cae67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.936+0000 7f66867fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66867fc640 1 -- 172.21.15.135:0/3534149909 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0059190 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66867fc640 1 -- 172.21.15.135:0/3534149909 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f66a0059330 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66980077c0 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66867fc640 1 -- 172.21.15.135:0/3534149909 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f66a00615d0 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 10 client.4833 fetch_fsmap learned FSMap version 13 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x5559a92e56b0 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66867fc640 1 -- 172.21.15.135:0/3534149909 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66980077c0 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66867fc640 1 -- 172.21.15.135:0/3534149909 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 13) ==== 4128+0+0 (secure 0 0 0) 0x7f66a0004b80 con 0x5559a95c1420 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 10 client.4833 fetch_fsmap finished waiting for FSMap version 13 2024-04-26T22:00:23.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 10 client.4833 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 20 client.4833 populate_metadata read hostname 'smithi135' 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a95f5010 0x5559a95bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 4 client.4833 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 --> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559a9517350 con 0x5559a95f5010 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a95f5010 0x5559a95bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a95f5010 0x5559a95bf2f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519860358f488a42 server_cookie=672186ca77ac4811 in_seq=0 out_seq=0 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.937+0000 7f66867fc640 10 client.4833 ms_handle_connect on v2:172.21.15.135:6835/1410523687 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66867fc640 1 -- 172.21.15.135:0/3534149909 <== mds.0 v2:172.21.15.135:6835/1410523687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559a9517350 con 0x5559a95f5010 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66867fc640 10 client.4833 handle_command_reply: tid=0 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 client.4833 shutdown 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 msgr2=0x5559a95c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 0x5559a95c1880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66a0058c30 tx=0x7f66a0059070 comp rx=0 tx=0).stop 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 shutdown_connections 2024-04-26T22:00:23.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a95f5010 0x5559a95bf2f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95c1420 0x5559a95c1880 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66867fc640 0 client.4833 ms_handle_reset on v2:172.21.15.135:6835/1410523687 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95e9ac0 0x5559a95c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 --2- 172.21.15.135:0/3534149909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 >> 172.21.15.135:0/3534149909 conn(0x5559a95c6760 msgr2=0x5559a95e4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 shutdown_connections 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 1 -- 172.21.15.135:0/3534149909 wait complete. 2024-04-26T22:00:23.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.938+0000 7f66b0649740 20 client.4833 trim_cache size 0 max 16384 2024-04-26T22:00:23.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 Processor -- start 2024-04-26T22:00:23.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 -- start start 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81fc320 con 0x5559a95e9ac0 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81f9f40 con 0x5559a95e3400 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f66a81fa3e0 con 0x5559a95c1420 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.956+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95bf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58132/0 (socket says 172.21.15.135:58132) 2024-04-26T22:00:23.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 -- 172.21.15.135:0/877625882 learned_addr learned my addr 172.21.15.135:0/877625882 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a67fc640 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 -- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 msgr2=0x5559a95e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 -- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 msgr2=0x5559a95e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 -- 172.21.15.135:0/877625882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a94be7f0 con 0x5559a95e9ac0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66a77fe640 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95bf2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66a00571e0 tx=0x7f66a005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752fc97653e9a034 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f668cff9640 1 -- 172.21.15.135:0/877625882 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00551d0 con 0x5559a95e9ac0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f668cff9640 1 -- 172.21.15.135:0/877625882 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a008b110 con 0x5559a95e9ac0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f668cff9640 1 -- 172.21.15.135:0/877625882 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0061810 con 0x5559a95e9ac0 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.957+0000 7f66b0649740 1 -- 172.21.15.135:0/877625882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 msgr2=0x5559a95bf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95bf2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66a00571e0 tx=0x7f66a005b000 comp rx=0 tx=0).stop 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 -- 172.21.15.135:0/877625882 shutdown_connections 2024-04-26T22:00:23.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a95bf2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 --2- 172.21.15.135:0/877625882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 -- 172.21.15.135:0/877625882 >> 172.21.15.135:0/877625882 conn(0x5559a95c6760 msgr2=0x5559a95c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 -- 172.21.15.135:0/877625882 shutdown_connections 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 -- 172.21.15.135:0/877625882 wait complete. 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 -- start start 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.958+0000 7f66b0649740 1 Processor -- start 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95da7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a9667fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5559a95bff80 con 0x5559a95e9ac0 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5559a95e7830 con 0x5559a95e3400 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x5559a95bedc0 con 0x5559a95c1420 2024-04-26T22:00:23.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 10 client.0 _ll_register_callbacks cb 0x7f66a0058d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a9667fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a9667fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58142/0 (socket says 172.21.15.135:58142) 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 -- 172.21.15.135:0/2806516402 learned_addr learned my addr 172.21.15.135:0/2806516402 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a67fc640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95da7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 -- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 msgr2=0x5559a95da7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95da7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 -- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 msgr2=0x5559a95e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 -- 172.21.15.135:0/2806516402 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a96684e0 con 0x5559a95e9ac0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66a77fe640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a9667fa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66a000cd20 tx=0x7f66a0089550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d652b20245233d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66867fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66867fc640 1 -- 172.21.15.135:0/2806516402 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00551d0 con 0x5559a95e9ac0 2024-04-26T22:00:23.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66867fc640 1 -- 172.21.15.135:0/2806516402 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f66a0059190 con 0x5559a95e9ac0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.959+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5559a95111c0 con 0x5559a95e9ac0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66867fc640 1 -- 172.21.15.135:0/2806516402 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f66a0059390 con 0x5559a95e9ac0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 10 client.4839 fetch_fsmap learned FSMap version 13 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x5559a96684e0 con 0x5559a95e9ac0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66867fc640 1 -- 172.21.15.135:0/2806516402 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5559a95111c0 con 0x5559a95e9ac0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66867fc640 1 -- 172.21.15.135:0/2806516402 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 13) ==== 4128+0+0 (secure 0 0 0) 0x7f66a00782d0 con 0x5559a95e9ac0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 10 client.4839 fetch_fsmap finished waiting for FSMap version 13 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 10 client.4839 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 20 client.4839 populate_metadata read hostname 'smithi135' 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a965e7e0 0x5559a95a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 4 client.4839 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 --> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] -- command(tid 0: {"prefix": "status"}) -- 0x5559a948c9a0 con 0x5559a965e7e0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a965e7e0 0x5559a95a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66867fc640 10 client.4839 ms_handle_connect on v2:172.21.15.135:6835/1410523687 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66a6ffd640 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a965e7e0 0x5559a95a4890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62521fc8723c6b2 server_cookie=ce91dd60da76056b in_seq=0 out_seq=0 2024-04-26T22:00:23.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66867fc640 1 -- 172.21.15.135:0/2806516402 <== mds.0 v2:172.21.15.135:6835/1410523687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5559a948c9a0 con 0x5559a965e7e0 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66867fc640 10 client.4839 handle_command_reply: tid=0 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.960+0000 7f66b0649740 1 client.4839 shutdown 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 msgr2=0x5559a9667fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a9667fa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66a000cd20 tx=0x7f66a0089550 comp rx=0 tx=0).stop 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 shutdown_connections 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:6835/1410523687,v1:172.21.15.135:6837/1410523687] conn(0x5559a965e7e0 0x5559a95a4890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5559a95e9ac0 0x5559a9667fa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66867fc640 0 client.4839 ms_handle_reset on v2:172.21.15.135:6835/1410523687 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5559a95e3400 0x5559a95e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 --2- 172.21.15.135:0/2806516402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5559a95c1420 0x5559a95da7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 >> 172.21.15.135:0/2806516402 conn(0x5559a95c6760 msgr2=0x5559a95c2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 shutdown_connections 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 1 -- 172.21.15.135:0/2806516402 wait complete. 2024-04-26T22:00:23.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66b0649740 20 client.4839 trim_cache size 0 max 16384 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_fsid": "c1882db8-5327-4bd0-9178-0a7c5b1ec94f", 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "whoami": 0, 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "id": 4278, 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "want_state": "up:active", 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "state": "up:active", 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "a", 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "rank_uptime": 26.318447703, 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "mdsmap_epoch": 8, 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch": 24, 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch_barrier": 20, 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout: "uptime": 33.360195195999999 2024-04-26T22:00:23.968 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:00:23.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f669406c920 msgr2=0x7f669406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f669406c920 0x7f669406ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6698001f10 tx=0x7f6698009040 comp rx=0 tx=0).stop 2024-04-26T22:00:23.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 msgr2=0x7f66a81f0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:23.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 0x7f66a81f0210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66a0001eb0 tx=0x7f66a000c820 comp rx=0 tx=0).stop 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 shutdown_connections 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f669406c920 0x7f669406ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8157010 0x7f66a81f0210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66a813fe20 0x7f66a81efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 --2- 172.21.15.135:0/1211922793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66a80c3c70 0x7f66a806fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 >> 172.21.15.135:0/1211922793 conn(0x7f66a80c5d70 msgr2=0x7f66a8158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 shutdown_connections 2024-04-26T22:00:23.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:23.961+0000 7f66ad92b640 1 -- 172.21.15.135:0/1211922793 wait complete. 2024-04-26T22:00:23.972 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:24.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.069+0000 7f814a893640 1 Processor -- start 2024-04-26T22:00:24.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.069+0000 7f814a893640 1 -- start start 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f814a893640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8144154160 0x7f8144156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f814a893640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f81440ca5a0 0x7f814415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f814a893640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81440c6e00 0x7f81440c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f814a893640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f814415a6e0 con 0x7f81440ca5a0 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f814a893640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f81440c9100 con 0x7f8144154160 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f814a893640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81440c8df0 con 0x7f81440c6e00 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f81437fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f81440ca5a0 0x7f814415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f81437fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f81440ca5a0 0x7f814415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58146/0 (socket says 172.21.15.135:58146) 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f81437fe640 1 -- 172.21.15.135:0/2772978625 learned_addr learned my addr 172.21.15.135:0/2772978625 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f8148e30640 1 --2- 172.21.15.135:0/2772978625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81440c6e00 0x7f81440c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.070+0000 7f8143fff640 1 --2- 172.21.15.135:0/2772978625 >> v2:172.21.15.139:3300/0 conn(0x7f8144154160 0x7f8144156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f81437fe640 1 -- 172.21.15.135:0/2772978625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81440c6e00 msgr2=0x7f81440c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f81437fe640 1 --2- 172.21.15.135:0/2772978625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81440c6e00 0x7f81440c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f81437fe640 1 -- 172.21.15.135:0/2772978625 >> v2:172.21.15.139:3300/0 conn(0x7f8144154160 msgr2=0x7f8144156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f81437fe640 1 --2- 172.21.15.135:0/2772978625 >> v2:172.21.15.139:3300/0 conn(0x7f8144154160 0x7f8144156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f81437fe640 1 -- 172.21.15.135:0/2772978625 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8144154540 con 0x7f81440ca5a0 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f8143fff640 1 --2- 172.21.15.135:0/2772978625 >> v2:172.21.15.139:3300/0 conn(0x7f8144154160 0x7f8144156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f81437fe640 1 --2- 172.21.15.135:0/2772978625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440ca5a0 0x7f814415a1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f814406da80 tx=0x7f813802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae9b5bdff9078d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f8142ffd640 1 -- 172.21.15.135:0/2772978625 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138037070 con 0x7f81440ca5a0 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f8142ffd640 1 -- 172.21.15.135:0/2772978625 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8138032440 con 0x7f81440ca5a0 2024-04-26T22:00:24.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.071+0000 7f8142ffd640 1 -- 172.21.15.135:0/2772978625 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138032810 con 0x7f81440ca5a0 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 -- 172.21.15.135:0/2772978625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440ca5a0 msgr2=0x7f814415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 --2- 172.21.15.135:0/2772978625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440ca5a0 0x7f814415a1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f814406da80 tx=0x7f813802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 -- 172.21.15.135:0/2772978625 shutdown_connections 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 --2- 172.21.15.135:0/2772978625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440ca5a0 0x7f814415a1a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 --2- 172.21.15.135:0/2772978625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81440c6e00 0x7f81440c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 --2- 172.21.15.135:0/2772978625 >> v2:172.21.15.139:3300/0 conn(0x7f8144154160 0x7f8144156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 -- 172.21.15.135:0/2772978625 >> 172.21.15.135:0/2772978625 conn(0x7f81440ae010 msgr2=0x7f81440ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 -- 172.21.15.135:0/2772978625 shutdown_connections 2024-04-26T22:00:24.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.072+0000 7f814a893640 1 -- 172.21.15.135:0/2772978625 wait complete. 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.073+0000 7f814a893640 1 Processor -- start 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.073+0000 7f814a893640 1 -- start start 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f814a893640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 0x7f81440b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f814a893640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81440ca5a0 0x7f81440cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f814a893640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8144154160 0x7f81440cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f814a893640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f814415aed0 con 0x7f81440c6e00 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f814a893640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f814415abf0 con 0x7f81440ca5a0 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f814a893640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81440c9590 con 0x7f8144154160 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 0x7f81440b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 0x7f81440b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58152/0 (socket says 172.21.15.135:58152) 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 -- 172.21.15.135:0/2252678979 learned_addr learned my addr 172.21.15.135:0/2252678979 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:24.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8148e30640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8144154160 0x7f81440cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f81437fe640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81440ca5a0 0x7f81440cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 -- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8144154160 msgr2=0x7f81440cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8144154160 0x7f81440cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 -- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81440ca5a0 msgr2=0x7f81440cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81440ca5a0 0x7f81440cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 -- 172.21.15.135:0/2252678979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8144158100 con 0x7f81440c6e00 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8148e30640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8144154160 0x7f81440cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.074+0000 7f8143fff640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 0x7f81440b1bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f813400d7d0 tx=0x7f813400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.075+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134004030 con 0x7f81440c6e00 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.075+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81340041d0 con 0x7f81440c6e00 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.075+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81441f0230 con 0x7f81440c6e00 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.075+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813400b6c0 con 0x7f81440c6e00 2024-04-26T22:00:24.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.075+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81441ed080 con 0x7f81440c6e00 2024-04-26T22:00:24.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.076+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81441ecb90 con 0x7f81440c6e00 2024-04-26T22:00:24.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.076+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f813400b860 con 0x7f81440c6e00 2024-04-26T22:00:24.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.076+0000 7f8140ff9640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f813006c8f0 0x7f813006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.077+0000 7f81437fe640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f813006c8f0 0x7f813006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.077+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f813408c090 con 0x7f81440c6e00 2024-04-26T22:00:24.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.077+0000 7f81437fe640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f813006c8f0 0x7f813006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8138034000 tx=0x7f813802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.081+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8144158100 con 0x7f81440c6e00 2024-04-26T22:00:24.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.255+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8144076650 con 0x7f81440c6e00 2024-04-26T22:00:24.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.256+0000 7f8140ff9640 1 -- 172.21.15.135:0/2252678979 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f813405a220 con 0x7f81440c6e00 2024-04-26T22:00:24.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:24.256 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:24.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f813006c8f0 msgr2=0x7f813006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f813006c8f0 0x7f813006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8138034000 tx=0x7f813802e000 comp rx=0 tx=0).stop 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 msgr2=0x7f81440b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 0x7f81440b1bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f813400d7d0 tx=0x7f813400dc90 comp rx=0 tx=0).stop 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 shutdown_connections 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f813006c8f0 0x7f813006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8144154160 0x7f81440cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81440ca5a0 0x7f81440cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 --2- 172.21.15.135:0/2252678979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81440c6e00 0x7f81440b1bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.258+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 >> 172.21.15.135:0/2252678979 conn(0x7f81440ae010 msgr2=0x7f814414ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.259+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 shutdown_connections 2024-04-26T22:00:24.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.259+0000 7f814a893640 1 -- 172.21.15.135:0/2252678979 wait complete. 2024-04-26T22:00:24.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:24.270 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:24.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.370+0000 7f2fe294e640 1 Processor -- start 2024-04-26T22:00:24.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.370+0000 7f2fe294e640 1 -- start start 2024-04-26T22:00:24.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe294e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0aecf0 0x7f2fdc0c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe294e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2fdc0c4820 0x7f2fdc073a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe294e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2fdc0c3d40 0x7f2fdc073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe294e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc156f60 con 0x7f2fdc0c3d40 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe294e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc157240 con 0x7f2fdc0c4820 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe294e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc157520 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2fdc0c3d40 0x7f2fdc073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2fdc0c3d40 0x7f2fdc073fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58166/0 (socket says 172.21.15.135:58166) 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 -- 172.21.15.135:0/4265369768 learned_addr learned my addr 172.21.15.135:0/4265369768 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fdb7fe640 1 --2- 172.21.15.135:0/4265369768 >> v2:172.21.15.139:3300/0 conn(0x7f2fdc0c4820 0x7f2fdc073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fdbfff640 1 --2- 172.21.15.135:0/4265369768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0aecf0 0x7f2fdc0c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 -- 172.21.15.135:0/4265369768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0aecf0 msgr2=0x7f2fdc0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 --2- 172.21.15.135:0/4265369768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0aecf0 0x7f2fdc0c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 -- 172.21.15.135:0/4265369768 >> v2:172.21.15.139:3300/0 conn(0x7f2fdc0c4820 msgr2=0x7f2fdc073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:24.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 --2- 172.21.15.135:0/4265369768 >> v2:172.21.15.139:3300/0 conn(0x7f2fdc0c4820 0x7f2fdc073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.371+0000 7f2fe0eeb640 1 -- 172.21.15.135:0/4265369768 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc157800 con 0x7f2fdc0c3d40 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe0eeb640 1 --2- 172.21.15.135:0/4265369768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc073fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0cb370 tx=0x7f2fd402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3a9b11104e4b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fdaffd640 1 -- 172.21.15.135:0/4265369768 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4036070 con 0x7f2fdc0c3d40 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fdaffd640 1 -- 172.21.15.135:0/4265369768 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd4031600 con 0x7f2fdc0c3d40 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fdaffd640 1 -- 172.21.15.135:0/4265369768 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40319d0 con 0x7f2fdc0c3d40 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 -- 172.21.15.135:0/4265369768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0c3d40 msgr2=0x7f2fdc073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 --2- 172.21.15.135:0/4265369768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc073fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0cb370 tx=0x7f2fd402a610 comp rx=0 tx=0).stop 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 -- 172.21.15.135:0/4265369768 shutdown_connections 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 --2- 172.21.15.135:0/4265369768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc073fb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 --2- 172.21.15.135:0/4265369768 >> v2:172.21.15.139:3300/0 conn(0x7f2fdc0c4820 0x7f2fdc073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 --2- 172.21.15.135:0/4265369768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0aecf0 0x7f2fdc0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 -- 172.21.15.135:0/4265369768 >> 172.21.15.135:0/4265369768 conn(0x7f2fdc0c6ed0 msgr2=0x7f2fdc0c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 -- 172.21.15.135:0/4265369768 shutdown_connections 2024-04-26T22:00:24.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.372+0000 7f2fe294e640 1 -- 172.21.15.135:0/4265369768 wait complete. 2024-04-26T22:00:24.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.373+0000 7f2fe294e640 1 Processor -- start 2024-04-26T22:00:24.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 -- start start 2024-04-26T22:00:24.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 0x7f2fdc14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0c4820 0x7f2fdc149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0af0d0 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc156c10 con 0x7f2fdc0c3d40 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe294e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0af250 con 0x7f2fdc0c4820 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 0x7f2fdc14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 0x7f2fdc14f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58170/0 (socket says 172.21.15.135:58170) 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fdbfff640 1 -- 172.21.15.135:0/1687388609 learned_addr learned my addr 172.21.15.135:0/1687388609 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.374+0000 7f2fe0eeb640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0c4820 0x7f2fdc149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdb7fe640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdbfff640 1 -- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0c4820 msgr2=0x7f2fdc149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdbfff640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0c4820 0x7f2fdc149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdbfff640 1 -- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc0c3d40 msgr2=0x7f2fdc149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdbfff640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdbfff640 1 -- 172.21.15.135:0/1687388609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc1ef8b0 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fdbfff640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 0x7f2fdc14f9a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c820 tx=0x7f2fd000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0015070 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd0004220 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc1efb90 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000ce10 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.375+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1f0150 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.376+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc0507a0 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.377+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fd0007830 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.377+0000 7f2fd8ff9640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2fc806c8f0 0x7f2fc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.377+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f2fd008bbe0 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.377+0000 7f2fdb7fe640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2fc806c8f0 0x7f2fc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.377+0000 7f2fdb7fe640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2fc806c8f0 0x7f2fc806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0ca2b0 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.381+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fd0059690 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.555+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fdc074700 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.556+0000 7f2fd8ff9640 1 -- 172.21.15.135:0/1687388609 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f2fd005d990 con 0x7f2fdc0aecf0 2024-04-26T22:00:24.556 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:24.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2fc806c8f0 msgr2=0x7f2fc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2fc806c8f0 0x7f2fc806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0ca2b0 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 msgr2=0x7f2fdc14f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 0x7f2fdc14f9a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c820 tx=0x7f2fd000cce0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 shutdown_connections 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2fc806c8f0 0x7f2fc806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fdc0c4820 0x7f2fdc149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc0c3d40 0x7f2fdc149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 --2- 172.21.15.135:0/1687388609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fdc0aecf0 0x7f2fdc14f9a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.557+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 >> 172.21.15.135:0/1687388609 conn(0x7f2fdc0c6ed0 msgr2=0x7f2fdc0ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.558+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 shutdown_connections 2024-04-26T22:00:24.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.558+0000 7f2fe294e640 1 -- 172.21.15.135:0/1687388609 wait complete. 2024-04-26T22:00:24.569 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:24.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.668+0000 7f835f020640 1 Processor -- start 2024-04-26T22:00:24.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.668+0000 7f835f020640 1 -- start start 2024-04-26T22:00:24.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f835f020640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83580740e0 0x7f8358157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f835f020640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835815a530 con 0x7f83580c6e20 2024-04-26T22:00:24.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f835f020640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835815a810 con 0x7f83580ca3a0 2024-04-26T22:00:24.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f835f020640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8358072930 con 0x7f83580740e0 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f835cdbc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f83580c6e20 0x7f8358156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46334/0 (socket says 172.21.15.135:46334) 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f835cdbc640 1 -- 172.21.15.135:0/2994981758 learned_addr learned my addr 172.21.15.135:0/2994981758 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f8357fff640 1 --2- 172.21.15.135:0/2994981758 >> v2:172.21.15.139:3300/0 conn(0x7f83580740e0 0x7f8358157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f83577fe640 1 -- 172.21.15.135:0/2994981758 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810742327 0 0) 0x7f835815a530 con 0x7f83580c6e20 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f83577fe640 1 -- 172.21.15.135:0/2994981758 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8344003520 con 0x7f83580c6e20 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f83577fe640 1 -- 172.21.15.135:0/2994981758 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3801668630 0 0) 0x7f8344003520 con 0x7f83580c6e20 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f8357fff640 1 -- 172.21.15.135:0/2994981758 >> v1:172.21.15.139:6789/0 conn(0x7f83580ca3a0 legacy=0x7f8358161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f8357fff640 1 -- 172.21.15.135:0/2994981758 >> v1:172.21.15.135:6789/0 conn(0x7f83580c6e20 legacy=0x7f8358156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.669+0000 7f8357fff640 1 -- 172.21.15.135:0/2994981758 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835815aaf0 con 0x7f83580740e0 2024-04-26T22:00:24.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.670+0000 7f8357fff640 1 --2- 172.21.15.135:0/2994981758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580740e0 0x7f8358157630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8348009d90 tx=0x7f834802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34504ef9b7a17b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.670+0000 7f83577fe640 1 -- 172.21.15.135:0/2994981758 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350001670 con 0x7f83580740e0 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.670+0000 7f83577fe640 1 -- 172.21.15.135:0/2994981758 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348007540 con 0x7f83580740e0 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.670+0000 7f83577fe640 1 -- 172.21.15.135:0/2994981758 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480078f0 con 0x7f83580740e0 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 -- 172.21.15.135:0/2994981758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580740e0 msgr2=0x7f8358157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 --2- 172.21.15.135:0/2994981758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580740e0 0x7f8358157630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8348009d90 tx=0x7f834802a5f0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 -- 172.21.15.135:0/2994981758 shutdown_connections 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 --2- 172.21.15.135:0/2994981758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580740e0 0x7f8358157630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 -- 172.21.15.135:0/2994981758 >> 172.21.15.135:0/2994981758 conn(0x7f83580ae0b0 msgr2=0x7f83580ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 -- 172.21.15.135:0/2994981758 shutdown_connections 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.671+0000 7f835f020640 1 -- 172.21.15.135:0/2994981758 wait complete. 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.672+0000 7f835f020640 1 Processor -- start 2024-04-26T22:00:24.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.672+0000 7f835f020640 1 -- start start 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835f020640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83580740e0 0x7f8358159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835f020640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 0x7f83580c1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835f020640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580ca3a0 0x7f83581ec710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835f020640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8358154540 con 0x7f83580c6e20 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835f020640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83581543c0 con 0x7f83580ca3a0 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835f020640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f835815a0c0 con 0x7f83580740e0 2024-04-26T22:00:24.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 0x7f83580c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 0x7f83580c1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58186/0 (socket says 172.21.15.135:58186) 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 -- 172.21.15.135:0/2143489408 learned_addr learned my addr 172.21.15.135:0/2143489408 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835cdbc640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83580740e0 0x7f8358159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f835d5bd640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580ca3a0 0x7f83581ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 -- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83580740e0 msgr2=0x7f8358159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83580740e0 0x7f8358159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 -- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580ca3a0 msgr2=0x7f83581ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580ca3a0 0x7f83581ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 -- 172.21.15.135:0/2143489408 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8358159860 con 0x7f83580c6e20 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.673+0000 7f8357fff640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 0x7f83580c1590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8348009eb0 tx=0x7f8348004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.674+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348009450 con 0x7f83580c6e20 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.674+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348004240 con 0x7f83580c6e20 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.674+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480045f0 con 0x7f83580c6e20 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.674+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83580c1f80 con 0x7f83580c6e20 2024-04-26T22:00:24.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.674+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8358159040 con 0x7f83580c6e20 2024-04-26T22:00:24.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.674+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83580766c0 con 0x7f83580c6e20 2024-04-26T22:00:24.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.675+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8348039040 con 0x7f83580c6e20 2024-04-26T22:00:24.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.675+0000 7f83557fa640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f833806c920 0x7f833806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.675+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f83480ad370 con 0x7f83580c6e20 2024-04-26T22:00:24.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.675+0000 7f835cdbc640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f833806c920 0x7f833806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.676+0000 7f835cdbc640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f833806c920 0x7f833806ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8350005880 tx=0x7f8350000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.680+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8348040030 con 0x7f83580c6e20 2024-04-26T22:00:24.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.855+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f83580507a0 con 0x7f83580c6e20 2024-04-26T22:00:24.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.856+0000 7f83557fa640 1 -- 172.21.15.135:0/2143489408 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f834807ada0 con 0x7f83580c6e20 2024-04-26T22:00:24.856 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.857+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f833806c920 msgr2=0x7f833806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.857+0000 7f835f020640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f833806c920 0x7f833806ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8350005880 tx=0x7f8350000bc0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.857+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 msgr2=0x7f83580c1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.857+0000 7f835f020640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 0x7f83580c1590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8348009eb0 tx=0x7f8348004a40 comp rx=0 tx=0).stop 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 shutdown_connections 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f833806c920 0x7f833806ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83580ca3a0 0x7f83581ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83580c6e20 0x7f83580c1590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 --2- 172.21.15.135:0/2143489408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83580740e0 0x7f8358159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 >> 172.21.15.135:0/2143489408 conn(0x7f83580ae0b0 msgr2=0x7f8358072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 shutdown_connections 2024-04-26T22:00:24.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.858+0000 7f835f020640 1 -- 172.21.15.135:0/2143489408 wait complete. 2024-04-26T22:00:24.869 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:24.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 Processor -- start 2024-04-26T22:00:24.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 -- start start 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 0x7fae84154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae84070f70 0x7fae840713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae840707e0 con 0x7fae84071cc0 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fae84071920 con 0x7fae84070f70 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae891e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae84155450 con 0x7fae8413ddd0 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.969+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 0x7fae84154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae7bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fae84071cc0 0x7fae84073700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:42002/0 (socket says 172.21.15.135:42002) 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae8359e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae84070f70 0x7fae840713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 0x7fae84154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:36356/0 (socket says 172.21.15.135:36356) 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae7bfff640 1 -- 172.21.15.135:0/2805830594 learned_addr learned my addr 172.21.15.135:0/2805830594 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae82d9d640 1 -- 172.21.15.135:0/2805830594 >> v2:172.21.15.139:3300/0 conn(0x7fae84070f70 msgr2=0x7fae840713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae82d9d640 1 --2- 172.21.15.135:0/2805830594 >> v2:172.21.15.139:3300/0 conn(0x7fae84070f70 0x7fae840713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae82d9d640 1 -- 172.21.15.135:0/2805830594 >> v1:172.21.15.139:6789/0 conn(0x7fae84071cc0 legacy=0x7fae84073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae82d9d640 1 -- 172.21.15.135:0/2805830594 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae84155730 con 0x7fae8413ddd0 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.970+0000 7fae8359e640 1 --2- 172.21.15.135:0/2805830594 >> v2:172.21.15.139:3300/0 conn(0x7fae84070f70 0x7fae840713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae82d9d640 1 --2- 172.21.15.135:0/2805830594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 0x7fae84154b00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae740097a0 tx=0x7fae7402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf9c45ba5b0a49a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae8259c640 1 -- 172.21.15.135:0/2805830594 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74037070 con 0x7fae8413ddd0 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae8259c640 1 -- 172.21.15.135:0/2805830594 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7402ae50 con 0x7fae8413ddd0 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae8259c640 1 -- 172.21.15.135:0/2805830594 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74032660 con 0x7fae8413ddd0 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 -- 172.21.15.135:0/2805830594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 msgr2=0x7fae84154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 --2- 172.21.15.135:0/2805830594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 0x7fae84154b00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae740097a0 tx=0x7fae7402a100 comp rx=0 tx=0).stop 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 -- 172.21.15.135:0/2805830594 shutdown_connections 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 --2- 172.21.15.135:0/2805830594 >> v2:172.21.15.139:3300/0 conn(0x7fae84070f70 0x7fae840713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 --2- 172.21.15.135:0/2805830594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8413ddd0 0x7fae84154b00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 -- 172.21.15.135:0/2805830594 >> 172.21.15.135:0/2805830594 conn(0x7fae840ca2f0 msgr2=0x7fae840ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 -- 172.21.15.135:0/2805830594 shutdown_connections 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.971+0000 7fae891e4640 1 -- 172.21.15.135:0/2805830594 wait complete. 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 Processor -- start 2024-04-26T22:00:24.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 -- start start 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae84070f70 0x7fae840c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae84071cc0 0x7fae841388d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 0x7fae84138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae84073f90 con 0x7fae8413ddd0 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae84073e10 con 0x7fae84071cc0 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae891e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae8413e1b0 con 0x7fae84070f70 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae8359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 0x7fae84138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae8359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 0x7fae84138e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58194/0 (socket says 172.21.15.135:58194) 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.973+0000 7fae8359e640 1 -- 172.21.15.135:0/1486902478 learned_addr learned my addr 172.21.15.135:0/1486902478 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae82d9d640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae84070f70 0x7fae840c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae7bfff640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae84071cc0 0x7fae841388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae8359e640 1 -- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae84070f70 msgr2=0x7fae840c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae8359e640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae84070f70 0x7fae840c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae8359e640 1 -- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae84071cc0 msgr2=0x7fae841388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae8359e640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae84071cc0 0x7fae841388d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae8359e640 1 -- 172.21.15.135:0/1486902478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae84156090 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae82d9d640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae84070f70 0x7fae840c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae8359e640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 0x7fae84138e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009f00 tx=0x7fae7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c018030 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c00ce50 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae84156370 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c003bb0 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.974+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae840c4600 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.975+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae840507a0 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.975+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae7c01b070 con 0x7fae8413ddd0 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.976+0000 7fae7b7fe640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:24.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.976+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fae7c08c850 con 0x7fae8413ddd0 2024-04-26T22:00:24.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.976+0000 7fae82d9d640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:24.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.976+0000 7fae82d9d640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae6406c8f0 0x7fae6406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae74034000 tx=0x7fae7402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:24.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:24.980+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae7c010040 con 0x7fae8413ddd0 2024-04-26T22:00:25.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.158+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fae84074110 con 0x7fae8413ddd0 2024-04-26T22:00:25.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.158+0000 7fae7b7fe640 1 -- 172.21.15.135:0/1486902478 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fae7c05a9e0 con 0x7fae8413ddd0 2024-04-26T22:00:25.159 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:25.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae6406c8f0 msgr2=0x7fae6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae6406c8f0 0x7fae6406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae74034000 tx=0x7fae7402e000 comp rx=0 tx=0).stop 2024-04-26T22:00:25.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 msgr2=0x7fae84138e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 0x7fae84138e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009f00 tx=0x7fae7c00c490 comp rx=0 tx=0).stop 2024-04-26T22:00:25.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 shutdown_connections 2024-04-26T22:00:25.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8413ddd0 0x7fae84138e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae84071cc0 0x7fae841388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 --2- 172.21.15.135:0/1486902478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae84070f70 0x7fae840c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 >> 172.21.15.135:0/1486902478 conn(0x7fae840ca2f0 msgr2=0x7fae840c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:25.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 shutdown_connections 2024-04-26T22:00:25.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.160+0000 7fae891e4640 1 -- 172.21.15.135:0/1486902478 wait complete. 2024-04-26T22:00:25.171 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:25.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.271+0000 7f74a6322640 1 Processor -- start 2024-04-26T22:00:25.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.271+0000 7f74a6322640 1 -- start start 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a6322640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f74a01572e0 0x7f74a0157730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a6322640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a00ad010 con 0x7f74a00c4250 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a6322640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a00c8300 con 0x7f74a00acc30 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a6322640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f74a00c8ba0 con 0x7f74a01572e0 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a48bf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f74a01572e0 0x7f74a0157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f749f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f74a00c4250 0x7f74a00c3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46350/0 (socket says 172.21.15.135:46350) 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a48bf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f74a01572e0 0x7f74a0157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58204/0 (socket says 172.21.15.135:58204) 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f749f7fe640 1 -- 172.21.15.135:0/3555658234 learned_addr learned my addr 172.21.15.135:0/3555658234 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:25.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a48bf640 1 -- 172.21.15.135:0/3555658234 >> v1:172.21.15.139:6789/0 conn(0x7f74a00acc30 legacy=0x7f74a015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:25.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a48bf640 1 -- 172.21.15.135:0/3555658234 >> v1:172.21.15.135:6789/0 conn(0x7f74a00c4250 legacy=0x7f74a00c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f74a48bf640 1 -- 172.21.15.135:0/3555658234 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a00c85e0 con 0x7f74a01572e0 2024-04-26T22:00:25.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.272+0000 7f749effd640 1 -- 172.21.15.135:0/3555658234 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287419216 0 0) 0x7f74a00ad010 con 0x7f74a00c4250 2024-04-26T22:00:25.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a48bf640 1 --2- 172.21.15.135:0/3555658234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a01572e0 0x7f74a0157730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7498009d90 tx=0x7f749802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bf36a0f63aea05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f749effd640 1 -- 172.21.15.135:0/3555658234 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498031600 con 0x7f74a01572e0 2024-04-26T22:00:25.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f749effd640 1 -- 172.21.15.135:0/3555658234 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74980317a0 con 0x7f74a01572e0 2024-04-26T22:00:25.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f749effd640 1 -- 172.21.15.135:0/3555658234 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498031b70 con 0x7f74a01572e0 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 -- 172.21.15.135:0/3555658234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a01572e0 msgr2=0x7f74a0157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 --2- 172.21.15.135:0/3555658234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a01572e0 0x7f74a0157730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7498009d90 tx=0x7f749802a610 comp rx=0 tx=0).stop 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 -- 172.21.15.135:0/3555658234 shutdown_connections 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 --2- 172.21.15.135:0/3555658234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a01572e0 0x7f74a0157730 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 -- 172.21.15.135:0/3555658234 >> 172.21.15.135:0/3555658234 conn(0x7f74a00c6e60 msgr2=0x7f74a00c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 -- 172.21.15.135:0/3555658234 shutdown_connections 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.273+0000 7f74a6322640 1 -- 172.21.15.135:0/3555658234 wait complete. 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.274+0000 7f74a6322640 1 Processor -- start 2024-04-26T22:00:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 -- start start 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a00acc30 0x7f74a01efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 0x7f74a014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74a01572e0 0x7f74a014f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f74a00c3ea0 con 0x7f74a00c4250 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0156e50 con 0x7f74a00acc30 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f74a6322640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f74a00c4020 con 0x7f74a01572e0 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f749f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 0x7f74a014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f749f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 0x7f74a014f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58216/0 (socket says 172.21.15.135:58216) 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.275+0000 7f749f7fe640 1 -- 172.21.15.135:0/3654301043 learned_addr learned my addr 172.21.15.135:0/3654301043 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f74a48bf640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74a01572e0 0x7f74a014f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749ffff640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a00acc30 0x7f74a01efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749f7fe640 1 -- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74a01572e0 msgr2=0x7f74a014f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749f7fe640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74a01572e0 0x7f74a014f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749f7fe640 1 -- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a00acc30 msgr2=0x7f74a01efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749f7fe640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a00acc30 0x7f74a01efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749f7fe640 1 -- 172.21.15.135:0/3654301043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a015a280 con 0x7f74a00c4250 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749ffff640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a00acc30 0x7f74a01efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749f7fe640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 0x7f74a014f2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f749400c4b0 tx=0x7f749400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494018070 con 0x7f74a00c4250 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7494007500 con 0x7f74a00c4250 2024-04-26T22:00:25.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494004a20 con 0x7f74a00c4250 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01f06b0 con 0x7f74a00c4250 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.276+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a014e4d0 con 0x7f74a00c4250 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.277+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a00507a0 con 0x7f74a00c4250 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.277+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f749401b070 con 0x7f74a00c4250 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.278+0000 7f749cff9640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f748c06c8f0 0x7f748c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.278+0000 7f749ffff640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f748c06c8f0 0x7f748c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.278+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f749408b060 con 0x7f74a00c4250 2024-04-26T22:00:25.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.278+0000 7f749ffff640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f748c06c8f0 0x7f748c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74a014d5a0 tx=0x7f7490009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.282+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74a015a280 con 0x7f74a00c4250 2024-04-26T22:00:25.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.456+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f74a00c46f0 con 0x7f74a00c4250 2024-04-26T22:00:25.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.456+0000 7f749cff9640 1 -- 172.21.15.135:0/3654301043 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f74940592b0 con 0x7f74a00c4250 2024-04-26T22:00:25.456 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:25.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f748c06c8f0 msgr2=0x7f748c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f748c06c8f0 0x7f748c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74a014d5a0 tx=0x7f7490009040 comp rx=0 tx=0).stop 2024-04-26T22:00:25.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 msgr2=0x7f74a014f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 0x7f74a014f2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f749400c4b0 tx=0x7f749400c970 comp rx=0 tx=0).stop 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 shutdown_connections 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f748c06c8f0 0x7f748c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74a01572e0 0x7f74a014f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74a00c4250 0x7f74a014f2a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 --2- 172.21.15.135:0/3654301043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a00acc30 0x7f74a01efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 >> 172.21.15.135:0/3654301043 conn(0x7f74a00c6e60 msgr2=0x7f74a00c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 shutdown_connections 2024-04-26T22:00:25.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.458+0000 7f74a6322640 1 -- 172.21.15.135:0/3654301043 wait complete. 2024-04-26T22:00:25.469 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:25.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.568+0000 7f6777939640 1 Processor -- start 2024-04-26T22:00:25.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.568+0000 7f6777939640 1 -- start start 2024-04-26T22:00:25.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.568+0000 7f6777939640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f67700ab3f0 0x7f67700c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.568+0000 7f6777939640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67700c3810 con 0x7f67700c52c0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6777939640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67700c3ab0 con 0x7f67701570d0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6777939640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f67700c4330 con 0x7f67700ab3f0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6775ed6640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f67700c52c0 0x7f67701574b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46360/0 (socket says 172.21.15.135:46360) 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6775ed6640 1 -- 172.21.15.135:0/2092526273 learned_addr learned my addr 172.21.15.135:0/2092526273 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6774ed4640 1 --2- 172.21.15.135:0/2092526273 >> v2:172.21.15.139:3300/0 conn(0x7f67700ab3f0 0x7f67700c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525695838 0 0) 0x7f67700c3810 con 0x7f67700c52c0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f675c003520 con 0x7f67700c52c0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827933591 0 0) 0x7f67700c3ab0 con 0x7f67701570d0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67700c3810 con 0x7f67701570d0 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6774ed4640 1 -- 172.21.15.135:0/2092526273 >> v1:172.21.15.139:6789/0 conn(0x7f67701570d0 legacy=0x7f6770150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6774ed4640 1 -- 172.21.15.135:0/2092526273 >> v1:172.21.15.135:6789/0 conn(0x7f67700c52c0 legacy=0x7f67701574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.569+0000 7f6774ed4640 1 -- 172.21.15.135:0/2092526273 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67700c4580 con 0x7f67700ab3f0 2024-04-26T22:00:25.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.570+0000 7f6774ed4640 1 --2- 172.21.15.135:0/2092526273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67700ab3f0 0x7f67700c4d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67600097a0 tx=0x7f676002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4a87baaf025bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.570+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760037070 con 0x7f67700ab3f0 2024-04-26T22:00:25.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.570+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6760032440 con 0x7f67700ab3f0 2024-04-26T22:00:25.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.570+0000 7f6767fff640 1 -- 172.21.15.135:0/2092526273 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676002ae60 con 0x7f67700ab3f0 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.570+0000 7f6777939640 1 -- 172.21.15.135:0/2092526273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67700ab3f0 msgr2=0x7f67700c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.570+0000 7f6777939640 1 --2- 172.21.15.135:0/2092526273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67700ab3f0 0x7f67700c4d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67600097a0 tx=0x7f676002a110 comp rx=0 tx=0).stop 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.571+0000 7f6777939640 1 -- 172.21.15.135:0/2092526273 shutdown_connections 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.571+0000 7f6777939640 1 --2- 172.21.15.135:0/2092526273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67700ab3f0 0x7f67700c4d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.571+0000 7f6777939640 1 -- 172.21.15.135:0/2092526273 >> 172.21.15.135:0/2092526273 conn(0x7f677006c310 msgr2=0x7f677006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.571+0000 7f6777939640 1 -- 172.21.15.135:0/2092526273 shutdown_connections 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.571+0000 7f6777939640 1 -- 172.21.15.135:0/2092526273 wait complete. 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 Processor -- start 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 -- start start 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67700ab3f0 0x7f67701e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 0x7f6770152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67701570d0 0x7f6770151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67701512e0 con 0x7f67700c52c0 2024-04-26T22:00:25.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6770151160 con 0x7f67701570d0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6777939640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67700aba60 con 0x7f67700ab3f0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6774ed4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 0x7f6770152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6774ed4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 0x7f6770152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58218/0 (socket says 172.21.15.135:58218) 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.572+0000 7f6774ed4640 1 -- 172.21.15.135:0/2481812146 learned_addr learned my addr 172.21.15.135:0/2481812146 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6775ed6640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67701570d0 0x7f6770151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f67756d5640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67700ab3f0 0x7f67701e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6774ed4640 1 -- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67700ab3f0 msgr2=0x7f67701e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6774ed4640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67700ab3f0 0x7f67701e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6774ed4640 1 -- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67701570d0 msgr2=0x7f6770151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6774ed4640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67701570d0 0x7f6770151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6774ed4640 1 -- 172.21.15.135:0/2481812146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67701e97c0 con 0x7f67700c52c0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6775ed6640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67701570d0 0x7f6770151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6774ed4640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 0x7f6770152c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f676002a150 tx=0x7f67600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760041030 con 0x7f67700c52c0 2024-04-26T22:00:25.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6760030440 con 0x7f67700c52c0 2024-04-26T22:00:25.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677014f3a0 con 0x7f67700c52c0 2024-04-26T22:00:25.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760037040 con 0x7f67700c52c0 2024-04-26T22:00:25.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.573+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677014e780 con 0x7f67700c52c0 2024-04-26T22:00:25.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.574+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6770150470 con 0x7f67700c52c0 2024-04-26T22:00:25.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.575+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6760044070 con 0x7f67700c52c0 2024-04-26T22:00:25.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.575+0000 7f6765ffb640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f675406c8f0 0x7f675406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.575+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f67600ac640 con 0x7f67700c52c0 2024-04-26T22:00:25.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.575+0000 7f67756d5640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f675406c8f0 0x7f675406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.575+0000 7f67756d5640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f675406c8f0 0x7f675406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f676c0010e0 tx=0x7f676c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.579+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67701e97c0 con 0x7f67700c52c0 2024-04-26T22:00:25.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.754+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f67700507a0 con 0x7f67700c52c0 2024-04-26T22:00:25.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.754+0000 7f6765ffb640 1 -- 172.21.15.135:0/2481812146 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f676007a750 con 0x7f67700c52c0 2024-04-26T22:00:25.754 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:25.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f675406c8f0 msgr2=0x7f675406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f675406c8f0 0x7f675406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f676c0010e0 tx=0x7f676c009040 comp rx=0 tx=0).stop 2024-04-26T22:00:25.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 msgr2=0x7f6770152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 0x7f6770152c00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f676002a150 tx=0x7f67600046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 shutdown_connections 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f675406c8f0 0x7f675406edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67701570d0 0x7f6770151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67700c52c0 0x7f6770152c00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 --2- 172.21.15.135:0/2481812146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67700ab3f0 0x7f67701e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 >> 172.21.15.135:0/2481812146 conn(0x7f677006c310 msgr2=0x7f67700c2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 shutdown_connections 2024-04-26T22:00:25.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.756+0000 7f6777939640 1 -- 172.21.15.135:0/2481812146 wait complete. 2024-04-26T22:00:25.767 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:25.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.864+0000 7f6c58789640 1 Processor -- start 2024-04-26T22:00:25.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.864+0000 7f6c58789640 1 -- start start 2024-04-26T22:00:25.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c58789640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50151090 0x7f6c5015ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c58789640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6c50157880 0x7f6c500c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c58789640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c5015b0c0 con 0x7f6c500c6e50 2024-04-26T22:00:25.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c58789640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500ae180 con 0x7f6c50157880 2024-04-26T22:00:25.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c58789640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500ae460 con 0x7f6c50151090 2024-04-26T22:00:25.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c56d26640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6c50157880 0x7f6c500c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c56d26640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6c50157880 0x7f6c500c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58224/0 (socket says 172.21.15.135:58224) 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.865+0000 7f6c56d26640 1 -- 172.21.15.135:0/4060761771 learned_addr learned my addr 172.21.15.135:0/4060761771 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c56525640 1 --2- 172.21.15.135:0/4060761771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50151090 0x7f6c5015ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c56d26640 1 -- 172.21.15.135:0/4060761771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50151090 msgr2=0x7f6c5015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c56d26640 1 --2- 172.21.15.135:0/4060761771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50151090 0x7f6c5015ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c56d26640 1 -- 172.21.15.135:0/4060761771 >> v1:172.21.15.139:6789/0 conn(0x7f6c500c6e50 legacy=0x7f6c50157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c56d26640 1 -- 172.21.15.135:0/4060761771 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500ae740 con 0x7f6c50157880 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c56d26640 1 --2- 172.21.15.135:0/4060761771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50157880 0x7f6c500c72b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009e50 tx=0x7f6c4802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cd0ba3043a1f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c55523640 1 -- 172.21.15.135:0/4060761771 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48036070 con 0x7f6c50157880 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.866+0000 7f6c55523640 1 -- 172.21.15.135:0/4060761771 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c48031600 con 0x7f6c50157880 2024-04-26T22:00:25.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c55523640 1 -- 172.21.15.135:0/4060761771 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480319d0 con 0x7f6c50157880 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 -- 172.21.15.135:0/4060761771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50157880 msgr2=0x7f6c500c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 --2- 172.21.15.135:0/4060761771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50157880 0x7f6c500c72b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009e50 tx=0x7f6c4802a630 comp rx=0 tx=0).stop 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 -- 172.21.15.135:0/4060761771 shutdown_connections 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 --2- 172.21.15.135:0/4060761771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50157880 0x7f6c500c72b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 --2- 172.21.15.135:0/4060761771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50151090 0x7f6c5015ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 -- 172.21.15.135:0/4060761771 >> 172.21.15.135:0/4060761771 conn(0x7f6c500cc4d0 msgr2=0x7f6c500cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 -- 172.21.15.135:0/4060761771 shutdown_connections 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.867+0000 7f6c58789640 1 -- 172.21.15.135:0/4060761771 wait complete. 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.868+0000 7f6c58789640 1 Processor -- start 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.868+0000 7f6c58789640 1 -- start start 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 0x7f6c50141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c50151090 0x7f6c500c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50157880 0x7f6c5014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500ca4f0 con 0x7f6c500c6e50 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500aea20 con 0x7f6c50151090 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500ca670 con 0x7f6c50157880 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 0x7f6c50141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 0x7f6c50141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58228/0 (socket says 172.21.15.135:58228) 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 -- 172.21.15.135:0/95544726 learned_addr learned my addr 172.21.15.135:0/95544726 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c55d24640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c50151090 0x7f6c500c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56d26640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50157880 0x7f6c5014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 -- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50157880 msgr2=0x7f6c5014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50157880 0x7f6c5014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 -- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c50151090 msgr2=0x7f6c500c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c50151090 0x7f6c500c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 -- 172.21.15.135:0/95544726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500c64e0 con 0x7f6c500c6e50 2024-04-26T22:00:25.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c56525640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 0x7f6c50141b20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c7f0 tx=0x7f6c4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c015070 con 0x7f6c500c6e50 2024-04-26T22:00:25.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4c004240 con 0x7f6c500c6e50 2024-04-26T22:00:25.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.869+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c501e2830 con 0x7f6c500c6e50 2024-04-26T22:00:25.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.870+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c007620 con 0x7f6c500c6e50 2024-04-26T22:00:25.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.870+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501e2db0 con 0x7f6c500c6e50 2024-04-26T22:00:25.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.870+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c501545a0 con 0x7f6c500c6e50 2024-04-26T22:00:25.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.871+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c4c0079d0 con 0x7f6c500c6e50 2024-04-26T22:00:25.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.871+0000 7f6c42ffd640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6c3406c8f0 0x7f6c3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:25.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.871+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f6c4c08c150 con 0x7f6c500c6e50 2024-04-26T22:00:25.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.871+0000 7f6c55d24640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6c3406c8f0 0x7f6c3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:25.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.872+0000 7f6c55d24640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6c3406c8f0 0x7f6c3406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c501422e0 tx=0x7f6c44007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:25.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:25.876+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c4c059c90 con 0x7f6c500c6e50 2024-04-26T22:00:26.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.087+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c500507a0 con 0x7f6c500c6e50 2024-04-26T22:00:26.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.087+0000 7f6c42ffd640 1 -- 172.21.15.135:0/95544726 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f6c4c05df90 con 0x7f6c500c6e50 2024-04-26T22:00:26.087 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:26.087 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:26.088 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:26.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6c3406c8f0 msgr2=0x7f6c3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6c3406c8f0 0x7f6c3406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c501422e0 tx=0x7f6c44007e30 comp rx=0 tx=0).stop 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 msgr2=0x7f6c50141b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 0x7f6c50141b20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c7f0 tx=0x7f6c4c00ccb0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 shutdown_connections 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6c3406c8f0 0x7f6c3406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c50157880 0x7f6c5014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c50151090 0x7f6c500c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 --2- 172.21.15.135:0/95544726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c500c6e50 0x7f6c50141b20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 >> 172.21.15.135:0/95544726 conn(0x7f6c500cc4d0 msgr2=0x7f6c50156440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 shutdown_connections 2024-04-26T22:00:26.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.089+0000 7f6c58789640 1 -- 172.21.15.135:0/95544726 wait complete. 2024-04-26T22:00:26.100 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:26.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.196+0000 7f7960b3f640 1 Processor -- start 2024-04-26T22:00:26.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.196+0000 7f7960b3f640 1 -- start start 2024-04-26T22:00:26.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7960b3f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c1570d0 0x7f795c150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7960b3f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f795c150f90 0x7f795c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7960b3f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795c0c3810 con 0x7f795c0ab3f0 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7960b3f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f795c0c4330 con 0x7f795c150f90 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7960b3f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f795c0c3ab0 con 0x7f795c1570d0 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f795ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f795c150f90 0x7f795c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f795a59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c1570d0 0x7f795c150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7953fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f795c0ab3f0 0x7f795c0c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:42052/0 (socket says 172.21.15.135:42052) 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f7953fff640 1 -- 172.21.15.135:0/2296054036 learned_addr learned my addr 172.21.15.135:0/2296054036 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f795ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f795c150f90 0x7f795c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:47726/0 (socket says 172.21.15.135:47726) 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.197+0000 7f795a59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c1570d0 0x7f795c150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:36426/0 (socket says 172.21.15.135:36426) 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.198+0000 7f795ad9d640 1 -- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c1570d0 msgr2=0x7f795c150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.198+0000 7f795ad9d640 1 --2- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c1570d0 0x7f795c150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.198+0000 7f795ad9d640 1 -- 172.21.15.135:0/2296054036 >> v1:172.21.15.139:6789/0 conn(0x7f795c0ab3f0 legacy=0x7f795c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.198+0000 7f795ad9d640 1 -- 172.21.15.135:0/2296054036 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c0c4580 con 0x7f795c150f90 2024-04-26T22:00:26.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.198+0000 7f7959d9b640 1 -- 172.21.15.135:0/2296054036 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299437107 0 0) 0x7f795c0c3810 con 0x7f795c0ab3f0 2024-04-26T22:00:26.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.198+0000 7f795ad9d640 1 --2- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c150f90 0x7f795c1574b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f795400bfa0 tx=0x7f795402c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcafbb58f7ae880e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7959d9b640 1 -- 172.21.15.135:0/2296054036 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954038070 con 0x7f795c150f90 2024-04-26T22:00:26.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7959d9b640 1 -- 172.21.15.135:0/2296054036 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7954009540 con 0x7f795c150f90 2024-04-26T22:00:26.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7959d9b640 1 -- 172.21.15.135:0/2296054036 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954009910 con 0x7f795c150f90 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 -- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c150f90 msgr2=0x7f795c1574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c150f90 0x7f795c1574b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f795400bfa0 tx=0x7f795402c750 comp rx=0 tx=0).stop 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 -- 172.21.15.135:0/2296054036 shutdown_connections 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c150f90 0x7f795c1574b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2296054036 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c1570d0 0x7f795c150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 -- 172.21.15.135:0/2296054036 >> 172.21.15.135:0/2296054036 conn(0x7f795c06c310 msgr2=0x7f795c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 -- 172.21.15.135:0/2296054036 shutdown_connections 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.199+0000 7f7960b3f640 1 -- 172.21.15.135:0/2296054036 wait complete. 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.200+0000 7f7960b3f640 1 Processor -- start 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 -- start start 2024-04-26T22:00:26.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c0ab3f0 0x7f795c072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 0x7f795c0c1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c1570d0 0x7f795c1ec960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0abbe0 con 0x7f795c150f90 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0aba60 con 0x7f795c1570d0 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7960b3f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f795c0c5490 con 0x7f795c0ab3f0 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7953fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 0x7f795c0c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7953fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 0x7f795c0c1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58234/0 (socket says 172.21.15.135:58234) 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f7953fff640 1 -- 172.21.15.135:0/2395205533 learned_addr learned my addr 172.21.15.135:0/2395205533 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f795ad9d640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c1570d0 0x7f795c1ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.201+0000 7f795a59c640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c0ab3f0 0x7f795c072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7953fff640 1 -- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c0ab3f0 msgr2=0x7f795c072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7953fff640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c0ab3f0 0x7f795c072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7953fff640 1 -- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c1570d0 msgr2=0x7f795c1ec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7953fff640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c1570d0 0x7f795c1ec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7953fff640 1 -- 172.21.15.135:0/2395205533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c1f07e0 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f795ad9d640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c1570d0 0x7f795c1ec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f795a59c640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c0ab3f0 0x7f795c072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7953fff640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 0x7f795c0c1970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f794400c400 tx=0x7f794400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944017070 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7944005ce0 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c074790 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944007310 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.202+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1f0a60 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.203+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c154810 con 0x7f795c150f90 2024-04-26T22:00:26.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.203+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794401a070 con 0x7f795c150f90 2024-04-26T22:00:26.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.204+0000 7f7952ffd640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.204+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f794408b220 con 0x7f795c150f90 2024-04-26T22:00:26.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.204+0000 7f795a59c640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.204+0000 7f795a59c640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f793c06c8f0 0x7f793c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f794c009ac0 tx=0x7f794c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.208+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f795c1f07e0 con 0x7f795c150f90 2024-04-26T22:00:26.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.386+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f795c0507a0 con 0x7f795c150f90 2024-04-26T22:00:26.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.387+0000 7f7952ffd640 1 -- 172.21.15.135:0/2395205533 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f79440593b0 con 0x7f795c150f90 2024-04-26T22:00:26.388 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:26.388 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f793c06c8f0 msgr2=0x7f793c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f793c06c8f0 0x7f793c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f794c009ac0 tx=0x7f794c008040 comp rx=0 tx=0).stop 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 msgr2=0x7f795c0c1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 0x7f795c0c1970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f794400c400 tx=0x7f794400c8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 shutdown_connections 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f795c1570d0 0x7f795c1ec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795c150f90 0x7f795c0c1970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 --2- 172.21.15.135:0/2395205533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f795c0ab3f0 0x7f795c072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 >> 172.21.15.135:0/2395205533 conn(0x7f795c06c310 msgr2=0x7f795c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 shutdown_connections 2024-04-26T22:00:26.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.390+0000 7f7960b3f640 1 -- 172.21.15.135:0/2395205533 wait complete. 2024-04-26T22:00:26.401 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:26.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.500+0000 7f7e94c07640 1 Processor -- start 2024-04-26T22:00:26.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.500+0000 7f7e94c07640 1 -- start start 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e94c07640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7e901570d0 0x7f7e90150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e94c07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7e900ab3f0 0x7f7e90150f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e94c07640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e900c3810 con 0x7f7e900c4d80 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e94c07640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7e900c4330 con 0x7f7e901570d0 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e94c07640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7e900c3ab0 con 0x7f7e900ab3f0 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8e59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7e901570d0 0x7f7e90150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8ed9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7e900c4d80 0x7f7e901574b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46368/0 (socket says 172.21.15.135:46368) 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8ed9d640 1 -- 172.21.15.135:0/1223636364 learned_addr learned my addr 172.21.15.135:0/1223636364 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8dd9b640 1 --2- 172.21.15.135:0/1223636364 >> v2:172.21.15.139:3300/0 conn(0x7f7e900ab3f0 0x7f7e90150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8e59c640 1 -- 172.21.15.135:0/1223636364 >> v2:172.21.15.139:3300/0 conn(0x7f7e900ab3f0 msgr2=0x7f7e90150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:26.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8e59c640 1 --2- 172.21.15.135:0/1223636364 >> v2:172.21.15.139:3300/0 conn(0x7f7e900ab3f0 0x7f7e90150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8e59c640 1 -- 172.21.15.135:0/1223636364 >> v1:172.21.15.135:6789/0 conn(0x7f7e900c4d80 legacy=0x7f7e901574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8e59c640 1 -- 172.21.15.135:0/1223636364 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e900c4580 con 0x7f7e901570d0 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.501+0000 7f7e8d59a640 1 -- 172.21.15.135:0/1223636364 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882046547 0 0) 0x7f7e900c3810 con 0x7f7e900c4d80 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.502+0000 7f7e8e59c640 1 --2- 172.21.15.135:0/1223636364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e901570d0 0x7f7e90150a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e900c44b0 tx=0x7f7e8002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32aef23e2cba371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.502+0000 7f7e8d59a640 1 -- 172.21.15.135:0/1223636364 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80037070 con 0x7f7e901570d0 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.502+0000 7f7e8d59a640 1 -- 172.21.15.135:0/1223636364 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8002add0 con 0x7f7e901570d0 2024-04-26T22:00:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.502+0000 7f7e8d59a640 1 -- 172.21.15.135:0/1223636364 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80032660 con 0x7f7e901570d0 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 -- 172.21.15.135:0/1223636364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e901570d0 msgr2=0x7f7e90150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 --2- 172.21.15.135:0/1223636364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e901570d0 0x7f7e90150a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e900c44b0 tx=0x7f7e8002a080 comp rx=0 tx=0).stop 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 -- 172.21.15.135:0/1223636364 shutdown_connections 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 --2- 172.21.15.135:0/1223636364 >> v2:172.21.15.139:3300/0 conn(0x7f7e900ab3f0 0x7f7e90150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 --2- 172.21.15.135:0/1223636364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e901570d0 0x7f7e90150a50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 -- 172.21.15.135:0/1223636364 >> 172.21.15.135:0/1223636364 conn(0x7f7e9006c310 msgr2=0x7f7e9006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 -- 172.21.15.135:0/1223636364 shutdown_connections 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.503+0000 7f7e94c07640 1 -- 172.21.15.135:0/1223636364 wait complete. 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.504+0000 7f7e94c07640 1 Processor -- start 2024-04-26T22:00:26.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.504+0000 7f7e94c07640 1 -- start start 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e900ab3f0 0x7f7e90152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 0x7f7e9014a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e901570d0 0x7f7e9014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e900abbe0 con 0x7f7e900c4d80 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7e900aba60 con 0x7f7e901570d0 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7e90157bc0 con 0x7f7e900ab3f0 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 0x7f7e9014a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 0x7f7e9014a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58248/0 (socket says 172.21.15.135:58248) 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 -- 172.21.15.135:0/3250701278 learned_addr learned my addr 172.21.15.135:0/3250701278 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8e59c640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e900ab3f0 0x7f7e90152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8ed9d640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e901570d0 0x7f7e9014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 -- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e900ab3f0 msgr2=0x7f7e90152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e900ab3f0 0x7f7e90152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 -- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e901570d0 msgr2=0x7f7e9014ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e901570d0 0x7f7e9014ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 -- 172.21.15.135:0/3250701278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e90154640 con 0x7f7e900c4d80 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8e59c640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e900ab3f0 0x7f7e90152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e8dd9b640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 0x7f7e9014a8f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e9b0 tx=0x7f7e8400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84017070 con 0x7f7e900c4d80 2024-04-26T22:00:26.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8400c9a0 con 0x7f7e900c4d80 2024-04-26T22:00:26.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e900c7850 con 0x7f7e900c4d80 2024-04-26T22:00:26.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400cd90 con 0x7f7e900c4d80 2024-04-26T22:00:26.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.505+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e901528a0 con 0x7f7e900c4d80 2024-04-26T22:00:26.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.506+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9006cb60 con 0x7f7e900c4d80 2024-04-26T22:00:26.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.507+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e8401a070 con 0x7f7e900c4d80 2024-04-26T22:00:26.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.507+0000 7f7e7effd640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7e7006c8f0 0x7f7e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:26.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.507+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f7e8408be10 con 0x7f7e900c4d80 2024-04-26T22:00:26.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.507+0000 7f7e8e59c640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7e7006c8f0 0x7f7e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:26.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.508+0000 7f7e8e59c640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7e7006c8f0 0x7f7e7006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e80034000 tx=0x7f7e8002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:26.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.511+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e84059f80 con 0x7f7e900c4d80 2024-04-26T22:00:26.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.726+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e900507a0 con 0x7f7e900c4d80 2024-04-26T22:00:26.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.727+0000 7f7e7effd640 1 -- 172.21.15.135:0/3250701278 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f7e84059880 con 0x7f7e900c4d80 2024-04-26T22:00:26.727 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:26.727 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:26.727 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:26.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7e7006c8f0 msgr2=0x7f7e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7e7006c8f0 0x7f7e7006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e80034000 tx=0x7f7e8002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 msgr2=0x7f7e9014a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 0x7f7e9014a8f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400e9b0 tx=0x7f7e8400ee70 comp rx=0 tx=0).stop 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 shutdown_connections 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7e7006c8f0 0x7f7e7006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e901570d0 0x7f7e9014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e900c4d80 0x7f7e9014a8f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 --2- 172.21.15.135:0/3250701278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e900ab3f0 0x7f7e90152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 >> 172.21.15.135:0/3250701278 conn(0x7f7e9006c310 msgr2=0x7f7e900ac2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 shutdown_connections 2024-04-26T22:00:26.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:26.729+0000 7f7e94c07640 1 -- 172.21.15.135:0/3250701278 wait complete. 2024-04-26T22:00:27.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T22:00:06.148422+0000', 'modified': '2024-04-26T22:00:10.177793+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'c', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.135:6836/3981978490', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3981978490}, {'type': 'v1', 'addr': '172.21.15.135:6836', 'nonce': 3981978490}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'b', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4275, 'qdb_cluster': [4275]} 2024-04-26T22:00:27.144 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:00:27.144 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T22:00:27.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.242+0000 7f6877b79640 1 Processor -- start 2024-04-26T22:00:27.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.242+0000 7f6877b79640 1 -- start start 2024-04-26T22:00:27.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.243+0000 7f6877b79640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 0x7f68701539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.243+0000 7f6877b79640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6870153ff0 con 0x7f687006d590 2024-04-26T22:00:27.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.243+0000 7f6877b79640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68701542d0 con 0x7f68700c5560 2024-04-26T22:00:27.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.243+0000 7f6877b79640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68701545b0 con 0x7f6870157be0 2024-04-26T22:00:27.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.243+0000 7f6875915640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f687006d590 0x7f68701574d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46370/0 (socket says 172.21.15.135:46370) 2024-04-26T22:00:27.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.243+0000 7f6875915640 1 -- 172.21.15.135:0/2391395202 learned_addr learned my addr 172.21.15.135:0/2391395202 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:27.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363381110 0 0) 0x7f6870153ff0 con 0x7f687006d590 2024-04-26T22:00:27.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 0x7f68701539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f685c003520 con 0x7f687006d590 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1936960606 0 0) 0x7f685c003520 con 0x7f687006d590 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 msgr2=0x7f68701539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 0x7f68701539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 >> v1:172.21.15.139:6789/0 conn(0x7f68700c5560 legacy=0x7f6870150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6870154890 con 0x7f687006d590 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f686c0019f0 con 0x7f687006d590 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 >> v1:172.21.15.135:6789/0 conn(0x7f687006d590 legacy=0x7f68701574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 0x7f685c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f685c003f60 0x7f685c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68701545b0 con 0x7f685c003f60 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f685c00bb50 con 0x7f685c004d40 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f685c00bd50 con 0x7f6870157be0 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f685c003f60 0x7f685c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875915640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 0x7f685c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6876116640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 -- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 msgr2=0x7f685c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870157be0 0x7f685c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 -- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c004d40 msgr2=0x7f685c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.244+0000 7f6875114640 1 -- 172.21.15.135:0/2391395202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c003520 con 0x7f685c003f60 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6875114640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f685c003f60 0x7f685c004800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6860009ac0 tx=0x7f685c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b643ec237180979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860036070 con 0x7f685c003f60 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6860031400 con 0x7f685c003f60 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6874913640 1 -- 172.21.15.135:0/2391395202 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860009690 con 0x7f685c003f60 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6877b79640 1 -- 172.21.15.135:0/2391395202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f685c003f60 msgr2=0x7f685c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6877b79640 1 --2- 172.21.15.135:0/2391395202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f685c003f60 0x7f685c004800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6860009ac0 tx=0x7f685c005830 comp rx=0 tx=0).stop 2024-04-26T22:00:27.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.245+0000 7f6876116640 1 -- 172.21.15.135:0/2391395202 reap_dead start 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.246+0000 7f6877b79640 1 -- 172.21.15.135:0/2391395202 shutdown_connections 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.246+0000 7f6877b79640 1 -- 172.21.15.135:0/2391395202 >> 172.21.15.135:0/2391395202 conn(0x7f68700c5ce0 msgr2=0x7f68700c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.246+0000 7f6877b79640 1 -- 172.21.15.135:0/2391395202 shutdown_connections 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.246+0000 7f6877b79640 1 -- 172.21.15.135:0/2391395202 wait complete. 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 Processor -- start 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 -- start start 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c003f60 0x7f6870075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68700c33e0 0x7f6870144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 0x7f68701ed970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68700c3c70 con 0x7f6870144ac0 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6870074b30 con 0x7f685c003f60 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6877b79640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68700c3df0 con 0x7f68700c33e0 2024-04-26T22:00:27.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6876116640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 0x7f68701ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6876116640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 0x7f68701ed970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58258/0 (socket says 172.21.15.135:58258) 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.247+0000 7f6876116640 1 -- 172.21.15.135:0/3942888653 learned_addr learned my addr 172.21.15.135:0/3942888653 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6875114640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68700c33e0 0x7f6870144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6875915640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c003f60 0x7f6870075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6876116640 1 -- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68700c33e0 msgr2=0x7f6870144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6876116640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68700c33e0 0x7f6870144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6876116640 1 -- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c003f60 msgr2=0x7f6870075960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6876116640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c003f60 0x7f6870075960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6876116640 1 -- 172.21.15.135:0/3942888653 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68701ea650 con 0x7f6870144ac0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6876116640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 0x7f68701ed970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68700c5c40 tx=0x7f686800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f68667fc640 1 -- 172.21.15.135:0/3942888653 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68680053a0 con 0x7f6870144ac0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f68667fc640 1 -- 172.21.15.135:0/3942888653 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6868018070 con 0x7f6870144ac0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f68667fc640 1 -- 172.21.15.135:0/3942888653 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868003830 con 0x7f6870144ac0 2024-04-26T22:00:27.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68701ea930 con 0x7f6870144ac0 2024-04-26T22:00:27.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.248+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6870075ea0 con 0x7f6870144ac0 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.250+0000 7f68667fc640 1 -- 172.21.15.135:0/3942888653 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68680039d0 con 0x7f6870144ac0 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.250+0000 7f68667fc640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f685406c8f0 0x7f685406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.250+0000 7f68667fc640 1 -- 172.21.15.135:0/3942888653 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f686808b8f0 con 0x7f6870144ac0 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.250+0000 7f6875915640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f685406c8f0 0x7f685406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.250+0000 7f6875915640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f685406c8f0 0x7f685406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68700762b0 tx=0x7f686c006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 Processor -- start 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 -- start start 2024-04-26T22:00:27.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d0c0 0x55c86317edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c86317e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c86313b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c8630a5740 con 0x55c86317d4a0 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c8630413a0 con 0x55c86317d0c0 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.251+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55c8630a1700 con 0x55c86317da00 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c86317e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c86317e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58270/0 (socket says 172.21.15.135:58270) 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 -- 172.21.15.135:0/1757709964 learned_addr learned my addr 172.21.15.135:0/1757709964 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6876116640 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c86313b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875915640 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d0c0 0x55c86317edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 -- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 msgr2=0x55c86313b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c86313b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 -- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d0c0 msgr2=0x55c86317edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d0c0 0x55c86317edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 -- 172.21.15.135:0/1757709964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86305d750 con 0x55c86317d4a0 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f6875114640 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c86317e450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6860034040 tx=0x7f686002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6547b902faa159c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f68537fe640 1 -- 172.21.15.135:0/1757709964 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860040070 con 0x55c86317d4a0 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f68537fe640 1 -- 172.21.15.135:0/1757709964 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6860004300 con 0x55c86317d4a0 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.252+0000 7f68537fe640 1 -- 172.21.15.135:0/1757709964 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860036040 con 0x55c86317d4a0 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 -- 172.21.15.135:0/1757709964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 msgr2=0x55c86317e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c86317e450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6860034040 tx=0x7f686002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 -- 172.21.15.135:0/1757709964 shutdown_connections 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c86313b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c86317e450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 --2- 172.21.15.135:0/1757709964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d0c0 0x55c86317edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 -- 172.21.15.135:0/1757709964 >> 172.21.15.135:0/1757709964 conn(0x55c86313d930 msgr2=0x55c863162050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 -- 172.21.15.135:0/1757709964 shutdown_connections 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 -- 172.21.15.135:0/1757709964 wait complete. 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 -- start start 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.253+0000 7f687a968740 1 Processor -- start 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317d4a0 0x55c863140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 0x55c8631794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c863186360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c86313cbb0 con 0x55c86317da00 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c86302cfe0 con 0x55c8631874a0 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55c86318a890 con 0x55c86317d4a0 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f687a968740 10 client.0 _ll_register_callbacks cb 0x55c863043150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f6875114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 0x55c8631794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f6875114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 0x55c8631794e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58276/0 (socket says 172.21.15.135:58276) 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f6875114640 1 -- 172.21.15.135:0/2351850533 learned_addr learned my addr 172.21.15.135:0/2351850533 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f6875915640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317d4a0 0x55c863140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.254+0000 7f6876116640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c863186360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875114640 1 -- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317d4a0 msgr2=0x55c863140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875114640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317d4a0 0x55c863140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875114640 1 -- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 msgr2=0x55c863186360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875114640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c863186360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875114640 1 -- 172.21.15.135:0/2351850533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c862db72d0 con 0x55c86317da00 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875915640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317d4a0 0x55c863140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875114640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 0x55c8631794e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6860009be0 tx=0x7f68600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11acd1210af8aa52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6850ff9640 1 -- 172.21.15.135:0/2351850533 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860040070 con 0x55c86317da00 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6850ff9640 1 -- 172.21.15.135:0/2351850533 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f6860003610 con 0x55c86317da00 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6860004300 con 0x55c86317da00 2024-04-26T22:00:27.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6850ff9640 1 -- 172.21.15.135:0/2351850533 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f6860056050 con 0x55c86317da00 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 10 client.4905 fetch_fsmap learned FSMap version 13 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x55c862e816b0 con 0x55c86317da00 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6850ff9640 1 -- 172.21.15.135:0/2351850533 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6860004300 con 0x55c86317da00 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6850ff9640 1 -- 172.21.15.135:0/2351850533 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 13) ==== 4128+0+0 (secure 0 0 0) 0x7f6860059020 con 0x55c86317da00 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 10 client.4905 fetch_fsmap finished waiting for FSMap version 13 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 10 client.4905 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 20 client.4905 populate_metadata read hostname 'smithi135' 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c8631949a0 0x55c863194d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 4 client.4905 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 --> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8630b3350 con 0x55c8631949a0 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.255+0000 7f6875915640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c8631949a0 0x55c863194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f6875915640 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c8631949a0 0x55c863194d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cbd058028bc55 server_cookie=3e422ec95d1365e8 in_seq=0 out_seq=0 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f6850ff9640 10 client.4905 ms_handle_connect on v2:172.21.15.135:6834/3981978490 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f6850ff9640 1 -- 172.21.15.135:0/2351850533 <== mds.0 v2:172.21.15.135:6834/3981978490 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8630b3350 con 0x55c8631949a0 2024-04-26T22:00:27.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f6850ff9640 10 client.4905 handle_command_reply: tid=0 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f687a968740 1 client.4905 shutdown 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 msgr2=0x55c8631794e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.256+0000 7f687a968740 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 0x55c8631794e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6860009be0 tx=0x7f68600046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 shutdown_connections 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c8631949a0 0x55c863194d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c863186360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317da00 0x55c8631794e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f6850ff9640 0 client.4905 ms_handle_reset on v2:172.21.15.135:6834/3981978490 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 --2- 172.21.15.135:0/2351850533 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317d4a0 0x55c863140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 >> 172.21.15.135:0/2351850533 conn(0x55c86317d0c0 msgr2=0x55c86317e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 shutdown_connections 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 1 -- 172.21.15.135:0/2351850533 wait complete. 2024-04-26T22:00:27.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.257+0000 7f687a968740 20 client.4905 trim_cache size 0 max 16384 2024-04-26T22:00:27.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 Processor -- start 2024-04-26T22:00:27.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 -- start start 2024-04-26T22:00:27.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d4a0 0x55c863180530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 0x55c8631a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68701f7470 con 0x55c8631874a0 2024-04-26T22:00:27.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68701f8500 con 0x55c86317d4a0 2024-04-26T22:00:27.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68700c01f0 con 0x55c86317da00 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.275+0000 7f6876116640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 0x55c8631a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 0x55c8631a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58280/0 (socket says 172.21.15.135:58280) 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 -- 172.21.15.135:0/3119839275 learned_addr learned my addr 172.21.15.135:0/3119839275 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6875915640 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d4a0 0x55c863180530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6875114640 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 -- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 msgr2=0x55c8631a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 -- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d4a0 msgr2=0x55c863180530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d4a0 0x55c863180530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 -- 172.21.15.135:0/3119839275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86317fae0 con 0x55c8631874a0 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6876116640 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 0x55c8631a5430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f68680050a0 tx=0x7f6868058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690249c9866d24b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6852ffd640 1 -- 172.21.15.135:0/3119839275 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686805d730 con 0x55c8631874a0 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6852ffd640 1 -- 172.21.15.135:0/3119839275 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6868088070 con 0x55c8631874a0 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.276+0000 7f6852ffd640 1 -- 172.21.15.135:0/3119839275 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868061f60 con 0x55c8631874a0 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 -- 172.21.15.135:0/3119839275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 msgr2=0x55c8631a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 0x55c8631a5430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f68680050a0 tx=0x7f6868058ce0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 -- 172.21.15.135:0/3119839275 shutdown_connections 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8631874a0 0x55c8631a5430 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 --2- 172.21.15.135:0/3119839275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c86317d4a0 0x55c863180530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 -- 172.21.15.135:0/3119839275 >> 172.21.15.135:0/3119839275 conn(0x55c86317d0c0 msgr2=0x55c863183030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 -- 172.21.15.135:0/3119839275 shutdown_connections 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 -- 172.21.15.135:0/3119839275 wait complete. 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 -- start start 2024-04-26T22:00:27.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.277+0000 7f687a968740 1 Processor -- start 2024-04-26T22:00:27.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c863189d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c8631f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c86313cbb0 con 0x55c86317d4a0 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c86302cfe0 con 0x55c8631874a0 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55c863186d70 con 0x55c86317da00 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f687a968740 10 client.0 _ll_register_callbacks cb 0x7f685c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c863189d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c863189d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58282/0 (socket says 172.21.15.135:58282) 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 -- 172.21.15.135:0/1155885859 learned_addr learned my addr 172.21.15.135:0/1155885859 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875114640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6876116640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c8631f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 -- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 msgr2=0x55c8631fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 -- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 msgr2=0x55c8631f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c8631f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.278+0000 7f6875915640 1 -- 172.21.15.135:0/1155885859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86318a4d0 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6875915640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c863189d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55c863186fc0 tx=0x7f6870153ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3576ea1bce2cba3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6850ff9640 1 -- 172.21.15.135:0/1155885859 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0229e0 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6850ff9640 1 -- 172.21.15.135:0/1155885859 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f686c02ebb0 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6868088070 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6850ff9640 1 -- 172.21.15.135:0/1155885859 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f686c039050 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 10 client.4911 fetch_fsmap learned FSMap version 13 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x55c8631fc2c0 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6850ff9640 1 -- 172.21.15.135:0/1155885859 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6868088070 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6850ff9640 1 -- 172.21.15.135:0/1155885859 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 13) ==== 4128+0+0 (secure 0 0 0) 0x7f686c022b80 con 0x55c86317d4a0 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 10 client.4911 fetch_fsmap finished waiting for FSMap version 13 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 10 client.4911 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T22:00:27.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 20 client.4911 populate_metadata read hostname 'smithi135' 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c863164b30 0x55c863164f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 4 client.4911 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 --> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] -- command(tid 0: {"prefix": "status"}) -- 0x55c8630ad1c0 con 0x55c863164b30 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.279+0000 7f6875114640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c863164b30 0x55c863164f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f6850ff9640 10 client.4911 ms_handle_connect on v2:172.21.15.135:6834/3981978490 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f6875114640 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c863164b30 0x55c863164f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0babd7d84f5b76e server_cookie=cf5631698a36aa29 in_seq=0 out_seq=0 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f6850ff9640 1 -- 172.21.15.135:0/1155885859 <== mds.0 v2:172.21.15.135:6834/3981978490 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55c8630ad1c0 con 0x55c863164b30 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f6850ff9640 10 client.4911 handle_command_reply: tid=0 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 client.4911 shutdown 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 msgr2=0x55c863189d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c863189d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55c863186fc0 tx=0x7f6870153ee0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 shutdown_connections 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:6834/3981978490,v1:172.21.15.135:6836/3981978490] conn(0x55c863164b30 0x55c863164f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c8631874a0 0x55c8631f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c86317da00 0x55c8631fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f6850ff9640 0 client.4911 ms_handle_reset on v2:172.21.15.135:6834/3981978490 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 --2- 172.21.15.135:0/1155885859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c86317d4a0 0x55c863189d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 >> 172.21.15.135:0/1155885859 conn(0x55c86317d0c0 msgr2=0x55c863136cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 shutdown_connections 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 1 -- 172.21.15.135:0/1155885859 wait complete. 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.280+0000 7f687a968740 20 client.4911 trim_cache size 0 max 16384 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_fsid": "c1882db8-5327-4bd0-9178-0a7c5b1ec94f", 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stdout: "whoami": 0, 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stdout: "id": 4275, 2024-04-26T22:00:27.287 INFO:teuthology.orchestra.run.smithi135.stdout: "want_state": "up:active", 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "state": "up:active", 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "b", 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "rank_uptime": 20.479693326, 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "mdsmap_epoch": 13, 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch": 24, 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout: "uptime": 36.680170660999998 2024-04-26T22:00:27.288 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f685406c8f0 msgr2=0x7f685406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f685406c8f0 0x7f685406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68700762b0 tx=0x7f686c006040 comp rx=0 tx=0).stop 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 msgr2=0x7f68701ed970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 0x7f68701ed970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68700c5c40 tx=0x7f686800f920 comp rx=0 tx=0).stop 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 shutdown_connections 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f685406c8f0 0x7f685406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6870144ac0 0x7f68701ed970 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68700c33e0 0x7f6870144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 --2- 172.21.15.135:0/3942888653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685c003f60 0x7f6870075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 >> 172.21.15.135:0/3942888653 conn(0x7f68700c5ce0 msgr2=0x7f68701533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 shutdown_connections 2024-04-26T22:00:27.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.281+0000 7f6877b79640 1 -- 172.21.15.135:0/3942888653 wait complete. 2024-04-26T22:00:27.291 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T22:00:27.305 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T22:00:27.305 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T22:00:27.305 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T22:00:27.305 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T22:00:27.328 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T22:00:27.328 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-04-26T22:00:27.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:27.328 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-26T22:00:27.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:27.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-04-26T22:00:27.408 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2024-04-26T22:00:27.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:00:27.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:27.411 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-26T22:00:27.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:27.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-04-26T22:00:27.508 INFO:teuthology.orchestra.run.smithi139.stderr:Cannot find device "ceph-brx" 2024-04-26T22:00:27.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:00:27.510 INFO:tasks.kclient:client.0 config is {'syntax': 'v2', 'cephfs_name': 'a'} 2024-04-26T22:00:27.510 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:00:27.511 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:27.511 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:00:27.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 Processor -- start 2024-04-26T22:00:27.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 -- start start 2024-04-26T22:00:27.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23480c40e0 0x7f23480720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c46e0 0x7f23480c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2348154ba0 con 0x7f23480c3060 2024-04-26T22:00:27.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2348154e80 con 0x7f23480c40e0 2024-04-26T22:00:27.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.617+0000 7f234e946640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2348155160 con 0x7f23480c46e0 2024-04-26T22:00:27.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23480c40e0 0x7f23480720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f234cee3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c46e0 0x7f23480c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23480c40e0 0x7f23480720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:47812/0 (socket says 172.21.15.135:47812) 2024-04-26T22:00:27.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f2347fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f23480c3060 0x7f2348151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:42066/0 (socket says 172.21.15.135:42066) 2024-04-26T22:00:27.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 -- 172.21.15.135:0/2017793485 learned_addr learned my addr 172.21.15.135:0/2017793485 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 -- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c46e0 msgr2=0x7f23480c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:27.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 --2- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c46e0 0x7f23480c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 -- 172.21.15.135:0/2017793485 >> v1:172.21.15.139:6789/0 conn(0x7f23480c3060 legacy=0x7f2348151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f2346ffd640 1 -- 172.21.15.135:0/2017793485 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292531875 0 0) 0x7f2348154ba0 con 0x7f23480c3060 2024-04-26T22:00:27.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f23477fe640 1 -- 172.21.15.135:0/2017793485 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348155440 con 0x7f23480c40e0 2024-04-26T22:00:27.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.618+0000 7f234cee3640 1 --2- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c46e0 0x7f23480c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.619+0000 7f23477fe640 1 --2- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23480720a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2338009870 tx=0x7f233802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4534e012102cba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.619+0000 7f2346ffd640 1 -- 172.21.15.135:0/2017793485 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338037070 con 0x7f23480c40e0 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.619+0000 7f2346ffd640 1 -- 172.21.15.135:0/2017793485 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2338032440 con 0x7f23480c40e0 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.619+0000 7f2346ffd640 1 -- 172.21.15.135:0/2017793485 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338032810 con 0x7f23480c40e0 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.619+0000 7f234e946640 1 -- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 msgr2=0x7f23480720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.619+0000 7f234e946640 1 --2- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23480720a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2338009870 tx=0x7f233802a1c0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.620+0000 7f234e946640 1 -- 172.21.15.135:0/2017793485 shutdown_connections 2024-04-26T22:00:27.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.620+0000 7f234e946640 1 --2- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c46e0 0x7f23480c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.620+0000 7f234e946640 1 --2- 172.21.15.135:0/2017793485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23480720a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.620+0000 7f234e946640 1 -- 172.21.15.135:0/2017793485 >> 172.21.15.135:0/2017793485 conn(0x7f234806c310 msgr2=0x7f234806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.620+0000 7f234e946640 1 -- 172.21.15.135:0/2017793485 shutdown_connections 2024-04-26T22:00:27.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.620+0000 7f234e946640 1 -- 172.21.15.135:0/2017793485 wait complete. 2024-04-26T22:00:27.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 Processor -- start 2024-04-26T22:00:27.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 -- start start 2024-04-26T22:00:27.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c3060 0x7f2348149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23481ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 0x7f234813d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23480c3440 con 0x7f23480c46e0 2024-04-26T22:00:27.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f23480c3aa0 con 0x7f23480c40e0 2024-04-26T22:00:27.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.621+0000 7f234e946640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f23480c35c0 con 0x7f23480c3060 2024-04-26T22:00:27.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 0x7f234813d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 0x7f234813d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58296/0 (socket says 172.21.15.135:58296) 2024-04-26T22:00:27.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 -- 172.21.15.135:0/457375352 learned_addr learned my addr 172.21.15.135:0/457375352 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:27.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f23477fe640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23481ed3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f2347fff640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c3060 0x7f2348149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 -- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c3060 msgr2=0x7f2348149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c3060 0x7f2348149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 -- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 msgr2=0x7f23481ed3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23481ed3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 -- 172.21.15.135:0/457375352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23481ea060 con 0x7f23480c46e0 2024-04-26T22:00:27.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234cee3640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 0x7f234813d570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f234000efc0 tx=0x7f2340009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340018030 con 0x7f23480c46e0 2024-04-26T22:00:27.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234000cde0 con 0x7f23480c46e0 2024-04-26T22:00:27.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23480c7d80 con 0x7f23480c46e0 2024-04-26T22:00:27.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340003bb0 con 0x7f23480c46e0 2024-04-26T22:00:27.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.622+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2348149ed0 con 0x7f23480c46e0 2024-04-26T22:00:27.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.623+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23480507a0 con 0x7f23480c46e0 2024-04-26T22:00:27.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.623+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2340005d10 con 0x7f23480c46e0 2024-04-26T22:00:27.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.624+0000 7f2344ff9640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f233406c920 0x7f233406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:27.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.624+0000 7f2347fff640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f233406c920 0x7f233406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:27.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.624+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f234008c750 con 0x7f23480c46e0 2024-04-26T22:00:27.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.624+0000 7f2347fff640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f233406c920 0x7f233406ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f233c000f60 tx=0x7f233c007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:27.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.628+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2340022070 con 0x7f23480c46e0 2024-04-26T22:00:27.676 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:00:27.682 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:00:27.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.844+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f234806fb80 con 0x7f23480c46e0 2024-04-26T22:00:27.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.844+0000 7f2344ff9640 1 -- 172.21.15.135:0/457375352 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+116 (secure 0 0 0) 0x7f2340010040 con 0x7f23480c46e0 2024-04-26T22:00:27.844 INFO:teuthology.orchestra.run.smithi135.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-04-26T22:00:27.844 INFO:teuthology.orchestra.run.smithi135.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f233406c920 msgr2=0x7f233406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f233406c920 0x7f233406ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f233c000f60 tx=0x7f233c007c00 comp rx=0 tx=0).stop 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 msgr2=0x7f234813d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 0x7f234813d570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f234000efc0 tx=0x7f2340009f90 comp rx=0 tx=0).stop 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 shutdown_connections 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f233406c920 0x7f233406ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23480c46e0 0x7f234813d570 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23480c40e0 0x7f23481ed3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 --2- 172.21.15.135:0/457375352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23480c3060 0x7f2348149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:27.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 >> 172.21.15.135:0/457375352 conn(0x7f234806c310 msgr2=0x7f234806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:27.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 shutdown_connections 2024-04-26T22:00:27.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27.846+0000 7f234e946640 1 -- 172.21.15.135:0/457375352 wait complete. 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:Mounting Ceph FS a; just confirmed its presence on cluster 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:self.cephfs_name = a 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:00:27.857 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:00:27.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:27.858 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-04-26T22:00:27.874 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-04-26T22:00:27.875 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42722sec preferred_lft 42722sec 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::f5dd:6dc8:6093:8a01/64 scope link noprefixroute 2024-04-26T22:00:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:00:27.876 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr flush dev ceph-brx 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set ceph-brx up 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:00:27.877 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-26T22:00:27.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:28.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:28.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:28.053 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:00:28.077 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-04-26T22:00:28.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:28.078 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-04-26T22:00:28.153 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-04-26T22:00:28.153 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:00:28.153 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:00:28.154 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:00:28.154 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:00:28.154 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:28.154 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-26T22:00:28.154 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:00:28.154 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:00:28.154 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:00:28.155 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-26T22:00:28.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:28.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:28.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:28.397 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-26T22:00:28.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:28.412 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2024-04-26T22:00:28.468 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:28.468 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-26T22:00:28.468 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:00:28.468 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T22:00:28.468 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-26T22:00:28.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:28.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:28.582 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T22:00:28.583 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-26T22:00:28.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:28.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:28.704 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:28.704 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-26T22:00:28.704 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.0 up 2024-04-26T22:00:28.704 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:00:28.704 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-26T22:00:28.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:28.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:28.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:00:28.778 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:28.802 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:00:28.802 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:00:28.803 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:28.857 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.a=/ 2024-04-26T22:00:28.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:28.857 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.a=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T22:00:28.948 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T22:00:28.948 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T22:00:29.054 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T22:00:29.055 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T22:00:29.055 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.139:3300,172.21.15.139:3301" 2024-04-26T22:00:29.055 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with new device syntax: 0@c1882db8-5327-4bd0-9178-0a7c5b1ec94f.a=/ 2024-04-26T22:00:29.055 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.135:3300/172.21.15.139:3300/172.21.15.139:3301" will pass to kernel 2024-04-26T22:00:29.055 INFO:teuthology.orchestra.run.smithi135.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T22:00:29.055 INFO:teuthology.orchestra.run.smithi135.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:00:29.056 INFO:teuthology.orchestra.run.smithi135.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:00:29.056 INFO:teuthology.orchestra.run.smithi135.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:00:29.056 INFO:teuthology.orchestra.run.smithi135.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:00:29.057 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:00:29.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:29.057 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:29.088 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:00:29.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:29.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:29.187 INFO:teuthology.orchestra.run.smithi135.stdout:client4774 2024-04-26T22:00:29.188 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T22:00:29.188 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:00:29.203 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:00:29.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:29.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:29.303 INFO:teuthology.orchestra.run.smithi135.stdout:client4774 2024-04-26T22:00:29.304 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:00:29.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:29.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:29.363 INFO:teuthology.orchestra.run.smithi135.stdout:client4774 2024-04-26T22:00:29.363 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1882db8-5327-4bd0-9178-0a7c5b1ec94f.client4774/status') 2024-04-26T22:00:29.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:29.423 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4774 (3)172.21.15.135:0/1049686908 2024-04-26T22:00:29.423 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2024-04-26T22:00:29.424 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2024-04-26T22:00:29.424 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2024-04-26T22:00:29.424 INFO:teuthology.orchestra.run.smithi135.stderr:71 bytes copied, 2.2311e-05 s, 3.2 MB/s 2024-04-26T22:00:29.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:29.431 INFO:tasks.kclient:client.1 config is {'syntax': 'v2', 'cephfs_name': 'b'} 2024-04-26T22:00:29.431 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:00:29.431 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:29.431 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:00:29.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.528+0000 7efca9afc640 1 Processor -- start 2024-04-26T22:00:29.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.528+0000 7efca9afc640 1 -- start start 2024-04-26T22:00:29.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca9afc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40bfa80 0x7efca415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca9afc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efca40ca2f0 0x7efca415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca9afc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca4157840 con 0x7efca40b1ac0 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca9afc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efca40b1ea0 con 0x7efca40ca2f0 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca9afc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efca40739f0 con 0x7efca40bfa80 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca3fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efca40ca2f0 0x7efca415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca3fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efca40ca2f0 0x7efca415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40864/0 (socket says 172.21.15.135:40864) 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca3fff640 1 -- 172.21.15.135:0/1319431388 learned_addr learned my addr 172.21.15.135:0/1319431388 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.529+0000 7efca2ffd640 1 --2- 172.21.15.135:0/1319431388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40bfa80 0x7efca415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca3fff640 1 -- 172.21.15.135:0/1319431388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40bfa80 msgr2=0x7efca415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca3fff640 1 --2- 172.21.15.135:0/1319431388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40bfa80 0x7efca415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca3fff640 1 -- 172.21.15.135:0/1319431388 >> v1:172.21.15.139:6789/0 conn(0x7efca40b1ac0 legacy=0x7efca40c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:29.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca3fff640 1 -- 172.21.15.135:0/1319431388 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca406db60 con 0x7efca40ca2f0 2024-04-26T22:00:29.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca3fff640 1 --2- 172.21.15.135:0/1319431388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40ca2f0 0x7efca415a7e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e50 tx=0x7efc9c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabe6612094817a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca27fc640 1 -- 172.21.15.135:0/1319431388 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c036070 con 0x7efca40ca2f0 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca27fc640 1 -- 172.21.15.135:0/1319431388 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc9c031600 con 0x7efca40ca2f0 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.530+0000 7efca27fc640 1 -- 172.21.15.135:0/1319431388 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0319d0 con 0x7efca40ca2f0 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 -- 172.21.15.135:0/1319431388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40ca2f0 msgr2=0x7efca415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 --2- 172.21.15.135:0/1319431388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40ca2f0 0x7efca415a7e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e50 tx=0x7efc9c02a680 comp rx=0 tx=0).stop 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 -- 172.21.15.135:0/1319431388 shutdown_connections 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 --2- 172.21.15.135:0/1319431388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40ca2f0 0x7efca415a7e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 --2- 172.21.15.135:0/1319431388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40bfa80 0x7efca415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 -- 172.21.15.135:0/1319431388 >> 172.21.15.135:0/1319431388 conn(0x7efca40cc7e0 msgr2=0x7efca40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 -- 172.21.15.135:0/1319431388 shutdown_connections 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.531+0000 7efca9afc640 1 -- 172.21.15.135:0/1319431388 wait complete. 2024-04-26T22:00:29.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.532+0000 7efca9afc640 1 Processor -- start 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.532+0000 7efca9afc640 1 -- start start 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca9afc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40b1ac0 0x7efca4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca9afc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 0x7efca41ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca9afc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca40ca2f0 0x7efca414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca9afc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efca406e120 con 0x7efca40bfa80 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca9afc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efca406d910 con 0x7efca40ca2f0 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca9afc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efca41574a0 con 0x7efca40b1ac0 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 0x7efca41ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 0x7efca41ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40878/0 (socket says 172.21.15.135:40878) 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 -- 172.21.15.135:0/619279699 learned_addr learned my addr 172.21.15.135:0/619279699 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca3fff640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca40ca2f0 0x7efca414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca37fe640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40b1ac0 0x7efca4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 -- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40b1ac0 msgr2=0x7efca4153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:29.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40b1ac0 0x7efca4153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 -- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca40ca2f0 msgr2=0x7efca414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca40ca2f0 0x7efca414c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.533+0000 7efca2ffd640 1 -- 172.21.15.135:0/619279699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca40ac230 con 0x7efca40bfa80 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.534+0000 7efca2ffd640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 0x7efca41ee350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d800 tx=0x7efc9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.534+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800b6c0 con 0x7efca40bfa80 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.534+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc98019070 con 0x7efca40bfa80 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.534+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca41f05f0 con 0x7efca40bfa80 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.534+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800ba80 con 0x7efca40bfa80 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.534+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca4153850 con 0x7efca40bfa80 2024-04-26T22:00:29.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.535+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca40694b0 con 0x7efca40bfa80 2024-04-26T22:00:29.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.535+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc9800bc20 con 0x7efca40bfa80 2024-04-26T22:00:29.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.536+0000 7efc8bfff640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc9006c920 0x7efc9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:29.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.536+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7efc9808c0f0 con 0x7efca40bfa80 2024-04-26T22:00:29.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.536+0000 7efca37fe640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc9006c920 0x7efc9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:29.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.536+0000 7efca37fe640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc9006c920 0x7efc9006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc94001780 tx=0x7efc94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:29.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.540+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efca40694b0 con 0x7efca40bfa80 2024-04-26T22:00:29.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.756+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efca40c9fc0 con 0x7efca40bfa80 2024-04-26T22:00:29.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.757+0000 7efc8bfff640 1 -- 172.21.15.135:0/619279699 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+116 (secure 0 0 0) 0x7efca40c9fc0 con 0x7efca40bfa80 2024-04-26T22:00:29.757 INFO:teuthology.orchestra.run.smithi135.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-04-26T22:00:29.757 INFO:teuthology.orchestra.run.smithi135.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc9006c920 msgr2=0x7efc9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc9006c920 0x7efc9006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc94001780 tx=0x7efc94009040 comp rx=0 tx=0).stop 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 msgr2=0x7efca41ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 0x7efca41ee350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d800 tx=0x7efc9800dcc0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 shutdown_connections 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc9006c920 0x7efc9006ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca40ca2f0 0x7efca414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efca40bfa80 0x7efca41ee350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 --2- 172.21.15.135:0/619279699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efca40b1ac0 0x7efca4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 >> 172.21.15.135:0/619279699 conn(0x7efca40cc7e0 msgr2=0x7efca41596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 shutdown_connections 2024-04-26T22:00:29.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:29.759+0000 7efca9afc640 1 -- 172.21.15.135:0/619279699 wait complete. 2024-04-26T22:00:29.769 INFO:tasks.cephfs.mount:Mounting Ceph FS b; just confirmed its presence on cluster 2024-04-26T22:00:29.769 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:00:29.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-26T22:00:29.770 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T22:00:29.770 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:29.770 INFO:tasks.cephfs.mount:self.cephfs_name = b 2024-04-26T22:00:29.770 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:00:29.770 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:00:29.770 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:00:29.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:29.770 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-26T22:00:29.788 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-26T22:00:29.789 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42699sec preferred_lft 42699sec 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::3f73:e9db:d625:df54/64 scope link noprefixroute 2024-04-26T22:00:29.790 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:00:29.790 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:00:29.790 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:29.790 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-26T22:00:29.790 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:00:29.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-04-26T22:00:29.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-04-26T22:00:29.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:00:29.791 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-26T22:00:29.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:29.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:29.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:29.983 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:00:30.007 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-26T22:00:30.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:30.008 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-26T22:00:30.089 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-26T22:00:30.089 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:00:30.089 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:00:30.089 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:00:30.090 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:00:30.090 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:30.090 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-26T22:00:30.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:00:30.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:00:30.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:00:30.090 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-26T22:00:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:30.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:30.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:30.358 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-26T22:00:30.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:30.373 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-04-26T22:00:30.428 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:30.428 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-26T22:00:30.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T22:00:30.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T22:00:30.428 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-26T22:00:30.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:30.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:30.544 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T22:00:30.545 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-26T22:00:30.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:30.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:30.726 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:00:30.726 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-26T22:00:30.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-26T22:00:30.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:00:30.726 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-26T22:00:30.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:30.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:30.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:00:30.801 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:30.817 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:00:30.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:00:30.817 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:30.871 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.b=/ 2024-04-26T22:00:30.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:30.872 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.b=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T22:00:30.966 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T22:00:30.966 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T22:00:31.097 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T22:00:31.097 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T22:00:31.097 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.139:3300,172.21.15.139:3301" 2024-04-26T22:00:31.097 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 1@c1882db8-5327-4bd0-9178-0a7c5b1ec94f.b=/ 2024-04-26T22:00:31.097 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.135:3300/172.21.15.139:3300/172.21.15.139:3301" will pass to kernel 2024-04-26T22:00:31.098 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T22:00:31.098 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:00:31.098 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:00:31.098 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:00:31.098 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:00:31.099 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:00:31.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:31.100 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:31.123 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:00:31.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:31.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:31.224 INFO:teuthology.orchestra.run.smithi139.stdout:client4923 2024-04-26T22:00:31.224 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T22:00:31.225 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:00:31.240 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:00:31.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:31.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:31.344 INFO:teuthology.orchestra.run.smithi139.stdout:client4923 2024-04-26T22:00:31.344 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:00:31.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:00:31.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:31.404 INFO:teuthology.orchestra.run.smithi139.stdout:client4923 2024-04-26T22:00:31.404 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1882db8-5327-4bd0-9178-0a7c5b1ec94f.client4923/status') 2024-04-26T22:00:31.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:00:31.466 INFO:teuthology.orchestra.run.smithi139.stdout:instance: client.4923 (3)172.21.15.139:0/664348076 2024-04-26T22:00:31.467 INFO:teuthology.orchestra.run.smithi139.stdout:blocklisted: false 2024-04-26T22:00:31.467 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records in 2024-04-26T22:00:31.467 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records out 2024-04-26T22:00:31.467 INFO:teuthology.orchestra.run.smithi139.stderr:70 bytes copied, 2.193e-05 s, 3.2 MB/s 2024-04-26T22:00:31.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-26T22:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:00:31.474 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-04-26T22:00:31.486 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:31.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.594+0000 7f0b5aa99640 1 Processor -- start 2024-04-26T22:00:31.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.594+0000 7f0b5aa99640 1 -- start start 2024-04-26T22:00:31.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b5aa99640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b540c4250 0x7f0b540c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b5aa99640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0b540c3cd0 0x7f0b541572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b5aa99640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b540ad010 con 0x7f0b540acc30 2024-04-26T22:00:31.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b5aa99640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0b540c8ba0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b5aa99640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0b54156e50 con 0x7f0b540c4250 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b59036640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0b540c3cd0 0x7f0b541572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b58835640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f0b540acc30 0x7f0b5415abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39208/0 (socket says 172.21.15.135:39208) 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b58835640 1 -- 172.21.15.135:0/3575834381 learned_addr learned my addr 172.21.15.135:0/3575834381 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b53fff640 1 --2- 172.21.15.135:0/3575834381 >> v2:172.21.15.139:3300/0 conn(0x7f0b540c4250 0x7f0b540c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b59036640 1 -- 172.21.15.135:0/3575834381 >> v2:172.21.15.139:3300/0 conn(0x7f0b540c4250 msgr2=0x7f0b540c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b59036640 1 --2- 172.21.15.135:0/3575834381 >> v2:172.21.15.139:3300/0 conn(0x7f0b540c4250 0x7f0b540c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b59036640 1 -- 172.21.15.135:0/3575834381 >> v1:172.21.15.135:6789/0 conn(0x7f0b540acc30 legacy=0x7f0b5415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b59036640 1 -- 172.21.15.135:0/3575834381 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540c8300 con 0x7f0b540c3cd0 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b537fe640 1 -- 172.21.15.135:0/3575834381 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329469749 0 0) 0x7f0b540ad010 con 0x7f0b540acc30 2024-04-26T22:00:31.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.595+0000 7f0b53fff640 1 --2- 172.21.15.135:0/3575834381 >> v2:172.21.15.139:3300/0 conn(0x7f0b540c4250 0x7f0b540c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:31.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.596+0000 7f0b59036640 1 --2- 172.21.15.135:0/3575834381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b541572e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b54071a50 tx=0x7f0b4c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b90ac44603735c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:31.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.596+0000 7f0b537fe640 1 -- 172.21.15.135:0/3575834381 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c007740 con 0x7f0b540c3cd0 2024-04-26T22:00:31.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.596+0000 7f0b537fe640 1 -- 172.21.15.135:0/3575834381 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b4c0078e0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.596+0000 7f0b537fe640 1 -- 172.21.15.135:0/3575834381 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c007cb0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 -- 172.21.15.135:0/3575834381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 msgr2=0x7f0b541572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/3575834381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b541572e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b54071a50 tx=0x7f0b4c02a5f0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 -- 172.21.15.135:0/3575834381 shutdown_connections 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/3575834381 >> v2:172.21.15.139:3300/0 conn(0x7f0b540c4250 0x7f0b540c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/3575834381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b541572e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 -- 172.21.15.135:0/3575834381 >> 172.21.15.135:0/3575834381 conn(0x7f0b540c6e60 msgr2=0x7f0b540c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 -- 172.21.15.135:0/3575834381 shutdown_connections 2024-04-26T22:00:31.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.597+0000 7f0b5aa99640 1 -- 172.21.15.135:0/3575834381 wait complete. 2024-04-26T22:00:31.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.598+0000 7f0b5aa99640 1 Processor -- start 2024-04-26T22:00:31.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.598+0000 7f0b5aa99640 1 -- start start 2024-04-26T22:00:31.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b5aa99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b540acc30 0x7f0b541592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b5aa99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b54070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b5aa99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b540c4250 0x7f0b54071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b5aa99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540abec0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b5aa99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540c88c0 con 0x7f0b540acc30 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b5aa99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0b540ac040 con 0x7f0b540c4250 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b54070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b54070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40906/0 (socket says 172.21.15.135:40906) 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 -- 172.21.15.135:0/744503104 learned_addr learned my addr 172.21.15.135:0/744503104 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b58835640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b540acc30 0x7f0b541592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b59036640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b540c4250 0x7f0b54071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 -- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b540c4250 msgr2=0x7f0b54071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b540c4250 0x7f0b54071060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 -- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b540acc30 msgr2=0x7f0b541592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b540acc30 0x7f0b541592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.599+0000 7f0b53fff640 1 -- 172.21.15.135:0/744503104 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b541e0400 con 0x7f0b540c3cd0 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.600+0000 7f0b53fff640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b54070b20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800d830 tx=0x7f0b4800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:31.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.600+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48004030 con 0x7f0b540c3cd0 2024-04-26T22:00:31.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.600+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b480041d0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.600+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b541ed430 con 0x7f0b540c3cd0 2024-04-26T22:00:31.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.600+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480096a0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.600+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b541e06c0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.601+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b540ac390 con 0x7f0b540c3cd0 2024-04-26T22:00:31.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.601+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b48009a50 con 0x7f0b540c3cd0 2024-04-26T22:00:31.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.602+0000 7f0b517fa640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0b4006c8f0 0x7f0b4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.602+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0b4808c180 con 0x7f0b540c3cd0 2024-04-26T22:00:31.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.602+0000 7f0b58835640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0b4006c8f0 0x7f0b4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.602+0000 7f0b58835640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0b4006c8f0 0x7f0b4006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b44000f60 tx=0x7f0b44007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:31.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.606+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b540ac390 con 0x7f0b540c3cd0 2024-04-26T22:00:31.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.781+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b540507a0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.782+0000 7f0b517fa640 1 -- 172.21.15.135:0/744503104 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f0b48059cc0 con 0x7f0b540c3cd0 2024-04-26T22:00:31.782 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:31.783 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:31.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.784+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0b4006c8f0 msgr2=0x7f0b4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.784+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0b4006c8f0 0x7f0b4006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b44000f60 tx=0x7f0b44007d50 comp rx=0 tx=0).stop 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.784+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 msgr2=0x7f0b54070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.784+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b54070b20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800d830 tx=0x7f0b4800dcf0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 shutdown_connections 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0b4006c8f0 0x7f0b4006edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b540c4250 0x7f0b54071060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b540c3cd0 0x7f0b54070b20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 --2- 172.21.15.135:0/744503104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b540acc30 0x7f0b541592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 >> 172.21.15.135:0/744503104 conn(0x7f0b540c6e60 msgr2=0x7f0b54157490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 shutdown_connections 2024-04-26T22:00:31.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.785+0000 7f0b5aa99640 1 -- 172.21.15.135:0/744503104 wait complete. 2024-04-26T22:00:31.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:31.796 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:31.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.896+0000 7fbefd322640 1 Processor -- start 2024-04-26T22:00:31.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.896+0000 7fbefd322640 1 -- start start 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbefd322640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbef8154160 0x7fbef8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbefd322640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbef815a1a0 0x7fbef80c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbefd322640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef80c8df0 con 0x7fbef80ca5a0 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbefd322640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbef80c9100 con 0x7fbef8154160 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbefd322640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbef8154540 con 0x7fbef815a1a0 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbef8154160 0x7fbef8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbef8154160 0x7fbef8156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40916/0 (socket says 172.21.15.135:40916) 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef67fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbef80ca5a0 0x7fbef80c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39222/0 (socket says 172.21.15.135:39222) 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 -- 172.21.15.135:0/851560890 learned_addr learned my addr 172.21.15.135:0/851560890 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef77fe640 1 --2- 172.21.15.135:0/851560890 >> v2:172.21.15.139:3300/0 conn(0x7fbef815a1a0 0x7fbef80c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 -- 172.21.15.135:0/851560890 >> v2:172.21.15.139:3300/0 conn(0x7fbef815a1a0 msgr2=0x7fbef80c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:31.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 --2- 172.21.15.135:0/851560890 >> v2:172.21.15.139:3300/0 conn(0x7fbef815a1a0 0x7fbef80c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 -- 172.21.15.135:0/851560890 >> v1:172.21.15.135:6789/0 conn(0x7fbef80ca5a0 legacy=0x7fbef80c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef6ffd640 1 -- 172.21.15.135:0/851560890 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef815a910 con 0x7fbef8154160 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.897+0000 7fbef5ffb640 1 -- 172.21.15.135:0/851560890 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 24415901 0 0) 0x7fbeec001110 con 0x7fbef80ca5a0 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.898+0000 7fbef6ffd640 1 --2- 172.21.15.135:0/851560890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef8154160 0x7fbef8156bb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbef80c9280 tx=0x7fbee802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab21a5cdaf94d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.898+0000 7fbef5ffb640 1 -- 172.21.15.135:0/851560890 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037070 con 0x7fbef8154160 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.898+0000 7fbef5ffb640 1 -- 172.21.15.135:0/851560890 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8032440 con 0x7fbef8154160 2024-04-26T22:00:31.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.898+0000 7fbef5ffb640 1 -- 172.21.15.135:0/851560890 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8032810 con 0x7fbef8154160 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.898+0000 7fbefd322640 1 -- 172.21.15.135:0/851560890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef8154160 msgr2=0x7fbef8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.898+0000 7fbefd322640 1 --2- 172.21.15.135:0/851560890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef8154160 0x7fbef8156bb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbef80c9280 tx=0x7fbee802a270 comp rx=0 tx=0).stop 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.899+0000 7fbefd322640 1 -- 172.21.15.135:0/851560890 shutdown_connections 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.899+0000 7fbefd322640 1 --2- 172.21.15.135:0/851560890 >> v2:172.21.15.139:3300/0 conn(0x7fbef815a1a0 0x7fbef80c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.899+0000 7fbefd322640 1 --2- 172.21.15.135:0/851560890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef8154160 0x7fbef8156bb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.899+0000 7fbefd322640 1 -- 172.21.15.135:0/851560890 >> 172.21.15.135:0/851560890 conn(0x7fbef80ae010 msgr2=0x7fbef80ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.899+0000 7fbefd322640 1 -- 172.21.15.135:0/851560890 shutdown_connections 2024-04-26T22:00:31.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.899+0000 7fbefd322640 1 -- 172.21.15.135:0/851560890 wait complete. 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 Processor -- start 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 -- start start 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef80ca5a0 0x7fbef814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef8154160 0x7fbef81f0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 0x7fbef81f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbef815a5b0 con 0x7fbef815a1a0 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbef815aed0 con 0x7fbef80ca5a0 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.900+0000 7fbefd322640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbef80c7510 con 0x7fbef8154160 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 0x7fbef81f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 0x7fbef81f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40930/0 (socket says 172.21.15.135:40930) 2024-04-26T22:00:31.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 -- 172.21.15.135:0/2889397813 learned_addr learned my addr 172.21.15.135:0/2889397813 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef6ffd640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef80ca5a0 0x7fbef814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef67fc640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef8154160 0x7fbef81f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 -- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef8154160 msgr2=0x7fbef81f0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef8154160 0x7fbef81f0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 -- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef80ca5a0 msgr2=0x7fbef814f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef80ca5a0 0x7fbef814f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 -- 172.21.15.135:0/2889397813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef8159d00 con 0x7fbef815a1a0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbef77fe640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 0x7fbef81f0eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbef000efc0 tx=0x7fbef000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0018020 con 0x7fbef815a1a0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0014480 con 0x7fbef815a1a0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef8159690 con 0x7fbef815a1a0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0014870 con 0x7fbef815a1a0 2024-04-26T22:00:31.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.901+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef814f560 con 0x7fbef815a1a0 2024-04-26T22:00:31.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.902+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef8076650 con 0x7fbef815a1a0 2024-04-26T22:00:31.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.903+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbef0005d10 con 0x7fbef815a1a0 2024-04-26T22:00:31.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.903+0000 7fbedf7fe640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbee406c920 0x7fbee406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:31.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.903+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fbef008ca30 con 0x7fbef815a1a0 2024-04-26T22:00:31.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.903+0000 7fbef6ffd640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbee406c920 0x7fbee406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:31.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.904+0000 7fbef6ffd640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbee406c920 0x7fbee406ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:31.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:31.908+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbef0022030 con 0x7fbef815a1a0 2024-04-26T22:00:32.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.083+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbef80cb390 con 0x7fbef815a1a0 2024-04-26T22:00:32.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.083+0000 7fbedf7fe640 1 -- 172.21.15.135:0/2889397813 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fbef0010040 con 0x7fbef815a1a0 2024-04-26T22:00:32.083 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:32.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbee406c920 msgr2=0x7fbee406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbee406c920 0x7fbee406ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).stop 2024-04-26T22:00:32.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 msgr2=0x7fbef81f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 0x7fbef81f0eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbef000efc0 tx=0x7fbef000c490 comp rx=0 tx=0).stop 2024-04-26T22:00:32.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 shutdown_connections 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbee406c920 0x7fbee406ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbef815a1a0 0x7fbef81f0eb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef8154160 0x7fbef81f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 --2- 172.21.15.135:0/2889397813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef80ca5a0 0x7fbef814f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 >> 172.21.15.135:0/2889397813 conn(0x7fbef80ae010 msgr2=0x7fbef80ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 shutdown_connections 2024-04-26T22:00:32.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.085+0000 7fbefd322640 1 -- 172.21.15.135:0/2889397813 wait complete. 2024-04-26T22:00:32.096 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:32.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 Processor -- start 2024-04-26T22:00:32.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 -- start start 2024-04-26T22:00:32.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22200c3060 0x7f22200720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 0x7f2220151e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2220154ba0 con 0x7f2220152360 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2220154e80 con 0x7f22200c3060 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.199+0000 7f2226d19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2220155160 con 0x7f22200c40e0 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f221ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 0x7f2220151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22200c3060 0x7f22200720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22200c3060 0x7f22200720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40090/0 (socket says 172.21.15.135:40090) 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f221ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 0x7f2220151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:59604/0 (socket says 172.21.15.135:59604) 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f22252b6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2220152360 0x7f22200c46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54100/0 (socket says 172.21.15.135:54100) 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 -- 172.21.15.135:0/4013303285 learned_addr learned my addr 172.21.15.135:0/4013303285 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 -- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 msgr2=0x7f2220151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 --2- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 0x7f2220151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 -- 172.21.15.135:0/4013303285 >> v1:172.21.15.139:6789/0 conn(0x7f2220152360 legacy=0x7f22200c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f2224ab5640 1 -- 172.21.15.135:0/4013303285 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2220155440 con 0x7f22200c3060 2024-04-26T22:00:32.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.200+0000 7f221ffff640 1 --2- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 0x7f2220151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:32.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f2224ab5640 1 --2- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f22200720a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2214009870 tx=0x7f221402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9ef7410a7d2c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f221f7fe640 1 -- 172.21.15.135:0/4013303285 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214037070 con 0x7f22200c3060 2024-04-26T22:00:32.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f221f7fe640 1 -- 172.21.15.135:0/4013303285 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2214032440 con 0x7f22200c3060 2024-04-26T22:00:32.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f221f7fe640 1 -- 172.21.15.135:0/4013303285 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214032810 con 0x7f22200c3060 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f2226d19640 1 -- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 msgr2=0x7f22200720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f2226d19640 1 --2- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f22200720a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2214009870 tx=0x7f221402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f2226d19640 1 -- 172.21.15.135:0/4013303285 shutdown_connections 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f2226d19640 1 --2- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22200c40e0 0x7f2220151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.201+0000 7f2226d19640 1 --2- 172.21.15.135:0/4013303285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f22200720a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.202+0000 7f2226d19640 1 -- 172.21.15.135:0/4013303285 >> 172.21.15.135:0/4013303285 conn(0x7f222006c310 msgr2=0x7f222006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.202+0000 7f2226d19640 1 -- 172.21.15.135:0/4013303285 shutdown_connections 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.202+0000 7f2226d19640 1 -- 172.21.15.135:0/4013303285 wait complete. 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 Processor -- start 2024-04-26T22:00:32.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 -- start start 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f2220151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 0x7f222014e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2220152360 0x7f2220153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22200c3440 con 0x7f22200c40e0 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22200c3aa0 con 0x7f22200c3060 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.203+0000 7f2226d19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22200c35c0 con 0x7f2220152360 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 0x7f222014e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 0x7f222014e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40946/0 (socket says 172.21.15.135:40946) 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 -- 172.21.15.135:0/3616293066 learned_addr learned my addr 172.21.15.135:0/3616293066 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f22252b6640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2220152360 0x7f2220153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f2224ab5640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f2220151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 -- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2220152360 msgr2=0x7f2220153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2220152360 0x7f2220153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 -- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 msgr2=0x7f2220151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f2220151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 -- 172.21.15.135:0/3616293066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200c7c60 con 0x7f22200c40e0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221ffff640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 0x7f222014e200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f220c00ec10 tx=0x7f220c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c017070 con 0x7f22200c40e0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220c00cc30 con 0x7f22200c40e0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22200c7f40 con 0x7f22200c40e0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c013470 con 0x7f22200c40e0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.204+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2220154100 con 0x7f22200c40e0 2024-04-26T22:00:32.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.205+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222006f560 con 0x7f22200c40e0 2024-04-26T22:00:32.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.206+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f220c01a070 con 0x7f22200c40e0 2024-04-26T22:00:32.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.206+0000 7f221d7fa640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.206+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f220c08c030 con 0x7f22200c40e0 2024-04-26T22:00:32.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.206+0000 7f2224ab5640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.207+0000 7f2224ab5640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2214034000 tx=0x7f221402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.211+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f220c059b70 con 0x7f22200c40e0 2024-04-26T22:00:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.391+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f22200507a0 con 0x7f22200c40e0 2024-04-26T22:00:32.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.392+0000 7f221d7fa640 1 -- 172.21.15.135:0/3616293066 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f220c05de70 con 0x7f22200c40e0 2024-04-26T22:00:32.392 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.394+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f220406c8f0 msgr2=0x7f220406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.394+0000 7f2226d19640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2214034000 tx=0x7f221402e000 comp rx=0 tx=0).stop 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.394+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 msgr2=0x7f222014e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.394+0000 7f2226d19640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 0x7f222014e200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f220c00ec10 tx=0x7f220c00c520 comp rx=0 tx=0).stop 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 shutdown_connections 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2220152360 0x7f2220153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22200c40e0 0x7f222014e200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 --2- 172.21.15.135:0/3616293066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22200c3060 0x7f2220151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 >> 172.21.15.135:0/3616293066 conn(0x7f222006c310 msgr2=0x7f2220071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 shutdown_connections 2024-04-26T22:00:32.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.395+0000 7f2226d19640 1 -- 172.21.15.135:0/3616293066 wait complete. 2024-04-26T22:00:32.405 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:32.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.512+0000 7fd492837640 1 Processor -- start 2024-04-26T22:00:32.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.512+0000 7fd492837640 1 -- start start 2024-04-26T22:00:32.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.512+0000 7fd492837640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c0c4110 0x7fd48c158640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.512+0000 7fd492837640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd48c0c9d70 con 0x7fd48c152080 2024-04-26T22:00:32.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.512+0000 7fd492837640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd48c0ca050 con 0x7fd48c070d70 2024-04-26T22:00:32.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.512+0000 7fd492837640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c1581b0 con 0x7fd48c0c4110 2024-04-26T22:00:32.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd48c152080 0x7fd48c06d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39236/0 (socket says 172.21.15.135:39236) 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48bfff640 1 -- 172.21.15.135:0/2772035901 learned_addr learned my addr 172.21.15.135:0/2772035901 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c0c4110 0x7fd48c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565801895 0 0) 0x7fd48c0c9d70 con 0x7fd48c152080 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd474003520 con 0x7fd48c152080 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3056004672 0 0) 0x7fd474003520 con 0x7fd48c152080 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c0c4110 msgr2=0x7fd48c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c0c4110 0x7fd48c158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 >> v1:172.21.15.139:6789/0 conn(0x7fd48c070d70 legacy=0x7fd48c15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c0c4690 con 0x7fd48c152080 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fd47c001670 con 0x7fd48c152080 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 >> v1:172.21.15.135:6789/0 conn(0x7fd48c152080 legacy=0x7fd48c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd474003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd474004390 0x7fd474004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd474005100 0x7fd47400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1581b0 con 0x7fd4740039e0 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd47400bfe0 con 0x7fd474004390 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd47400c1e0 con 0x7fd474005100 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c0c4110 0x7fd48c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48bfff640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd474003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd490dd4640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd474005100 0x7fd47400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.513+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd474004390 0x7fd474004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48bfff640 1 -- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd474005100 msgr2=0x7fd47400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48bfff640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd474005100 0x7fd47400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48bfff640 1 -- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd474004390 msgr2=0x7fd474004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48bfff640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd474004390 0x7fd474004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48bfff640 1 -- 172.21.15.135:0/2772035901 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd474003520 con 0x7fd4740039e0 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd474004390 0x7fd474004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48bfff640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd474003dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd47c007d10 tx=0x7fd474005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c6e237b341260c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c007d50 con 0x7fd4740039e0 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c033070 con 0x7fd4740039e0 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.514+0000 7fd48affd640 1 -- 172.21.15.135:0/2772035901 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c005d30 con 0x7fd4740039e0 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd492837640 1 -- 172.21.15.135:0/2772035901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 msgr2=0x7fd474003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd492837640 1 --2- 172.21.15.135:0/2772035901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd474003dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd47c007d10 tx=0x7fd474005830 comp rx=0 tx=0).stop 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd490dd4640 1 -- 172.21.15.135:0/2772035901 reap_dead start 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd492837640 1 -- 172.21.15.135:0/2772035901 shutdown_connections 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd492837640 1 -- 172.21.15.135:0/2772035901 >> 172.21.15.135:0/2772035901 conn(0x7fd48c0c91e0 msgr2=0x7fd48c0c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd492837640 1 -- 172.21.15.135:0/2772035901 shutdown_connections 2024-04-26T22:00:32.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.515+0000 7fd492837640 1 -- 172.21.15.135:0/2772035901 wait complete. 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.517+0000 7fd492837640 1 Processor -- start 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.517+0000 7fd492837640 1 -- start start 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd492837640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd48c0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd492837640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c154440 0x7fd48c149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd492837640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c1499a0 0x7fd48c1eb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd492837640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c06da30 con 0x7fd4740039e0 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd492837640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0700f0 con 0x7fd48c1499a0 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd492837640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c06dbb0 con 0x7fd48c154440 2024-04-26T22:00:32.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd48c0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.518+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd48c0adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40954/0 (socket says 172.21.15.135:40954) 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 -- 172.21.15.135:0/1192392470 learned_addr learned my addr 172.21.15.135:0/1192392470 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd490dd4640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c1499a0 0x7fd48c1eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c154440 0x7fd48c149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 -- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c154440 msgr2=0x7fd48c149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c154440 0x7fd48c149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 -- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c1499a0 msgr2=0x7fd48c1eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c1499a0 0x7fd48c1eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 -- 172.21.15.135:0/1192392470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c073b50 con 0x7fd4740039e0 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c154440 0x7fd48c149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd490dd4640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c1499a0 0x7fd48c1eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd48bfff640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd48c0adf50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00c380 tx=0x7fd47c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c007d50 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c007480 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c073e30 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c02f5b0 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.519+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c0743f0 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.520+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c0696c0 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.521+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd47c02f750 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.521+0000 7fd488ff9640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd47806c8f0 0x7fd47806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.521+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd47806c8f0 0x7fd47806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.521+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fd47c0afee0 con 0x7fd4740039e0 2024-04-26T22:00:32.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.522+0000 7fd48b7fe640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd47806c8f0 0x7fd47806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd480006e80 tx=0x7fd480008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.526+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd47c07e070 con 0x7fd4740039e0 2024-04-26T22:00:32.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.709+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fd48c0507a0 con 0x7fd4740039e0 2024-04-26T22:00:32.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.709+0000 7fd488ff9640 1 -- 172.21.15.135:0/1192392470 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fd47c07d970 con 0x7fd4740039e0 2024-04-26T22:00:32.709 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:32.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd47806c8f0 msgr2=0x7fd47806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd47806c8f0 0x7fd47806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd480006e80 tx=0x7fd480008040 comp rx=0 tx=0).stop 2024-04-26T22:00:32.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 msgr2=0x7fd48c0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd48c0adf50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00c380 tx=0x7fd47c001a40 comp rx=0 tx=0).stop 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 shutdown_connections 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd47806c8f0 0x7fd47806edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd48c1499a0 0x7fd48c1eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd48c154440 0x7fd48c149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 --2- 172.21.15.135:0/1192392470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4740039e0 0x7fd48c0adf50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 >> 172.21.15.135:0/1192392470 conn(0x7fd48c0c91e0 msgr2=0x7fd48c158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 shutdown_connections 2024-04-26T22:00:32.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.711+0000 7fd492837640 1 -- 172.21.15.135:0/1192392470 wait complete. 2024-04-26T22:00:32.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:32.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.829+0000 7f20e8e07640 1 Processor -- start 2024-04-26T22:00:32.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.830+0000 7f20e8e07640 1 -- start start 2024-04-26T22:00:32.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.830+0000 7f20e8e07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f20e4154160 0x7f20e4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.830+0000 7f20e8e07640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f20e415a1a0 0x7f20e40c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.830+0000 7f20e8e07640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20e40c8df0 con 0x7f20e40ca5a0 2024-04-26T22:00:32.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.830+0000 7f20e8e07640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f20e40c9100 con 0x7f20e415a1a0 2024-04-26T22:00:32.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.830+0000 7f20e8e07640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f20e4154540 con 0x7f20e4154160 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f20e415a1a0 0x7f20e40c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f20e415a1a0 0x7f20e40c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40964/0 (socket says 172.21.15.135:40964) 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 -- 172.21.15.135:0/1443664238 learned_addr learned my addr 172.21.15.135:0/1443664238 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 -- 172.21.15.135:0/1443664238 >> v2:172.21.15.139:3300/0 conn(0x7f20e4154160 msgr2=0x7f20e4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 --2- 172.21.15.135:0/1443664238 >> v2:172.21.15.139:3300/0 conn(0x7f20e4154160 0x7f20e4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 -- 172.21.15.135:0/1443664238 >> v1:172.21.15.139:6789/0 conn(0x7f20e40ca5a0 legacy=0x7f20e40c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:32.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 -- 172.21.15.135:0/1443664238 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e415a910 con 0x7f20e415a1a0 2024-04-26T22:00:32.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.831+0000 7f20e2d9d640 1 --2- 172.21.15.135:0/1443664238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e415a1a0 0x7f20e40c9590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20e40c9280 tx=0x7f20d402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6f883156f2756b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e159a640 1 -- 172.21.15.135:0/1443664238 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4036070 con 0x7f20e415a1a0 2024-04-26T22:00:32.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e159a640 1 -- 172.21.15.135:0/1443664238 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d4031600 con 0x7f20e415a1a0 2024-04-26T22:00:32.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e159a640 1 -- 172.21.15.135:0/1443664238 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d40319d0 con 0x7f20e415a1a0 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 -- 172.21.15.135:0/1443664238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e415a1a0 msgr2=0x7f20e40c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1443664238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e415a1a0 0x7f20e40c9590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20e40c9280 tx=0x7f20d402a6d0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 -- 172.21.15.135:0/1443664238 shutdown_connections 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1443664238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e415a1a0 0x7f20e40c9590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1443664238 >> v2:172.21.15.139:3300/0 conn(0x7f20e4154160 0x7f20e4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 -- 172.21.15.135:0/1443664238 >> 172.21.15.135:0/1443664238 conn(0x7f20e40ae010 msgr2=0x7f20e40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 -- 172.21.15.135:0/1443664238 shutdown_connections 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.832+0000 7f20e8e07640 1 -- 172.21.15.135:0/1443664238 wait complete. 2024-04-26T22:00:32.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 Processor -- start 2024-04-26T22:00:32.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 -- start start 2024-04-26T22:00:32.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20e40ca5a0 0x7f20e41f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 0x7f20e4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20e415a1a0 0x7f20e4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20e415a5b0 con 0x7f20e4154160 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20e415aed0 con 0x7f20e415a1a0 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e8e07640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f20e40c7510 con 0x7f20e40ca5a0 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e1d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 0x7f20e4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e1d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 0x7f20e4152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40980/0 (socket says 172.21.15.135:40980) 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.834+0000 7f20e1d9b640 1 -- 172.21.15.135:0/1970124097 learned_addr learned my addr 172.21.15.135:0/1970124097 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e2d9d640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20e415a1a0 0x7f20e4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e259c640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20e40ca5a0 0x7f20e41f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e1d9b640 1 -- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20e40ca5a0 msgr2=0x7f20e41f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e1d9b640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20e40ca5a0 0x7f20e41f0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e1d9b640 1 -- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20e415a1a0 msgr2=0x7f20e4153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e1d9b640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20e415a1a0 0x7f20e4153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e1d9b640 1 -- 172.21.15.135:0/1970124097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e4154a20 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e259c640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20e40ca5a0 0x7f20e41f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e1d9b640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 0x7f20e4152dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c440 tx=0x7f20d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8017070 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d80035f0 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e41f0a60 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80039e0 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.835+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e40c3b40 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.836+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4076650 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.836+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20d801a070 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.837+0000 7f20d2ffd640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c406c8f0 0x7f20c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.837+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f20d808b1a0 con 0x7f20e4154160 2024-04-26T22:00:32.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.837+0000 7f20e259c640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c406c8f0 0x7f20c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:32.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.837+0000 7f20e259c640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c406c8f0 0x7f20c406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20dc006fd0 tx=0x7f20dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:32.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:32.841+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20d8059330 con 0x7f20e4154160 2024-04-26T22:00:33.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.026+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f20e40507a0 con 0x7f20e4154160 2024-04-26T22:00:33.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.026+0000 7f20d2ffd640 1 -- 172.21.15.135:0/1970124097 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f20d80010c0 con 0x7f20e4154160 2024-04-26T22:00:33.026 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:33.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c406c8f0 msgr2=0x7f20c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c406c8f0 0x7f20c406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20dc006fd0 tx=0x7f20dc008040 comp rx=0 tx=0).stop 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 msgr2=0x7f20e4152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 0x7f20e4152dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c440 tx=0x7f20d800c900 comp rx=0 tx=0).stop 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 shutdown_connections 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f20c406c8f0 0x7f20c406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20e415a1a0 0x7f20e4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20e4154160 0x7f20e4152dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 --2- 172.21.15.135:0/1970124097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20e40ca5a0 0x7f20e41f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 >> 172.21.15.135:0/1970124097 conn(0x7f20e40ae010 msgr2=0x7f20e40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 shutdown_connections 2024-04-26T22:00:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.028+0000 7f20e8e07640 1 -- 172.21.15.135:0/1970124097 wait complete. 2024-04-26T22:00:33.040 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:33.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.142+0000 7fb72f9a4640 1 Processor -- start 2024-04-26T22:00:33.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.142+0000 7fb72f9a4640 1 -- start start 2024-04-26T22:00:33.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72f9a4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7280ab7d0 0x7fb7280c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72f9a4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 0x7fb728157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72f9a4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7280c50d0 con 0x7fb72806dee0 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72f9a4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb728156df0 con 0x7fb7280ab7d0 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72f9a4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb7280c53b0 con 0x7fb72815a120 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72df41640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 0x7fb728157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72df41640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 0x7fb728157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:59662/0 (socket says 172.21.15.135:59662) 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72cf3f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7280ab7d0 0x7fb7280c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.143+0000 7fb72df41640 1 -- 172.21.15.135:0/816338944 learned_addr learned my addr 172.21.15.135:0/816338944 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.144+0000 7fb72df41640 1 -- 172.21.15.135:0/816338944 >> v2:172.21.15.139:3300/0 conn(0x7fb7280ab7d0 msgr2=0x7fb7280c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.144+0000 7fb72df41640 1 --2- 172.21.15.135:0/816338944 >> v2:172.21.15.139:3300/0 conn(0x7fb7280ab7d0 0x7fb7280c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.144+0000 7fb72df41640 1 -- 172.21.15.135:0/816338944 >> v1:172.21.15.139:6789/0 conn(0x7fb72806dee0 legacy=0x7fb7280c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.144+0000 7fb71ffff640 1 -- 172.21.15.135:0/816338944 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920972001 0 0) 0x7fb7280c50d0 con 0x7fb72806dee0 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.144+0000 7fb72df41640 1 -- 172.21.15.135:0/816338944 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7280c5690 con 0x7fb72815a120 2024-04-26T22:00:33.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.144+0000 7fb72cf3f640 1 --2- 172.21.15.135:0/816338944 >> v2:172.21.15.139:3300/0 conn(0x7fb7280ab7d0 0x7fb7280c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.145+0000 7fb72df41640 1 --2- 172.21.15.135:0/816338944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 0x7fb728157280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb720009e50 tx=0x7fb72002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe8a3b0bd05600b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.145+0000 7fb71ffff640 1 -- 172.21.15.135:0/816338944 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720036070 con 0x7fb72815a120 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.145+0000 7fb71ffff640 1 -- 172.21.15.135:0/816338944 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720031600 con 0x7fb72815a120 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.145+0000 7fb71ffff640 1 -- 172.21.15.135:0/816338944 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200319d0 con 0x7fb72815a120 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.145+0000 7fb72f9a4640 1 -- 172.21.15.135:0/816338944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 msgr2=0x7fb728157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.145+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/816338944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 0x7fb728157280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb720009e50 tx=0x7fb72002a5f0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.146+0000 7fb72f9a4640 1 -- 172.21.15.135:0/816338944 shutdown_connections 2024-04-26T22:00:33.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.146+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/816338944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72815a120 0x7fb728157280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.146+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/816338944 >> v2:172.21.15.139:3300/0 conn(0x7fb7280ab7d0 0x7fb7280c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.146+0000 7fb72f9a4640 1 -- 172.21.15.135:0/816338944 >> 172.21.15.135:0/816338944 conn(0x7fb7280c6d80 msgr2=0x7fb7280c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.146+0000 7fb72f9a4640 1 -- 172.21.15.135:0/816338944 shutdown_connections 2024-04-26T22:00:33.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.146+0000 7fb72f9a4640 1 -- 172.21.15.135:0/816338944 wait complete. 2024-04-26T22:00:33.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 Processor -- start 2024-04-26T22:00:33.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 -- start start 2024-04-26T22:00:33.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 0x7fb72814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7280ab7d0 0x7fb7281eca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72815a120 0x7fb7281ecf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280c3dc0 con 0x7fb72806dee0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280c5970 con 0x7fb72815a120 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.147+0000 7fb72f9a4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb7280c3f40 con 0x7fb7280ab7d0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 0x7fb72814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 0x7fb72814f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40996/0 (socket says 172.21.15.135:40996) 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 -- 172.21.15.135:0/3151203246 learned_addr learned my addr 172.21.15.135:0/3151203246 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72df41640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72815a120 0x7fb7281ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72cf3f640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7280ab7d0 0x7fb7281eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 -- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7280ab7d0 msgr2=0x7fb7281eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7280ab7d0 0x7fb7281eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 -- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72815a120 msgr2=0x7fb7281ecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72815a120 0x7fb7281ecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 -- 172.21.15.135:0/3151203246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281effa0 con 0x7fb72806dee0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72d740640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 0x7fb72814f120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb71800c4e0 tx=0x7fb71800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718017070 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7180035f0 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180039e0 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.148+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7281f0280 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.149+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7281f0840 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.150+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb728074c80 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.150+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb71801a070 con 0x7fb72806dee0 2024-04-26T22:00:33.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.150+0000 7fb71dffb640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb70c06c8f0 0x7fb70c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.150+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fb71808b040 con 0x7fb72806dee0 2024-04-26T22:00:33.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.151+0000 7fb72cf3f640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb70c06c8f0 0x7fb70c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.151+0000 7fb72cf3f640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb70c06c8f0 0x7fb70c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb724009560 tx=0x7fb724009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.158+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb718058b80 con 0x7fb72806dee0 2024-04-26T22:00:33.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.340+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fb7280507a0 con 0x7fb72806dee0 2024-04-26T22:00:33.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.340+0000 7fb71dffb640 1 -- 172.21.15.135:0/3151203246 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fb71805ce80 con 0x7fb72806dee0 2024-04-26T22:00:33.340 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:33.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb70c06c8f0 msgr2=0x7fb70c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb70c06c8f0 0x7fb70c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb724009560 tx=0x7fb724009190 comp rx=0 tx=0).stop 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 msgr2=0x7fb72814f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 0x7fb72814f120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb71800c4e0 tx=0x7fb71800c9a0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 shutdown_connections 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb70c06c8f0 0x7fb70c06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72815a120 0x7fb7281ecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7280ab7d0 0x7fb7281eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.342+0000 7fb72f9a4640 1 --2- 172.21.15.135:0/3151203246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72806dee0 0x7fb72814f120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.343+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 >> 172.21.15.135:0/3151203246 conn(0x7fb7280c6d80 msgr2=0x7fb72815a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.343+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 shutdown_connections 2024-04-26T22:00:33.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.343+0000 7fb72f9a4640 1 -- 172.21.15.135:0/3151203246 wait complete. 2024-04-26T22:00:33.353 INFO:tasks.mds_thrash:mds thrasher using random seed: 1714168833 2024-04-26T22:00:33.353 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:33.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.451+0000 7f0d60f1b640 1 Processor -- start 2024-04-26T22:00:33.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.451+0000 7f0d60f1b640 1 -- start start 2024-04-26T22:00:33.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d60f1b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c1570d0 0x7f0d5c150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d60f1b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d5c0ab3f0 0x7f0d5c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d60f1b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d5c0c3810 con 0x7f0d5c0c4d80 2024-04-26T22:00:33.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d60f1b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5c0c4330 con 0x7f0d5c0ab3f0 2024-04-26T22:00:33.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d60f1b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c0c3ab0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d5c0ab3f0 0x7f0d5c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d5c0ab3f0 0x7f0d5c150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41008/0 (socket says 172.21.15.135:41008) 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d5ad9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f0d5c0c4d80 0x7f0d5c1574b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39244/0 (socket says 172.21.15.135:39244) 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 -- 172.21.15.135:0/1457146136 learned_addr learned my addr 172.21.15.135:0/1457146136 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d5a59c640 1 --2- 172.21.15.135:0/1457146136 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c1570d0 0x7f0d5c150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 -- 172.21.15.135:0/1457146136 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c1570d0 msgr2=0x7f0d5c150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 --2- 172.21.15.135:0/1457146136 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c1570d0 0x7f0d5c150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 -- 172.21.15.135:0/1457146136 >> v1:172.21.15.135:6789/0 conn(0x7f0d5c0c4d80 legacy=0x7f0d5c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d5959a640 1 -- 172.21.15.135:0/1457146136 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310726888 0 0) 0x7f0d5c0c3810 con 0x7f0d5c0c4d80 2024-04-26T22:00:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.452+0000 7f0d59d9b640 1 -- 172.21.15.135:0/1457146136 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0c4580 con 0x7f0d5c0ab3f0 2024-04-26T22:00:33.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.453+0000 7f0d59d9b640 1 --2- 172.21.15.135:0/1457146136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c0ab3f0 0x7f0d5c150f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d5002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d713bc0d48a11c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.453+0000 7f0d5959a640 1 -- 172.21.15.135:0/1457146136 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50037070 con 0x7f0d5c0ab3f0 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.453+0000 7f0d5959a640 1 -- 172.21.15.135:0/1457146136 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d50032440 con 0x7f0d5c0ab3f0 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.453+0000 7f0d5959a640 1 -- 172.21.15.135:0/1457146136 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50032810 con 0x7f0d5c0ab3f0 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 -- 172.21.15.135:0/1457146136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c0ab3f0 msgr2=0x7f0d5c150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/1457146136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c0ab3f0 0x7f0d5c150f90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d5002a130 comp rx=0 tx=0).stop 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 -- 172.21.15.135:0/1457146136 shutdown_connections 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/1457146136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c0ab3f0 0x7f0d5c150f90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/1457146136 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c1570d0 0x7f0d5c150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 -- 172.21.15.135:0/1457146136 >> 172.21.15.135:0/1457146136 conn(0x7f0d5c06c310 msgr2=0x7f0d5c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 -- 172.21.15.135:0/1457146136 shutdown_connections 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.454+0000 7f0d60f1b640 1 -- 172.21.15.135:0/1457146136 wait complete. 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.455+0000 7f0d60f1b640 1 Processor -- start 2024-04-26T22:00:33.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.455+0000 7f0d60f1b640 1 -- start start 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c0ab3f0 0x7f0d5c14f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d5c0c4d80 0x7f0d5c1ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 0x7f0d5c1edb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0abbe0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c0aba60 con 0x7f0d5c0c4d80 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5c157bc0 con 0x7f0d5c0ab3f0 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 0x7f0d5c1edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 0x7f0d5c1edb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41022/0 (socket says 172.21.15.135:41022) 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 -- 172.21.15.135:0/2289898458 learned_addr learned my addr 172.21.15.135:0/2289898458 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5a59c640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c0ab3f0 0x7f0d5c14f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 -- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c0ab3f0 msgr2=0x7f0d5c14f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c0ab3f0 0x7f0d5c14f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 -- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d5c0c4d80 msgr2=0x7f0d5c1ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d59d9b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d5c0c4d80 0x7f0d5c1ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d5c0c4d80 0x7f0d5c1ed5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 -- 172.21.15.135:0/2289898458 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c0c70b0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d5ad9d640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 0x7f0d5c1edb30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d54001560 tx=0x7f0d5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54017070 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5400a640 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c1560b0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400aa10 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.456+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c14fdf0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.457+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c0507a0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.458+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d5401a070 con 0x7f0d5c1570d0 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.458+0000 7f0d4affd640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0d3c06c970 0x7f0d3c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.458+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0d5408b3e0 con 0x7f0d5c1570d0 2024-04-26T22:00:33.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.458+0000 7f0d5a59c640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0d3c06c970 0x7f0d3c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.459+0000 7f0d5a59c640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0d3c06c970 0x7f0d3c06ee30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006fd0 tx=0x7f0d4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.462+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0d54059650 con 0x7f0d5c1570d0 2024-04-26T22:00:33.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.638+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d5c0c4110 con 0x7f0d5c1570d0 2024-04-26T22:00:33.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.639+0000 7f0d4affd640 1 -- 172.21.15.135:0/2289898458 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7f0d54058f50 con 0x7f0d5c1570d0 2024-04-26T22:00:33.639 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:33.639 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0d3c06c970 msgr2=0x7f0d3c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0d3c06c970 0x7f0d3c06ee30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006fd0 tx=0x7f0d4c008040 comp rx=0 tx=0).stop 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 msgr2=0x7f0d5c1edb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 0x7f0d5c1edb30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d54001560 tx=0x7f0d5400cc70 comp rx=0 tx=0).stop 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 shutdown_connections 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0d3c06c970 0x7f0d3c06ee30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c1570d0 0x7f0d5c1edb30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d5c0c4d80 0x7f0d5c1ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 --2- 172.21.15.135:0/2289898458 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d5c0ab3f0 0x7f0d5c14f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 >> 172.21.15.135:0/2289898458 conn(0x7f0d5c06c310 msgr2=0x7f0d5c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 shutdown_connections 2024-04-26T22:00:33.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.641+0000 7f0d60f1b640 1 -- 172.21.15.135:0/2289898458 wait complete. 2024-04-26T22:00:33.652 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:33.653 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:33.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.752+0000 7f562f0a9640 1 Processor -- start 2024-04-26T22:00:33.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.752+0000 7f562f0a9640 1 -- start start 2024-04-26T22:00:33.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f562f0a9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56280c4160 0x7f562806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f562f0a9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56280c4b50 con 0x7f56281510c0 2024-04-26T22:00:33.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f562f0a9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56281514a0 con 0x7f562806d860 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f562f0a9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f56280c50d0 con 0x7f56280c4160 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f5627fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56280c4160 0x7f562806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f562ce45640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f56281510c0 0x7f5628072f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39250/0 (socket says 172.21.15.135:39250) 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.753+0000 7f562ce45640 1 -- 172.21.15.135:0/3299016787 learned_addr learned my addr 172.21.15.135:0/3299016787 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f5627fff640 1 -- 172.21.15.135:0/3299016787 >> v1:172.21.15.139:6789/0 conn(0x7f562806d860 legacy=0x7f562815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f5627fff640 1 -- 172.21.15.135:0/3299016787 >> v1:172.21.15.135:6789/0 conn(0x7f56281510c0 legacy=0x7f5628072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f5627fff640 1 -- 172.21.15.135:0/3299016787 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280736a0 con 0x7f56280c4160 2024-04-26T22:00:33.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f56277fe640 1 -- 172.21.15.135:0/3299016787 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 484907050 0 0) 0x7f56280c4b50 con 0x7f56281510c0 2024-04-26T22:00:33.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f5627fff640 1 --2- 172.21.15.135:0/3299016787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56280c4160 0x7f562806d320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f561c0097a0 tx=0x7f561c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d181bd59f3693168 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f56277fe640 1 -- 172.21.15.135:0/3299016787 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c037070 con 0x7f56280c4160 2024-04-26T22:00:33.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f56277fe640 1 -- 172.21.15.135:0/3299016787 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c032440 con 0x7f56280c4160 2024-04-26T22:00:33.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.754+0000 7f56277fe640 1 -- 172.21.15.135:0/3299016787 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c02ae60 con 0x7f56280c4160 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 -- 172.21.15.135:0/3299016787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56280c4160 msgr2=0x7f562806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 --2- 172.21.15.135:0/3299016787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56280c4160 0x7f562806d320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f561c0097a0 tx=0x7f561c02a110 comp rx=0 tx=0).stop 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 -- 172.21.15.135:0/3299016787 shutdown_connections 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 --2- 172.21.15.135:0/3299016787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56280c4160 0x7f562806d320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 -- 172.21.15.135:0/3299016787 >> 172.21.15.135:0/3299016787 conn(0x7f56280c5ce0 msgr2=0x7f56280c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 -- 172.21.15.135:0/3299016787 shutdown_connections 2024-04-26T22:00:33.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.755+0000 7f562f0a9640 1 -- 172.21.15.135:0/3299016787 wait complete. 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.756+0000 7f562f0a9640 1 Processor -- start 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.756+0000 7f562f0a9640 1 -- start start 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562f0a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f562806d860 0x7f56280c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562f0a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56280c4160 0x7f56280756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562f0a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 0x7f562814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562f0a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56280c5400 con 0x7f56281510c0 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562f0a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56280c4df0 con 0x7f56280c4160 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562f0a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f56280c5920 con 0x7f562806d860 2024-04-26T22:00:33.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 0x7f562814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 0x7f562814b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41046/0 (socket says 172.21.15.135:41046) 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 -- 172.21.15.135:0/2674770379 learned_addr learned my addr 172.21.15.135:0/2674770379 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562ce45640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f562806d860 0x7f56280c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f5627fff640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56280c4160 0x7f56280756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 -- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f562806d860 msgr2=0x7f56280c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f562806d860 0x7f56280c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 -- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56280c4160 msgr2=0x7f56280756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56280c4160 0x7f56280756f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.757+0000 7f562d646640 1 -- 172.21.15.135:0/2674770379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56281ea8e0 con 0x7f56281510c0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.758+0000 7f562d646640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 0x7f562814b460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f562000c7a0 tx=0x7f562000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.758+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620003b70 con 0x7f56281510c0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.758+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5620003d30 con 0x7f56281510c0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.758+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56281eab80 con 0x7f56281510c0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.758+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562000aa90 con 0x7f56281510c0 2024-04-26T22:00:33.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.758+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56281eb140 con 0x7f56281510c0 2024-04-26T22:00:33.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.759+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5628068dc0 con 0x7f56281510c0 2024-04-26T22:00:33.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.759+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f562000ac30 con 0x7f56281510c0 2024-04-26T22:00:33.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.760+0000 7f56257fa640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f561406c8f0 0x7f561406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:33.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.760+0000 7f562ce45640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f561406c8f0 0x7f561406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:33.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.760+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f562008bc00 con 0x7f56281510c0 2024-04-26T22:00:33.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.760+0000 7f562ce45640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f561406c8f0 0x7f561406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5618007c30 tx=0x7f5618007b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:33.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.764+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5620092080 con 0x7f56281510c0 2024-04-26T22:00:33.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.942+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56280507a0 con 0x7f56281510c0 2024-04-26T22:00:33.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.942+0000 7f56257fa640 1 -- 172.21.15.135:0/2674770379 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f5620059740 con 0x7f56281510c0 2024-04-26T22:00:33.942 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:33.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f561406c8f0 msgr2=0x7f561406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f561406c8f0 0x7f561406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5618007c30 tx=0x7f5618007b80 comp rx=0 tx=0).stop 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 msgr2=0x7f562814b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 0x7f562814b460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f562000c7a0 tx=0x7f562000cc60 comp rx=0 tx=0).stop 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 shutdown_connections 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f561406c8f0 0x7f561406edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56281510c0 0x7f562814b460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56280c4160 0x7f56280756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 --2- 172.21.15.135:0/2674770379 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f562806d860 0x7f56280c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 >> 172.21.15.135:0/2674770379 conn(0x7f56280c5ce0 msgr2=0x7f5628153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 shutdown_connections 2024-04-26T22:00:33.945 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:33.944+0000 7f562f0a9640 1 -- 172.21.15.135:0/2674770379 wait complete. 2024-04-26T22:00:33.955 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:34.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.054+0000 7ff83582a640 1 Processor -- start 2024-04-26T22:00:34.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff83582a640 1 -- start start 2024-04-26T22:00:34.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff83582a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8300c9830 0x7ff830153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff83582a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff830156de0 con 0x7ff8300c7000 2024-04-26T22:00:34.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff83582a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff830157b60 con 0x7ff830154220 2024-04-26T22:00:34.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff83582a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff8300cad40 con 0x7ff8300c9830 2024-04-26T22:00:34.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff82effd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff8300c7000 0x7ff8300ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39266/0 (socket says 172.21.15.135:39266) 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.055+0000 7ff82effd640 1 -- 172.21.15.135:0/344053736 learned_addr learned my addr 172.21.15.135:0/344053736 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 --2- 172.21.15.135:0/344053736 >> v2:172.21.15.139:3300/0 conn(0x7ff8300c9830 0x7ff830153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762182414 0 0) 0x7ff830156de0 con 0x7ff8300c7000 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff818003520 con 0x7ff8300c7000 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 41538064 0 0) 0x7ff818003520 con 0x7ff8300c7000 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 >> v2:172.21.15.139:3300/0 conn(0x7ff8300c9830 msgr2=0x7ff830153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 --2- 172.21.15.135:0/344053736 >> v2:172.21.15.139:3300/0 conn(0x7ff8300c9830 0x7ff830153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 >> v1:172.21.15.139:6789/0 conn(0x7ff830154220 legacy=0x7ff830072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff830157080 con 0x7ff8300c7000 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7ff8200019f0 con 0x7ff8300c7000 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 >> v1:172.21.15.135:6789/0 conn(0x7ff8300c7000 legacy=0x7ff8300ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8180039b0 0x7ff818003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff818004800 0x7ff818004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8180051c0 0x7ff81800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300cad40 con 0x7ff818004800 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff81800c060 con 0x7ff8180039b0 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff81800c260 con 0x7ff8180051c0 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff818004800 0x7ff818004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82effd640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8180039b0 0x7ff818003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82f7fe640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8180051c0 0x7ff81800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 -- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8180051c0 msgr2=0x7ff81800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8180051c0 0x7ff81800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 -- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8180039b0 msgr2=0x7ff818003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8180039b0 0x7ff818003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82e7fc640 1 -- 172.21.15.135:0/344053736 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff818003520 con 0x7ff818004800 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.056+0000 7ff82f7fe640 1 -- 172.21.15.135:0/344053736 reap_dead start 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.057+0000 7ff82e7fc640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff818004800 0x7ff818004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff824009a90 tx=0x7ff818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ddb874db7d9824 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.057+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824036070 con 0x7ff818004800 2024-04-26T22:00:34.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.057+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff824031400 con 0x7ff818004800 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.057+0000 7ff82dffb640 1 -- 172.21.15.135:0/344053736 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824009690 con 0x7ff818004800 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 -- 172.21.15.135:0/344053736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff818004800 msgr2=0x7ff818004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff818004800 0x7ff818004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff824009a90 tx=0x7ff818005830 comp rx=0 tx=0).stop 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 -- 172.21.15.135:0/344053736 shutdown_connections 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 --2- 172.21.15.135:0/344053736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff818004800 0x7ff818004c80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 -- 172.21.15.135:0/344053736 >> 172.21.15.135:0/344053736 conn(0x7ff8300adf20 msgr2=0x7ff8300ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 -- 172.21.15.135:0/344053736 shutdown_connections 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.058+0000 7ff83582a640 1 -- 172.21.15.135:0/344053736 wait complete. 2024-04-26T22:00:34.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.059+0000 7ff83582a640 1 Processor -- start 2024-04-26T22:00:34.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.059+0000 7ff83582a640 1 -- start start 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff818004800 0x7ff8301efb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff83014ae80 0x7ff83014a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 0x7ff8300c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300c73e0 con 0x7ff83014a6f0 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff830157640 con 0x7ff83014ae80 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff8300c7560 con 0x7ff818004800 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 0x7ff8300c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 0x7ff8300c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41074/0 (socket says 172.21.15.135:41074) 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 -- 172.21.15.135:0/1141332292 learned_addr learned my addr 172.21.15.135:0/1141332292 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82e7fc640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff83014ae80 0x7ff83014a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82effd640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff818004800 0x7ff8301efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 -- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff818004800 msgr2=0x7ff8301efb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff818004800 0x7ff8301efb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 -- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff83014ae80 msgr2=0x7ff83014a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff83014ae80 0x7ff83014a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 -- 172.21.15.135:0/1141332292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83014aba0 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff82f7fe640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 0x7ff8300c42a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff830154600 tx=0x7ff82800e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828019070 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8280050d0 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8300c61c0 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8280035f0 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.060+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83015a900 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.061+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8300766c0 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.062+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff82801c070 con 0x7ff83014a6f0 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.062+0000 7ff817fff640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.062+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7ff82808af80 con 0x7ff83014a6f0 2024-04-26T22:00:34.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.062+0000 7ff82effd640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.063+0000 7ff82effd640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff81c06c8f0 0x7ff81c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff8200013d0 tx=0x7ff820006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.067+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff83014aba0 con 0x7ff83014a6f0 2024-04-26T22:00:34.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.242+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7ff8300507a0 con 0x7ff83014a6f0 2024-04-26T22:00:34.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.242+0000 7ff817fff640 1 -- 172.21.15.135:0/1141332292 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7ff828058ac0 con 0x7ff83014a6f0 2024-04-26T22:00:34.242 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:34.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff81c06c8f0 msgr2=0x7ff81c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff81c06c8f0 0x7ff81c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff8200013d0 tx=0x7ff820006040 comp rx=0 tx=0).stop 2024-04-26T22:00:34.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 msgr2=0x7ff8300c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 0x7ff8300c42a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff830154600 tx=0x7ff82800e550 comp rx=0 tx=0).stop 2024-04-26T22:00:34.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 shutdown_connections 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff81c06c8f0 0x7ff81c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83014a6f0 0x7ff8300c42a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff83014ae80 0x7ff83014a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 --2- 172.21.15.135:0/1141332292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff818004800 0x7ff8301efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 >> 172.21.15.135:0/1141332292 conn(0x7ff8300adf20 msgr2=0x7ff8300ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 shutdown_connections 2024-04-26T22:00:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.244+0000 7ff83582a640 1 -- 172.21.15.135:0/1141332292 wait complete. 2024-04-26T22:00:34.255 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:34.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.354+0000 7f8b279bd640 1 Processor -- start 2024-04-26T22:00:34.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b279bd640 1 -- start start 2024-04-26T22:00:34.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b279bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c40e0 0x7f8b200720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b279bd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8b200c46e0 0x7f8b200c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b279bd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b20154ba0 con 0x7f8b200c3060 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b279bd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8b20154e80 con 0x7f8b200c46e0 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b279bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8b20155160 con 0x7f8b200c40e0 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b25f5a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8b200c46e0 0x7f8b200c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b25759640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8b200c3060 0x7f8b20151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39272/0 (socket says 172.21.15.135:39272) 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.355+0000 7f8b25759640 1 -- 172.21.15.135:0/1910019580 learned_addr learned my addr 172.21.15.135:0/1910019580 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b24f58640 1 --2- 172.21.15.135:0/1910019580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c40e0 0x7f8b200720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b25f5a640 1 -- 172.21.15.135:0/1910019580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c40e0 msgr2=0x7f8b200720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b25f5a640 1 --2- 172.21.15.135:0/1910019580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c40e0 0x7f8b200720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b25f5a640 1 -- 172.21.15.135:0/1910019580 >> v1:172.21.15.135:6789/0 conn(0x7f8b200c3060 legacy=0x7f8b20151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b25f5a640 1 -- 172.21.15.135:0/1910019580 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b20155440 con 0x7f8b200c46e0 2024-04-26T22:00:34.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b17fff640 1 -- 172.21.15.135:0/1910019580 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119851164 0 0) 0x7f8b20154ba0 con 0x7f8b200c3060 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b25f5a640 1 --2- 172.21.15.135:0/1910019580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c46e0 0x7f8b200c4b70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8b180097a0 tx=0x7f8b1802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92534e08ed058b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.356+0000 7f8b17fff640 1 -- 172.21.15.135:0/1910019580 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18037070 con 0x7f8b200c46e0 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b17fff640 1 -- 172.21.15.135:0/1910019580 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1802ae50 con 0x7f8b200c46e0 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b17fff640 1 -- 172.21.15.135:0/1910019580 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18032660 con 0x7f8b200c46e0 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 -- 172.21.15.135:0/1910019580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c46e0 msgr2=0x7f8b200c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1910019580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c46e0 0x7f8b200c4b70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8b180097a0 tx=0x7f8b1802a100 comp rx=0 tx=0).stop 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 -- 172.21.15.135:0/1910019580 shutdown_connections 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1910019580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c40e0 0x7f8b200720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1910019580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c46e0 0x7f8b200c4b70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 -- 172.21.15.135:0/1910019580 >> 172.21.15.135:0/1910019580 conn(0x7f8b2006c310 msgr2=0x7f8b2006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 -- 172.21.15.135:0/1910019580 shutdown_connections 2024-04-26T22:00:34.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.357+0000 7f8b279bd640 1 -- 172.21.15.135:0/1910019580 wait complete. 2024-04-26T22:00:34.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.358+0000 7f8b279bd640 1 Processor -- start 2024-04-26T22:00:34.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.358+0000 7f8b279bd640 1 -- start start 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b279bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c3060 0x7f8b20148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b279bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 0x7f8b201ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b279bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b200c46e0 0x7f8b200c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b279bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200c3440 con 0x7f8b200c40e0 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b279bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200c3aa0 con 0x7f8b200c46e0 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b279bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8b200c35c0 con 0x7f8b200c3060 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 0x7f8b201ed3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 0x7f8b201ed3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41080/0 (socket says 172.21.15.135:41080) 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 -- 172.21.15.135:0/1885288945 learned_addr learned my addr 172.21.15.135:0/1885288945 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:34.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b25759640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c3060 0x7f8b20148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b25f5a640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b200c46e0 0x7f8b200c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 -- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c3060 msgr2=0x7f8b20148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c3060 0x7f8b20148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 -- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b200c46e0 msgr2=0x7f8b200c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b200c46e0 0x7f8b200c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 -- 172.21.15.135:0/1885288945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200c7f50 con 0x7f8b200c40e0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b25759640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c3060 0x7f8b20148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b25f5a640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b200c46e0 0x7f8b200c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.359+0000 7f8b24f58640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 0x7f8b201ed3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d800 tx=0x7f8b1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.360+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000b6c0 con 0x7f8b200c40e0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.360+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b10019070 con 0x7f8b200c40e0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.360+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b20156f70 con 0x7f8b200c40e0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.360+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000ba80 con 0x7f8b200c40e0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.360+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b20157210 con 0x7f8b200c40e0 2024-04-26T22:00:34.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.361+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b200507a0 con 0x7f8b200c40e0 2024-04-26T22:00:34.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.361+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b1000bc20 con 0x7f8b200c40e0 2024-04-26T22:00:34.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.362+0000 7f8b15ffb640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8b0406c8f0 0x7f8b0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.362+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f8b1008c2a0 con 0x7f8b200c40e0 2024-04-26T22:00:34.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.362+0000 7f8b25759640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8b0406c8f0 0x7f8b0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.362+0000 7f8b25759640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8b0406c8f0 0x7f8b0406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0012e0 tx=0x7f8b1c005b90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.366+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b10011340 con 0x7f8b200c40e0 2024-04-26T22:00:34.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.539+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f8b201ed900 con 0x7f8b200c40e0 2024-04-26T22:00:34.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.539+0000 7f8b15ffb640 1 -- 172.21.15.135:0/1885288945 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f8b1005a430 con 0x7f8b200c40e0 2024-04-26T22:00:34.539 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8b0406c8f0 msgr2=0x7f8b0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8b0406c8f0 0x7f8b0406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0012e0 tx=0x7f8b1c005b90 comp rx=0 tx=0).stop 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 msgr2=0x7f8b201ed3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 0x7f8b201ed3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d800 tx=0x7f8b1000dcc0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 shutdown_connections 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8b0406c8f0 0x7f8b0406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b200c46e0 0x7f8b200c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b200c40e0 0x7f8b201ed3c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 --2- 172.21.15.135:0/1885288945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b200c3060 0x7f8b20148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 >> 172.21.15.135:0/1885288945 conn(0x7f8b2006c310 msgr2=0x7f8b2006f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.541+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 shutdown_connections 2024-04-26T22:00:34.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.542+0000 7f8b279bd640 1 -- 172.21.15.135:0/1885288945 wait complete. 2024-04-26T22:00:34.552 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:34.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.652+0000 7fc04319e640 1 Processor -- start 2024-04-26T22:00:34.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.652+0000 7fc04319e640 1 -- start start 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc04319e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc03c0c3060 0x7fc03c0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc04319e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc03c0c40e0 0x7fc03c151e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc04319e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03c154ba0 con 0x7fc03c152360 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc04319e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc03c154e80 con 0x7fc03c0c3060 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc04319e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc03c155160 con 0x7fc03c0c40e0 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc03c0c3060 0x7fc03c0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc03c0c3060 0x7fc03c0720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41096/0 (socket says 172.21.15.135:41096) 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 -- 172.21.15.135:0/3203812835 learned_addr learned my addr 172.21.15.135:0/3203812835 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:34.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 -- 172.21.15.135:0/3203812835 >> v2:172.21.15.139:3300/0 conn(0x7fc03c0c40e0 msgr2=0x7fc03c151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 --2- 172.21.15.135:0/3203812835 >> v2:172.21.15.139:3300/0 conn(0x7fc03c0c40e0 0x7fc03c151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 -- 172.21.15.135:0/3203812835 >> v1:172.21.15.139:6789/0 conn(0x7fc03c152360 legacy=0x7fc03c0c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.653+0000 7fc040f3a640 1 -- 172.21.15.135:0/3203812835 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c155440 con 0x7fc03c0c3060 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc040f3a640 1 --2- 172.21.15.135:0/3203812835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c0c3060 0x7fc03c0720a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06d250 tx=0x7fc03002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b836cde34ea928 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc03b7fe640 1 -- 172.21.15.135:0/3203812835 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030037070 con 0x7fc03c0c3060 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc03b7fe640 1 -- 172.21.15.135:0/3203812835 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030032440 con 0x7fc03c0c3060 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc03b7fe640 1 -- 172.21.15.135:0/3203812835 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030032810 con 0x7fc03c0c3060 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc04319e640 1 -- 172.21.15.135:0/3203812835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c0c3060 msgr2=0x7fc03c0720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc04319e640 1 --2- 172.21.15.135:0/3203812835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c0c3060 0x7fc03c0720a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06d250 tx=0x7fc03002a1a0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc04319e640 1 -- 172.21.15.135:0/3203812835 shutdown_connections 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc04319e640 1 --2- 172.21.15.135:0/3203812835 >> v2:172.21.15.139:3300/0 conn(0x7fc03c0c40e0 0x7fc03c151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc04319e640 1 --2- 172.21.15.135:0/3203812835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c0c3060 0x7fc03c0720a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.654+0000 7fc04319e640 1 -- 172.21.15.135:0/3203812835 >> 172.21.15.135:0/3203812835 conn(0x7fc03c06c310 msgr2=0x7fc03c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.655+0000 7fc04319e640 1 -- 172.21.15.135:0/3203812835 shutdown_connections 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.655+0000 7fc04319e640 1 -- 172.21.15.135:0/3203812835 wait complete. 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 Processor -- start 2024-04-26T22:00:34.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 -- start start 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc03c0c3060 0x7fc03c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03c0c40e0 0x7fc03c1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 0x7fc03c141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c0c3440 con 0x7fc03c152360 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c0c3aa0 con 0x7fc03c0c40e0 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.656+0000 7fc04319e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c0c35c0 con 0x7fc03c0c3060 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 0x7fc03c141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 0x7fc03c141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41114/0 (socket says 172.21.15.135:41114) 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 -- 172.21.15.135:0/2371865451 learned_addr learned my addr 172.21.15.135:0/2371865451 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 -- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc03c0c3060 msgr2=0x7fc03c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc040f3a640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc03c0c3060 0x7fc03c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc03bfff640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03c0c40e0 0x7fc03c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc03c0c3060 0x7fc03c153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 -- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03c0c40e0 msgr2=0x7fc03c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03c0c40e0 0x7fc03c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 -- 172.21.15.135:0/2371865451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c1507c0 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc03bfff640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03c0c40e0 0x7fc03c1411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04173b640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 0x7fc03c141730 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c7b0 tx=0x7fc03400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034017070 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034003da0 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c151180 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340052b0 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.657+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c151740 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.658+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c0507a0 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.659+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc03401a070 con 0x7fc03c152360 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.659+0000 7fc0397fa640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc02006c8f0 0x7fc02006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.659+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc03408b510 con 0x7fc03c152360 2024-04-26T22:00:34.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.659+0000 7fc040f3a640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc02006c8f0 0x7fc02006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.659+0000 7fc040f3a640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc02006c8f0 0x7fc02006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc030034000 tx=0x7fc03002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.663+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc034059610 con 0x7fc03c152360 2024-04-26T22:00:34.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.837+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7fc03c154100 con 0x7fc03c152360 2024-04-26T22:00:34.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.837+0000 7fc0397fa640 1 -- 172.21.15.135:0/2371865451 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7fc034058f10 con 0x7fc03c152360 2024-04-26T22:00:34.837 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:34.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc02006c8f0 msgr2=0x7fc02006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc02006c8f0 0x7fc02006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc030034000 tx=0x7fc03002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 msgr2=0x7fc03c141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 0x7fc03c141730 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c7b0 tx=0x7fc03400cc70 comp rx=0 tx=0).stop 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 shutdown_connections 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc02006c8f0 0x7fc02006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03c152360 0x7fc03c141730 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03c0c40e0 0x7fc03c1411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 --2- 172.21.15.135:0/2371865451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc03c0c3060 0x7fc03c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.839+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 >> 172.21.15.135:0/2371865451 conn(0x7fc03c06c310 msgr2=0x7fc03c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.840+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 shutdown_connections 2024-04-26T22:00:34.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.840+0000 7fc04319e640 1 -- 172.21.15.135:0/2371865451 wait complete. 2024-04-26T22:00:34.850 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:34.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.951+0000 7fcfe74fc640 1 Processor -- start 2024-04-26T22:00:34.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 -- start start 2024-04-26T22:00:34.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfe01510c0 0x7fcfe0072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcfe00734d0 0x7fcfe006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe00c4160 0x7fcfe0157070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe00c50d0 con 0x7fcfe01510c0 2024-04-26T22:00:34.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe00c4b50 con 0x7fcfe00734d0 2024-04-26T22:00:34.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe74fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe01514a0 con 0x7fcfe00c4160 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe5298640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfe01510c0 0x7fcfe0072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe5298640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfe01510c0 0x7fcfe0072f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41116/0 (socket says 172.21.15.135:41116) 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.952+0000 7fcfe5298640 1 -- 172.21.15.135:0/1579956578 learned_addr learned my addr 172.21.15.135:0/1579956578 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe4a97640 1 --2- 172.21.15.135:0/1579956578 >> v2:172.21.15.139:3300/0 conn(0x7fcfe00734d0 0x7fcfe006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5a99640 1 --2- 172.21.15.135:0/1579956578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe00c4160 0x7fcfe0157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5298640 1 -- 172.21.15.135:0/1579956578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe00c4160 msgr2=0x7fcfe0157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5298640 1 --2- 172.21.15.135:0/1579956578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe00c4160 0x7fcfe0157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5298640 1 -- 172.21.15.135:0/1579956578 >> v2:172.21.15.139:3300/0 conn(0x7fcfe00734d0 msgr2=0x7fcfe006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5298640 1 --2- 172.21.15.135:0/1579956578 >> v2:172.21.15.139:3300/0 conn(0x7fcfe00734d0 0x7fcfe006d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5298640 1 -- 172.21.15.135:0/1579956578 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe0154920 con 0x7fcfe01510c0 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfe5298640 1 --2- 172.21.15.135:0/1579956578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe01510c0 0x7fcfe0072f90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0070b50 tx=0x7fcfdc02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2733d242d995c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.953+0000 7fcfd7fff640 1 -- 172.21.15.135:0/1579956578 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc038070 con 0x7fcfe01510c0 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfd7fff640 1 -- 172.21.15.135:0/1579956578 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfdc031710 con 0x7fcfe01510c0 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfd7fff640 1 -- 172.21.15.135:0/1579956578 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc031ae0 con 0x7fcfe01510c0 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 -- 172.21.15.135:0/1579956578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe01510c0 msgr2=0x7fcfe0072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/1579956578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe01510c0 0x7fcfe0072f90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0070b50 tx=0x7fcfdc02c770 comp rx=0 tx=0).stop 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 -- 172.21.15.135:0/1579956578 shutdown_connections 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/1579956578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe00c4160 0x7fcfe0157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/1579956578 >> v2:172.21.15.139:3300/0 conn(0x7fcfe00734d0 0x7fcfe006d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/1579956578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe01510c0 0x7fcfe0072f90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 -- 172.21.15.135:0/1579956578 >> 172.21.15.135:0/1579956578 conn(0x7fcfe00c5ce0 msgr2=0x7fcfe00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 -- 172.21.15.135:0/1579956578 shutdown_connections 2024-04-26T22:00:34.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.954+0000 7fcfe74fc640 1 -- 172.21.15.135:0/1579956578 wait complete. 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.955+0000 7fcfe74fc640 1 Processor -- start 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.955+0000 7fcfe74fc640 1 -- start start 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe74fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfe00c4160 0x7fcfe0152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe74fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe01510c0 0x7fcfe01e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe74fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 0x7fcfe01e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe74fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00c5920 con 0x7fcfe014af80 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe74fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00c5400 con 0x7fcfe00c4160 2024-04-26T22:00:34.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe74fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe006f500 con 0x7fcfe01510c0 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 0x7fcfe01e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 0x7fcfe01e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41132/0 (socket says 172.21.15.135:41132) 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 -- 172.21.15.135:0/3074777507 learned_addr learned my addr 172.21.15.135:0/3074777507 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5298640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfe00c4160 0x7fcfe0152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe4a97640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe01510c0 0x7fcfe01e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 -- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe01510c0 msgr2=0x7fcfe01e88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe01510c0 0x7fcfe01e88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 -- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfe00c4160 msgr2=0x7fcfe0152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfe00c4160 0x7fcfe0152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 -- 172.21.15.135:0/3074777507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe01ec530 con 0x7fcfe014af80 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.956+0000 7fcfe5a99640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 0x7fcfe01e8e30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8007e30 tx=0x7fcfd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.957+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8014620 con 0x7fcfe014af80 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.957+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd8019070 con 0x7fcfe014af80 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.957+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe01ec810 con 0x7fcfe014af80 2024-04-26T22:00:34.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.957+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8014a00 con 0x7fcfe014af80 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.957+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe01532b0 con 0x7fcfe014af80 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.958+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe01e85c0 con 0x7fcfe014af80 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.958+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfd8014ba0 con 0x7fcfe014af80 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.958+0000 7fcfd5ffb640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcfc406c8f0 0x7fcfc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.959+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fcfd808c6f0 con 0x7fcfe014af80 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.959+0000 7fcfe5298640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcfc406c8f0 0x7fcfc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:34.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.959+0000 7fcfe5298640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcfc406c8f0 0x7fcfc406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc035000 tx=0x7fcfdc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:34.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:34.963+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcfd805a7f0 con 0x7fcfe014af80 2024-04-26T22:00:35.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.140+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fcfe0070930 con 0x7fcfe014af80 2024-04-26T22:00:35.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.140+0000 7fcfd5ffb640 1 -- 172.21.15.135:0/3074777507 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7fcfd805a0f0 con 0x7fcfe014af80 2024-04-26T22:00:35.140 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:35.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcfc406c8f0 msgr2=0x7fcfc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcfc406c8f0 0x7fcfc406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc035000 tx=0x7fcfdc030000 comp rx=0 tx=0).stop 2024-04-26T22:00:35.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 msgr2=0x7fcfe01e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 0x7fcfe01e8e30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8007e30 tx=0x7fcfd800e4b0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 shutdown_connections 2024-04-26T22:00:35.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcfc406c8f0 0x7fcfc406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfe014af80 0x7fcfe01e8e30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfe01510c0 0x7fcfe01e88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 --2- 172.21.15.135:0/3074777507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfe00c4160 0x7fcfe0152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 >> 172.21.15.135:0/3074777507 conn(0x7fcfe00c5ce0 msgr2=0x7fcfe014b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 shutdown_connections 2024-04-26T22:00:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.143+0000 7fcfe74fc640 1 -- 172.21.15.135:0/3074777507 wait complete. 2024-04-26T22:00:35.155 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-04-26T22:00:35.155 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:35.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.252+0000 7fc95ba2a640 1 Processor -- start 2024-04-26T22:00:35.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.252+0000 7fc95ba2a640 1 -- start start 2024-04-26T22:00:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc95ba2a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc95413feb0 0x7fc9541576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc95ba2a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c2b80 0x7fc95406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc95ba2a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc954156c10 con 0x7fc9540c5050 2024-04-26T22:00:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc95ba2a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc95415a8b0 con 0x7fc95413feb0 2024-04-26T22:00:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc95ba2a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc954157240 con 0x7fc9540c2b80 2024-04-26T22:00:35.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc95413feb0 0x7fc9541576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc95413feb0 0x7fc9541576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41146/0 (socket says 172.21.15.135:41146) 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 -- 172.21.15.135:0/3510678639 learned_addr learned my addr 172.21.15.135:0/3510678639 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc959fc7640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c2b80 0x7fc95406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 -- 172.21.15.135:0/3510678639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c2b80 msgr2=0x7fc95406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c2b80 0x7fc95406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 -- 172.21.15.135:0/3510678639 >> v1:172.21.15.139:6789/0 conn(0x7fc9540c5050 legacy=0x7fc95415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc9597c6640 1 -- 172.21.15.135:0/3510678639 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9540c5570 con 0x7fc95413feb0 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.253+0000 7fc959fc7640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c2b80 0x7fc95406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:35.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc9597c6640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541576d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc950009870 tx=0x7fc95002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c716ec9455788 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc94bfff640 1 -- 172.21.15.135:0/3510678639 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950037070 con 0x7fc95413feb0 2024-04-26T22:00:35.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc94bfff640 1 -- 172.21.15.135:0/3510678639 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc950032440 con 0x7fc95413feb0 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc94bfff640 1 -- 172.21.15.135:0/3510678639 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950032810 con 0x7fc95413feb0 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3510678639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 msgr2=0x7fc9541576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541576d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc950009870 tx=0x7fc95002a180 comp rx=0 tx=0).stop 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3510678639 shutdown_connections 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c2b80 0x7fc95406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3510678639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541576d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.254+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3510678639 >> 172.21.15.135:0/3510678639 conn(0x7fc9540c5ce0 msgr2=0x7fc9540c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.255+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3510678639 shutdown_connections 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.255+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3510678639 wait complete. 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 Processor -- start 2024-04-26T22:00:35.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 -- start start 2024-04-26T22:00:35.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9540c2b80 0x7fc95413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c5050 0x7fc95413cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc954140290 con 0x7fc95413feb0 2024-04-26T22:00:35.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc954156ed0 con 0x7fc9540c2b80 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc95ba2a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc954140410 con 0x7fc9540c5050 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc959fc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc959fc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541f03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41158/0 (socket says 172.21.15.135:41158) 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc959fc7640 1 -- 172.21.15.135:0/3260911494 learned_addr learned my addr 172.21.15.135:0/3260911494 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc9597c6640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9540c2b80 0x7fc95413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.256+0000 7fc958fc5640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c5050 0x7fc95413cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc959fc7640 1 -- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c5050 msgr2=0x7fc95413cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc959fc7640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c5050 0x7fc95413cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc959fc7640 1 -- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9540c2b80 msgr2=0x7fc95413ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc959fc7640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9540c2b80 0x7fc95413ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc959fc7640 1 -- 172.21.15.135:0/3260911494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9541e0be0 con 0x7fc95413feb0 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc959fc7640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541f03a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00cf20 tx=0x7fc94c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c01a070 con 0x7fc95413feb0 2024-04-26T22:00:35.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c00d840 con 0x7fc95413feb0 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc954157ea0 con 0x7fc95413feb0 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00dc30 con 0x7fc95413feb0 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.257+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541e0ec0 con 0x7fc95413feb0 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.258+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9540696e0 con 0x7fc95413feb0 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.259+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc94c01d070 con 0x7fc95413feb0 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.259+0000 7fc949ffb640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c920 0x7fc93806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.259+0000 7fc9597c6640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c920 0x7fc93806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.259+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc94c08c570 con 0x7fc95413feb0 2024-04-26T22:00:35.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.260+0000 7fc9597c6640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c920 0x7fc93806ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc950034000 tx=0x7fc95002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.263+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc94c05a170 con 0x7fc95413feb0 2024-04-26T22:00:35.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.473+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9540507a0 con 0x7fc95413feb0 2024-04-26T22:00:35.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.473+0000 7fc949ffb640 1 -- 172.21.15.135:0/3260911494 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7fc94c05e470 con 0x7fc95413feb0 2024-04-26T22:00:35.473 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:35.474 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:35.474 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:35.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.475+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c920 msgr2=0x7fc93806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.475+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c920 0x7fc93806ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc950034000 tx=0x7fc95002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.475+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 msgr2=0x7fc9541f03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.475+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541f03a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00cf20 tx=0x7fc94c00a7d0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.475+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 shutdown_connections 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc93806c920 0x7fc93806ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc95413feb0 0x7fc9541f03a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9540c5050 0x7fc95413cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 --2- 172.21.15.135:0/3260911494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9540c2b80 0x7fc95413ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 >> 172.21.15.135:0/3260911494 conn(0x7fc9540c5ce0 msgr2=0x7fc95406f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 shutdown_connections 2024-04-26T22:00:35.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.476+0000 7fc95ba2a640 1 -- 172.21.15.135:0/3260911494 wait complete. 2024-04-26T22:00:35.486 INFO:tasks.mds_thrash:Ready to start thrashing 2024-04-26T22:00:35.487 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-04-26T22:00:35.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:35.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.583+0000 7fea70b71640 1 Processor -- start 2024-04-26T22:00:35.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.583+0000 7fea70b71640 1 -- start start 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea70b71640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c14ec70 0x7fea6c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea70b71640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea6c074ba0 0x7fea6c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea70b71640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6c0733a0 con 0x7fea6c06cf20 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea70b71640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fea6c074540 con 0x7fea6c074ba0 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea70b71640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c06ee00 con 0x7fea6c14ec70 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea6ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fea6c06cf20 0x7fea6c0737c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54178/0 (socket says 172.21.15.135:54178) 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea6ad9d640 1 -- 172.21.15.135:0/1058344658 learned_addr learned my addr 172.21.15.135:0/1058344658 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea6a59c640 1 --2- 172.21.15.135:0/1058344658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c14ec70 0x7fea6c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.584+0000 7fea69d9b640 1 --2- 172.21.15.135:0/1058344658 >> v2:172.21.15.139:3300/0 conn(0x7fea6c074ba0 0x7fea6c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6959a640 1 -- 172.21.15.135:0/1058344658 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327958872 0 0) 0x7fea6c0733a0 con 0x7fea6c06cf20 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6959a640 1 -- 172.21.15.135:0/1058344658 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea54003520 con 0x7fea6c06cf20 2024-04-26T22:00:35.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6a59c640 1 -- 172.21.15.135:0/1058344658 >> v2:172.21.15.139:3300/0 conn(0x7fea6c074ba0 msgr2=0x7fea6c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:35.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6a59c640 1 --2- 172.21.15.135:0/1058344658 >> v2:172.21.15.139:3300/0 conn(0x7fea6c074ba0 0x7fea6c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6a59c640 1 -- 172.21.15.135:0/1058344658 >> v1:172.21.15.139:6789/0 conn(0x7fea6c06cf20 legacy=0x7fea6c0737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6a59c640 1 -- 172.21.15.135:0/1058344658 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c0c30f0 con 0x7fea6c14ec70 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.585+0000 7fea6a59c640 1 --2- 172.21.15.135:0/1058344658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c14ec70 0x7fea6c06e8c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009d90 tx=0x7fea5c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10aa21866055d720 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea6959a640 1 -- 172.21.15.135:0/1058344658 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c031600 con 0x7fea6c14ec70 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea6959a640 1 -- 172.21.15.135:0/1058344658 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c0317a0 con 0x7fea6c14ec70 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea6959a640 1 -- 172.21.15.135:0/1058344658 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c031b50 con 0x7fea6c14ec70 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 -- 172.21.15.135:0/1058344658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c14ec70 msgr2=0x7fea6c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 --2- 172.21.15.135:0/1058344658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c14ec70 0x7fea6c06e8c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009d90 tx=0x7fea5c02a5d0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 -- 172.21.15.135:0/1058344658 shutdown_connections 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 --2- 172.21.15.135:0/1058344658 >> v2:172.21.15.139:3300/0 conn(0x7fea6c074ba0 0x7fea6c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 --2- 172.21.15.135:0/1058344658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c14ec70 0x7fea6c06e8c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 -- 172.21.15.135:0/1058344658 >> 172.21.15.135:0/1058344658 conn(0x7fea6c06e090 msgr2=0x7fea6c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 -- 172.21.15.135:0/1058344658 shutdown_connections 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.586+0000 7fea70b71640 1 -- 172.21.15.135:0/1058344658 wait complete. 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 Processor -- start 2024-04-26T22:00:35.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 -- start start 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea6c06cf20 0x7fea6c1477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c074ba0 0x7fea6c147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 0x7fea6c1ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c072290 con 0x7fea6c14ec70 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c14f050 con 0x7fea6c06cf20 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea70b71640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c155900 con 0x7fea6c074ba0 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea6ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 0x7fea6c1ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea6ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 0x7fea6c1ee2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41162/0 (socket says 172.21.15.135:41162) 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.588+0000 7fea6ad9d640 1 -- 172.21.15.135:0/1697095357 learned_addr learned my addr 172.21.15.135:0/1697095357 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:35.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6a59c640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea6c06cf20 0x7fea6c1477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea69d9b640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c074ba0 0x7fea6c147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6ad9d640 1 -- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c074ba0 msgr2=0x7fea6c147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6ad9d640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c074ba0 0x7fea6c147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6ad9d640 1 -- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea6c06cf20 msgr2=0x7fea6c1477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6ad9d640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea6c06cf20 0x7fea6c1477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6ad9d640 1 -- 172.21.15.135:0/1697095357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c1deec0 con 0x7fea6c14ec70 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea6ad9d640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 0x7fea6c1ee2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c430 tx=0x7fea6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64017070 con 0x7fea6c14ec70 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea64005ce0 con 0x7fea6c14ec70 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c1df140 con 0x7fea6c14ec70 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64007310 con 0x7fea6c14ec70 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.589+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c1df700 con 0x7fea6c14ec70 2024-04-26T22:00:35.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.590+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6c152420 con 0x7fea6c14ec70 2024-04-26T22:00:35.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.591+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea6401a070 con 0x7fea6c14ec70 2024-04-26T22:00:35.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.591+0000 7fea5affd640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fea4c06c8f0 0x7fea4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.591+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fea6408b3d0 con 0x7fea6c14ec70 2024-04-26T22:00:35.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.591+0000 7fea6a59c640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fea4c06c8f0 0x7fea4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.591+0000 7fea6a59c640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fea4c06c8f0 0x7fea4c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea5c034000 tx=0x7fea5c033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.595+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea6c1deec0 con 0x7fea6c14ec70 2024-04-26T22:00:35.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.769+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea6c0507a0 con 0x7fea4c06c8f0 2024-04-26T22:00:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.779+0000 7fea5affd640 1 -- 172.21.15.135:0/1697095357 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490299 (secure 0 0 0) 0x7fea6c0507a0 con 0x7fea4c06c8f0 2024-04-26T22:00:35.785 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:35.788 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fea4c06c8f0 msgr2=0x7fea4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fea4c06c8f0 0x7fea4c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea5c034000 tx=0x7fea5c033040 comp rx=0 tx=0).stop 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 msgr2=0x7fea6c1ee2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 0x7fea6c1ee2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c430 tx=0x7fea6400c8f0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 shutdown_connections 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fea4c06c8f0 0x7fea4c06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c14ec70 0x7fea6c1ee2b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea6c074ba0 0x7fea6c147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 --2- 172.21.15.135:0/1697095357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea6c06cf20 0x7fea6c1477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 >> 172.21.15.135:0/1697095357 conn(0x7fea6c06e090 msgr2=0x7fea6c06efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:35.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 shutdown_connections 2024-04-26T22:00:35.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.783+0000 7fea70b71640 1 -- 172.21.15.135:0/1697095357 wait complete. 2024-04-26T22:00:35.794 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-26T22:00:35.077788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595444,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219456,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755264,"statfs":{"total":773094113280,"available":772869390336,"internally_reserved":0,"allocated":3350528,"data_stored":1984976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002759"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:29.054882+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:29.054882+0000","last_peered":"2024-04-26T22:00:29.054882+0000","last_clean":"2024-04-26T22:00:29.054882+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:29.054882+0000","last_undegraded":"2024-04-26T22:00:29.054882+0000","last_fullsized":"2024-04-26T22:00:29.054882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":102763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:35.802 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.932+0000 7fc2f1c1d640 1 Processor -- start 2024-04-26T22:00:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.932+0000 7fc2f1c1d640 1 -- start start 2024-04-26T22:00:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2f1c1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2f1c1d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ec0c3d10 con 0x7fc2ec06f790 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2f1c1d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ec0c4290 con 0x7fc2ec150d20 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2f1c1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ec157d90 con 0x7fc2ec15a360 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2e3fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc2ec06f790 0x7fc2ec157680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39280/0 (socket says 172.21.15.135:39280) 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2e3fff640 1 -- 172.21.15.135:0/2471713462 learned_addr learned my addr 172.21.15.135:0/2471713462 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.933+0000 7fc2ebfff640 1 --2- 172.21.15.135:0/2471713462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.934+0000 7fc2ebfff640 1 -- 172.21.15.135:0/2471713462 >> v1:172.21.15.139:6789/0 conn(0x7fc2ec150d20 legacy=0x7fc2ec0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.934+0000 7fc2ebfff640 1 -- 172.21.15.135:0/2471713462 >> v1:172.21.15.135:6789/0 conn(0x7fc2ec06f790 legacy=0x7fc2ec157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.934+0000 7fc2ebfff640 1 -- 172.21.15.135:0/2471713462 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec0c3fb0 con 0x7fc2ec15a360 2024-04-26T22:00:35.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2ebfff640 1 --2- 172.21.15.135:0/2471713462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec06d320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009dc0 tx=0x7fc2e402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8307b6bc52bf584a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2eaffd640 1 -- 172.21.15.135:0/2471713462 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e402ae60 con 0x7fc2ec15a360 2024-04-26T22:00:35.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2eaffd640 1 -- 172.21.15.135:0/2471713462 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e4007740 con 0x7fc2ec15a360 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2eaffd640 1 -- 172.21.15.135:0/2471713462 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4007b10 con 0x7fc2ec15a360 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/2471713462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 msgr2=0x7fc2ec06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/2471713462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec06d320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009dc0 tx=0x7fc2e402a640 comp rx=0 tx=0).stop 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/2471713462 shutdown_connections 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/2471713462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec06d320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/2471713462 >> 172.21.15.135:0/2471713462 conn(0x7fc2ec0c5ce0 msgr2=0x7fc2ec0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/2471713462 shutdown_connections 2024-04-26T22:00:35.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.935+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/2471713462 wait complete. 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.936+0000 7fc2f1c1d640 1 Processor -- start 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 -- start start 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 0x7fc2ec0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2ec150d20 0x7fc2ec1e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec1e2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec157160 con 0x7fc2ec06f790 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec15a7d0 con 0x7fc2ec150d20 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2f1c1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ec151100 con 0x7fc2ec15a360 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 0x7fc2ec0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 0x7fc2ec0733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41174/0 (socket says 172.21.15.135:41174) 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 -- 172.21.15.135:0/1157652094 learned_addr learned my addr 172.21.15.135:0/1157652094 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2e3fff640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2ec150d20 0x7fc2ec1e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2ebfff640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec1e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 -- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 msgr2=0x7fc2ec1e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec1e2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 -- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2ec150d20 msgr2=0x7fc2ec1e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.937+0000 7fc2eb7fe640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2ec150d20 0x7fc2ec1e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2eb7fe640 1 -- 172.21.15.135:0/1157652094 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec1ed4f0 con 0x7fc2ec06f790 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2ebfff640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec1e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2e3fff640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2ec150d20 0x7fc2ec1e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2eb7fe640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 0x7fc2ec0733d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c400 tx=0x7fc2dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0052d0 con 0x7fc2ec06f790 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc005490 con 0x7fc2ec06f790 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec1ed7d0 con 0x7fc2ec06f790 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc007260 con 0x7fc2ec06f790 2024-04-26T22:00:35.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.938+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec1e1a10 con 0x7fc2ec06f790 2024-04-26T22:00:35.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.939+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec1e1c10 con 0x7fc2ec06f790 2024-04-26T22:00:35.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.939+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2dc007400 con 0x7fc2ec06f790 2024-04-26T22:00:35.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.940+0000 7fc2e8ff9640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc2cc06c920 0x7fc2cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:35.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.940+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fc2dc08b320 con 0x7fc2ec06f790 2024-04-26T22:00:35.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.940+0000 7fc2e3fff640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc2cc06c920 0x7fc2cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:35.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.940+0000 7fc2e3fff640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc2cc06c920 0x7fc2cc06ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec1e30b0 tx=0x7fc2d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:35.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:35.944+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2ec1e1c10 con 0x7fc2ec06f790 2024-04-26T22:00:36.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.115+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2ec1e1570 con 0x7fc2cc06c920 2024-04-26T22:00:36.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.125+0000 7fc2e8ff9640 1 -- 172.21.15.135:0/1157652094 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490299 (secure 0 0 0) 0x7fc2d40013a0 con 0x7fc2cc06c920 2024-04-26T22:00:36.130 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:36.133 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:36.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc2cc06c920 msgr2=0x7fc2cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc2cc06c920 0x7fc2cc06ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec1e30b0 tx=0x7fc2d4009040 comp rx=0 tx=0).stop 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 msgr2=0x7fc2ec0733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 0x7fc2ec0733d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c400 tx=0x7fc2dc00c8c0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 shutdown_connections 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc2cc06c920 0x7fc2cc06ede0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2ec15a360 0x7fc2ec1e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2ec150d20 0x7fc2ec1e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 --2- 172.21.15.135:0/1157652094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2ec06f790 0x7fc2ec0733d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 >> 172.21.15.135:0/1157652094 conn(0x7fc2ec0c5ce0 msgr2=0x7fc2ec06d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 shutdown_connections 2024-04-26T22:00:36.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.130+0000 7fc2f1c1d640 1 -- 172.21.15.135:0/1157652094 wait complete. 2024-04-26T22:00:36.140 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-26T22:00:35.077788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595444,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219456,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755264,"statfs":{"total":773094113280,"available":772869390336,"internally_reserved":0,"allocated":3350528,"data_stored":1984976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002759"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:29.054882+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:29.054882+0000","last_peered":"2024-04-26T22:00:29.054882+0000","last_clean":"2024-04-26T22:00:29.054882+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:29.054882+0000","last_undegraded":"2024-04-26T22:00:29.054882+0000","last_fullsized":"2024-04-26T22:00:29.054882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":102763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:36.147 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-04-26T22:00:36.148 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:00:36.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.277+0000 7f68b3c37640 1 Processor -- start 2024-04-26T22:00:36.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b3c37640 1 -- start start 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b3c37640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68ac1570d0 0x7f68ac150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b3c37640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac150f90 0x7f68ac1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b3c37640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68ac0c3810 con 0x7f68ac0ab3f0 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b3c37640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f68ac0c4330 con 0x7f68ac1570d0 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b3c37640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68ac0c3ab0 con 0x7f68ac150f90 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.278+0000 7f68b19d3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68ac1570d0 0x7f68ac150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68ac1570d0 0x7f68ac150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41184/0 (socket says 172.21.15.135:41184) 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 -- 172.21.15.135:0/434639968 learned_addr learned my addr 172.21.15.135:0/434639968 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 -- 172.21.15.135:0/434639968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac150f90 msgr2=0x7f68ac1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:36.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b21d4640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac150f90 0x7f68ac1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac150f90 0x7f68ac1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 -- 172.21.15.135:0/434639968 >> v1:172.21.15.139:6789/0 conn(0x7f68ac0ab3f0 legacy=0x7f68ac0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:36.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 -- 172.21.15.135:0/434639968 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac0c4580 con 0x7f68ac1570d0 2024-04-26T22:00:36.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b21d4640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac150f90 0x7f68ac1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:36.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.279+0000 7f68b19d3640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac1570d0 0x7f68ac150a50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0c44b0 tx=0x7f68a402a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6c6b80de5ad584 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b09d1640 1 -- 172.21.15.135:0/434639968 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4036070 con 0x7f68ac1570d0 2024-04-26T22:00:36.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b09d1640 1 -- 172.21.15.135:0/434639968 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a4031600 con 0x7f68ac1570d0 2024-04-26T22:00:36.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b09d1640 1 -- 172.21.15.135:0/434639968 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40319d0 con 0x7f68ac1570d0 2024-04-26T22:00:36.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 -- 172.21.15.135:0/434639968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac1570d0 msgr2=0x7f68ac150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac1570d0 0x7f68ac150a50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0c44b0 tx=0x7f68a402a590 comp rx=0 tx=0).stop 2024-04-26T22:00:36.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 -- 172.21.15.135:0/434639968 shutdown_connections 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac150f90 0x7f68ac1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 --2- 172.21.15.135:0/434639968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac1570d0 0x7f68ac150a50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 -- 172.21.15.135:0/434639968 >> 172.21.15.135:0/434639968 conn(0x7f68ac06c310 msgr2=0x7f68ac06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 -- 172.21.15.135:0/434639968 shutdown_connections 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.280+0000 7f68b3c37640 1 -- 172.21.15.135:0/434639968 wait complete. 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.281+0000 7f68b3c37640 1 Processor -- start 2024-04-26T22:00:36.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.281+0000 7f68b3c37640 1 -- start start 2024-04-26T22:00:36.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b3c37640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68ac0ab3f0 0x7f68ac148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b3c37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 0x7f68ac1eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b3c37640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac1570d0 0x7f68ac155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b3c37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac0abbe0 con 0x7f68ac150f90 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b3c37640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac0aba60 con 0x7f68ac0ab3f0 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b3c37640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68ac0c5490 con 0x7f68ac1570d0 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 0x7f68ac1eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 0x7f68ac1eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41190/0 (socket says 172.21.15.135:41190) 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 -- 172.21.15.135:0/202506739 learned_addr learned my addr 172.21.15.135:0/202506739 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b21d4640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac1570d0 0x7f68ac155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b19d3640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68ac0ab3f0 0x7f68ac148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 -- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac1570d0 msgr2=0x7f68ac155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac1570d0 0x7f68ac155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 -- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68ac0ab3f0 msgr2=0x7f68ac148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68ac0ab3f0 0x7f68ac148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 -- 172.21.15.135:0/202506739 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac1ee550 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b19d3640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68ac0ab3f0 0x7f68ac148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.282+0000 7f68b11d2640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 0x7f68ac1eac80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c430 tx=0x7f689c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.283+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c011790 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.283+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689c017070 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.283+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac1ee830 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.283+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c011bb0 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.283+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac156a40 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.284+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ac0507a0 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.284+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f689c01c460 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.284+0000 7f68a27fc640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f689006c8f0 0x7f689006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.285+0000 7f68b19d3640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f689006c8f0 0x7f689006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.285+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f689c08bc90 con 0x7f68ac150f90 2024-04-26T22:00:36.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.285+0000 7f68b19d3640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f689006c8f0 0x7f689006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68a4033000 tx=0x7f68a4009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.289+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f689c059e20 con 0x7f68ac150f90 2024-04-26T22:00:36.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.460+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68ac0c4110 con 0x7f689006c8f0 2024-04-26T22:00:36.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.471+0000 7f68a27fc640 1 -- 172.21.15.135:0/202506739 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490299 (secure 0 0 0) 0x7f68ac0c4110 con 0x7f689006c8f0 2024-04-26T22:00:36.476 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:36.479 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f689006c8f0 msgr2=0x7f689006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f689006c8f0 0x7f689006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68a4033000 tx=0x7f68a4009000 comp rx=0 tx=0).stop 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 msgr2=0x7f68ac1eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 0x7f68ac1eac80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c430 tx=0x7f689c00c8f0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 shutdown_connections 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f689006c8f0 0x7f689006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68ac1570d0 0x7f68ac155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68ac150f90 0x7f68ac1eac80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 --2- 172.21.15.135:0/202506739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68ac0ab3f0 0x7f68ac148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 >> 172.21.15.135:0/202506739 conn(0x7f68ac06c310 msgr2=0x7f68ac0c2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 shutdown_connections 2024-04-26T22:00:36.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.475+0000 7f68b3c37640 1 -- 172.21.15.135:0/202506739 wait complete. 2024-04-26T22:00:36.486 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-26T22:00:35.077788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595444,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219456,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755264,"statfs":{"total":773094113280,"available":772869390336,"internally_reserved":0,"allocated":3350528,"data_stored":1984976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002759"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810496+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:00:07.810496+0000","last_peered":"2024-04-26T22:00:07.810496+0000","last_clean":"2024-04-26T22:00:07.810496+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:00:07.810496+0000","last_undegraded":"2024-04-26T22:00:07.810496+0000","last_fullsized":"2024-04-26T22:00:07.810496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807075+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:00:07.807075+0000","last_peered":"2024-04-26T22:00:07.807075+0000","last_clean":"2024-04-26T22:00:07.807075+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:00:07.807075+0000","last_undegraded":"2024-04-26T22:00:07.807075+0000","last_fullsized":"2024-04-26T22:00:07.807075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259258+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:00:08.259258+0000","last_peered":"2024-04-26T22:00:08.259258+0000","last_clean":"2024-04-26T22:00:08.259258+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:00:08.259258+0000","last_undegraded":"2024-04-26T22:00:08.259258+0000","last_fullsized":"2024-04-26T22:00:08.259258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258419+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:00:08.258419+0000","last_peered":"2024-04-26T22:00:08.258419+0000","last_clean":"2024-04-26T22:00:08.258419+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:00:08.258419+0000","last_undegraded":"2024-04-26T22:00:08.258419+0000","last_fullsized":"2024-04-26T22:00:08.258419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808726+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:00:07.808726+0000","last_peered":"2024-04-26T22:00:07.808726+0000","last_clean":"2024-04-26T22:00:07.808726+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:00:07.808726+0000","last_undegraded":"2024-04-26T22:00:07.808726+0000","last_fullsized":"2024-04-26T22:00:07.808726+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259347+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:00:08.259347+0000","last_peered":"2024-04-26T22:00:08.259347+0000","last_clean":"2024-04-26T22:00:08.259347+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:00:08.259347+0000","last_undegraded":"2024-04-26T22:00:08.259347+0000","last_fullsized":"2024-04-26T22:00:08.259347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806974+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:00:07.806974+0000","last_peered":"2024-04-26T22:00:07.806974+0000","last_clean":"2024-04-26T22:00:07.806974+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:00:07.806974+0000","last_undegraded":"2024-04-26T22:00:07.806974+0000","last_fullsized":"2024-04-26T22:00:07.806974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810346+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:00:07.810346+0000","last_peered":"2024-04-26T22:00:07.810346+0000","last_clean":"2024-04-26T22:00:07.810346+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:00:07.810346+0000","last_undegraded":"2024-04-26T22:00:07.810346+0000","last_fullsized":"2024-04-26T22:00:07.810346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258847+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:00:08.258847+0000","last_peered":"2024-04-26T22:00:08.258847+0000","last_clean":"2024-04-26T22:00:08.258847+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:00:08.258847+0000","last_undegraded":"2024-04-26T22:00:08.258847+0000","last_fullsized":"2024-04-26T22:00:08.258847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259655+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:00:08.259655+0000","last_peered":"2024-04-26T22:00:08.259655+0000","last_clean":"2024-04-26T22:00:08.259655+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:00:08.259655+0000","last_undegraded":"2024-04-26T22:00:08.259655+0000","last_fullsized":"2024-04-26T22:00:08.259655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260183+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:00:08.260183+0000","last_peered":"2024-04-26T22:00:08.260183+0000","last_clean":"2024-04-26T22:00:08.260183+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:00:08.260183+0000","last_undegraded":"2024-04-26T22:00:08.260183+0000","last_fullsized":"2024-04-26T22:00:08.260183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810768+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:00:07.810768+0000","last_peered":"2024-04-26T22:00:07.810768+0000","last_clean":"2024-04-26T22:00:07.810768+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:00:07.810768+0000","last_undegraded":"2024-04-26T22:00:07.810768+0000","last_fullsized":"2024-04-26T22:00:07.810768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259550+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:00:08.259550+0000","last_peered":"2024-04-26T22:00:08.259550+0000","last_clean":"2024-04-26T22:00:08.259550+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:00:08.259550+0000","last_undegraded":"2024-04-26T22:00:08.259550+0000","last_fullsized":"2024-04-26T22:00:08.259550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259363+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:00:08.259363+0000","last_peered":"2024-04-26T22:00:08.259363+0000","last_clean":"2024-04-26T22:00:08.259363+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:00:08.259363+0000","last_undegraded":"2024-04-26T22:00:08.259363+0000","last_fullsized":"2024-04-26T22:00:08.259363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260537+0000","last_change":"2024-04-26T21:59:57.646827+0000","last_active":"2024-04-26T22:00:08.260537+0000","last_peered":"2024-04-26T22:00:08.260537+0000","last_clean":"2024-04-26T22:00:08.260537+0000","last_became_active":"2024-04-26T21:59:57.646539+0000","last_became_peered":"2024-04-26T21:59:57.646539+0000","last_unstale":"2024-04-26T22:00:08.260537+0000","last_undegraded":"2024-04-26T22:00:08.260537+0000","last_fullsized":"2024-04-26T22:00:08.260537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810671+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:00:07.810671+0000","last_peered":"2024-04-26T22:00:07.810671+0000","last_clean":"2024-04-26T22:00:07.810671+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:00:07.810671+0000","last_undegraded":"2024-04-26T22:00:07.810671+0000","last_fullsized":"2024-04-26T22:00:07.810671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809091+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:00:07.809091+0000","last_peered":"2024-04-26T22:00:07.809091+0000","last_clean":"2024-04-26T22:00:07.809091+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:00:07.809091+0000","last_undegraded":"2024-04-26T22:00:07.809091+0000","last_fullsized":"2024-04-26T22:00:07.809091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809470+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:00:07.809470+0000","last_peered":"2024-04-26T22:00:07.809470+0000","last_clean":"2024-04-26T22:00:07.809470+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:00:07.809470+0000","last_undegraded":"2024-04-26T22:00:07.809470+0000","last_fullsized":"2024-04-26T22:00:07.809470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259202+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:00:08.259202+0000","last_peered":"2024-04-26T22:00:08.259202+0000","last_clean":"2024-04-26T22:00:08.259202+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:00:08.259202+0000","last_undegraded":"2024-04-26T22:00:08.259202+0000","last_fullsized":"2024-04-26T22:00:08.259202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811219+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:00:07.811219+0000","last_peered":"2024-04-26T22:00:07.811219+0000","last_clean":"2024-04-26T22:00:07.811219+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:00:07.811219+0000","last_undegraded":"2024-04-26T22:00:07.811219+0000","last_fullsized":"2024-04-26T22:00:07.811219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804897+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:00:07.804897+0000","last_peered":"2024-04-26T22:00:07.804897+0000","last_clean":"2024-04-26T22:00:07.804897+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:00:07.804897+0000","last_undegraded":"2024-04-26T22:00:07.804897+0000","last_fullsized":"2024-04-26T22:00:07.804897+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258805+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:00:08.258805+0000","last_peered":"2024-04-26T22:00:08.258805+0000","last_clean":"2024-04-26T22:00:08.258805+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:00:08.258805+0000","last_undegraded":"2024-04-26T22:00:08.258805+0000","last_fullsized":"2024-04-26T22:00:08.258805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808995+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:00:07.808995+0000","last_peered":"2024-04-26T22:00:07.808995+0000","last_clean":"2024-04-26T22:00:07.808995+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:00:07.808995+0000","last_undegraded":"2024-04-26T22:00:07.808995+0000","last_fullsized":"2024-04-26T22:00:07.808995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811013+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:00:07.811013+0000","last_peered":"2024-04-26T22:00:07.811013+0000","last_clean":"2024-04-26T22:00:07.811013+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:00:07.811013+0000","last_undegraded":"2024-04-26T22:00:07.811013+0000","last_fullsized":"2024-04-26T22:00:07.811013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811418+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:00:07.811418+0000","last_peered":"2024-04-26T22:00:07.811418+0000","last_clean":"2024-04-26T22:00:07.811418+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:00:07.811418+0000","last_undegraded":"2024-04-26T22:00:07.811418+0000","last_fullsized":"2024-04-26T22:00:07.811418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258689+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:00:08.258689+0000","last_peered":"2024-04-26T22:00:08.258689+0000","last_clean":"2024-04-26T22:00:08.258689+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:00:08.258689+0000","last_undegraded":"2024-04-26T22:00:08.258689+0000","last_fullsized":"2024-04-26T22:00:08.258689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808916+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:00:07.808916+0000","last_peered":"2024-04-26T22:00:07.808916+0000","last_clean":"2024-04-26T22:00:07.808916+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:00:07.808916+0000","last_undegraded":"2024-04-26T22:00:07.808916+0000","last_fullsized":"2024-04-26T22:00:07.808916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809208+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:00:07.809208+0000","last_peered":"2024-04-26T22:00:07.809208+0000","last_clean":"2024-04-26T22:00:07.809208+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:00:07.809208+0000","last_undegraded":"2024-04-26T22:00:07.809208+0000","last_fullsized":"2024-04-26T22:00:07.809208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260759+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:00:08.260759+0000","last_peered":"2024-04-26T22:00:08.260759+0000","last_clean":"2024-04-26T22:00:08.260759+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:00:08.260759+0000","last_undegraded":"2024-04-26T22:00:08.260759+0000","last_fullsized":"2024-04-26T22:00:08.260759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811510+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:00:07.811510+0000","last_peered":"2024-04-26T22:00:07.811510+0000","last_clean":"2024-04-26T22:00:07.811510+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:00:07.811510+0000","last_undegraded":"2024-04-26T22:00:07.811510+0000","last_fullsized":"2024-04-26T22:00:07.811510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258845+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:00:08.258845+0000","last_peered":"2024-04-26T22:00:08.258845+0000","last_clean":"2024-04-26T22:00:08.258845+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:00:08.258845+0000","last_undegraded":"2024-04-26T22:00:08.258845+0000","last_fullsized":"2024-04-26T22:00:08.258845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811265+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:00:07.811265+0000","last_peered":"2024-04-26T22:00:07.811265+0000","last_clean":"2024-04-26T22:00:07.811265+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:00:07.811265+0000","last_undegraded":"2024-04-26T22:00:07.811265+0000","last_fullsized":"2024-04-26T22:00:07.811265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258350+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:00:08.258350+0000","last_peered":"2024-04-26T22:00:08.258350+0000","last_clean":"2024-04-26T22:00:08.258350+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:00:08.258350+0000","last_undegraded":"2024-04-26T22:00:08.258350+0000","last_fullsized":"2024-04-26T22:00:08.258350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807183+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:00:07.807183+0000","last_peered":"2024-04-26T22:00:07.807183+0000","last_clean":"2024-04-26T22:00:07.807183+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:00:07.807183+0000","last_undegraded":"2024-04-26T22:00:07.807183+0000","last_fullsized":"2024-04-26T22:00:07.807183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810175+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:00:07.810175+0000","last_peered":"2024-04-26T22:00:07.810175+0000","last_clean":"2024-04-26T22:00:07.810175+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:00:07.810175+0000","last_undegraded":"2024-04-26T22:00:07.810175+0000","last_fullsized":"2024-04-26T22:00:07.810175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810753+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:00:07.810753+0000","last_peered":"2024-04-26T22:00:07.810753+0000","last_clean":"2024-04-26T22:00:07.810753+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:00:07.810753+0000","last_undegraded":"2024-04-26T22:00:07.810753+0000","last_fullsized":"2024-04-26T22:00:07.810753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811113+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:00:07.811113+0000","last_peered":"2024-04-26T22:00:07.811113+0000","last_clean":"2024-04-26T22:00:07.811113+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:00:07.811113+0000","last_undegraded":"2024-04-26T22:00:07.811113+0000","last_fullsized":"2024-04-26T22:00:07.811113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259272+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:00:08.259272+0000","last_peered":"2024-04-26T22:00:08.259272+0000","last_clean":"2024-04-26T22:00:08.259272+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:00:08.259272+0000","last_undegraded":"2024-04-26T22:00:08.259272+0000","last_fullsized":"2024-04-26T22:00:08.259272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809271+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:00:07.809271+0000","last_peered":"2024-04-26T22:00:07.809271+0000","last_clean":"2024-04-26T22:00:07.809271+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:00:07.809271+0000","last_undegraded":"2024-04-26T22:00:07.809271+0000","last_fullsized":"2024-04-26T22:00:07.809271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258229+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:00:08.258229+0000","last_peered":"2024-04-26T22:00:08.258229+0000","last_clean":"2024-04-26T22:00:08.258229+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:00:08.258229+0000","last_undegraded":"2024-04-26T22:00:08.258229+0000","last_fullsized":"2024-04-26T22:00:08.258229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809396+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:00:07.809396+0000","last_peered":"2024-04-26T22:00:07.809396+0000","last_clean":"2024-04-26T22:00:07.809396+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:00:07.809396+0000","last_undegraded":"2024-04-26T22:00:07.809396+0000","last_fullsized":"2024-04-26T22:00:07.809396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809138+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:00:07.809138+0000","last_peered":"2024-04-26T22:00:07.809138+0000","last_clean":"2024-04-26T22:00:07.809138+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:00:07.809138+0000","last_undegraded":"2024-04-26T22:00:07.809138+0000","last_fullsized":"2024-04-26T22:00:07.809138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259841+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:00:08.259841+0000","last_peered":"2024-04-26T22:00:08.259841+0000","last_clean":"2024-04-26T22:00:08.259841+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:00:08.259841+0000","last_undegraded":"2024-04-26T22:00:08.259841+0000","last_fullsized":"2024-04-26T22:00:08.259841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806877+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:00:07.806877+0000","last_peered":"2024-04-26T22:00:07.806877+0000","last_clean":"2024-04-26T22:00:07.806877+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:00:07.806877+0000","last_undegraded":"2024-04-26T22:00:07.806877+0000","last_fullsized":"2024-04-26T22:00:07.806877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807152+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:00:07.807152+0000","last_peered":"2024-04-26T22:00:07.807152+0000","last_clean":"2024-04-26T22:00:07.807152+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:00:07.807152+0000","last_undegraded":"2024-04-26T22:00:07.807152+0000","last_fullsized":"2024-04-26T22:00:07.807152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260639+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:00:08.260639+0000","last_peered":"2024-04-26T22:00:08.260639+0000","last_clean":"2024-04-26T22:00:08.260639+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:00:08.260639+0000","last_undegraded":"2024-04-26T22:00:08.260639+0000","last_fullsized":"2024-04-26T22:00:08.260639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809231+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:00:07.809231+0000","last_peered":"2024-04-26T22:00:07.809231+0000","last_clean":"2024-04-26T22:00:07.809231+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:00:07.809231+0000","last_undegraded":"2024-04-26T22:00:07.809231+0000","last_fullsized":"2024-04-26T22:00:07.809231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808751+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:00:07.808751+0000","last_peered":"2024-04-26T22:00:07.808751+0000","last_clean":"2024-04-26T22:00:07.808751+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:00:07.808751+0000","last_undegraded":"2024-04-26T22:00:07.808751+0000","last_fullsized":"2024-04-26T22:00:07.808751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806929+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:00:07.806929+0000","last_peered":"2024-04-26T22:00:07.806929+0000","last_clean":"2024-04-26T22:00:07.806929+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:00:07.806929+0000","last_undegraded":"2024-04-26T22:00:07.806929+0000","last_fullsized":"2024-04-26T22:00:07.806929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811275+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:00:07.811275+0000","last_peered":"2024-04-26T22:00:07.811275+0000","last_clean":"2024-04-26T22:00:07.811275+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:00:07.811275+0000","last_undegraded":"2024-04-26T22:00:07.811275+0000","last_fullsized":"2024-04-26T22:00:07.811275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259525+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:00:08.259525+0000","last_peered":"2024-04-26T22:00:08.259525+0000","last_clean":"2024-04-26T22:00:08.259525+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:00:08.259525+0000","last_undegraded":"2024-04-26T22:00:08.259525+0000","last_fullsized":"2024-04-26T22:00:08.259525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809120+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:00:07.809120+0000","last_peered":"2024-04-26T22:00:07.809120+0000","last_clean":"2024-04-26T22:00:07.809120+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:00:07.809120+0000","last_undegraded":"2024-04-26T22:00:07.809120+0000","last_fullsized":"2024-04-26T22:00:07.809120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804934+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:00:07.804934+0000","last_peered":"2024-04-26T22:00:07.804934+0000","last_clean":"2024-04-26T22:00:07.804934+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:00:07.804934+0000","last_undegraded":"2024-04-26T22:00:07.804934+0000","last_fullsized":"2024-04-26T22:00:07.804934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808632+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:00:07.808632+0000","last_peered":"2024-04-26T22:00:07.808632+0000","last_clean":"2024-04-26T22:00:07.808632+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:00:07.808632+0000","last_undegraded":"2024-04-26T22:00:07.808632+0000","last_fullsized":"2024-04-26T22:00:07.808632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804608+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:00:07.804608+0000","last_peered":"2024-04-26T22:00:07.804608+0000","last_clean":"2024-04-26T22:00:07.804608+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:00:07.804608+0000","last_undegraded":"2024-04-26T22:00:07.804608+0000","last_fullsized":"2024-04-26T22:00:07.804608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809286+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:00:07.809286+0000","last_peered":"2024-04-26T22:00:07.809286+0000","last_clean":"2024-04-26T22:00:07.809286+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:00:07.809286+0000","last_undegraded":"2024-04-26T22:00:07.809286+0000","last_fullsized":"2024-04-26T22:00:07.809286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260826+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:00:08.260826+0000","last_peered":"2024-04-26T22:00:08.260826+0000","last_clean":"2024-04-26T22:00:08.260826+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:00:08.260826+0000","last_undegraded":"2024-04-26T22:00:08.260826+0000","last_fullsized":"2024-04-26T22:00:08.260826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806645+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:00:07.806645+0000","last_peered":"2024-04-26T22:00:07.806645+0000","last_clean":"2024-04-26T22:00:07.806645+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:00:07.806645+0000","last_undegraded":"2024-04-26T22:00:07.806645+0000","last_fullsized":"2024-04-26T22:00:07.806645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258983+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:00:08.258983+0000","last_peered":"2024-04-26T22:00:08.258983+0000","last_clean":"2024-04-26T22:00:08.258983+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:00:08.258983+0000","last_undegraded":"2024-04-26T22:00:08.258983+0000","last_fullsized":"2024-04-26T22:00:08.258983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807120+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:00:07.807120+0000","last_peered":"2024-04-26T22:00:07.807120+0000","last_clean":"2024-04-26T22:00:07.807120+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:00:07.807120+0000","last_undegraded":"2024-04-26T22:00:07.807120+0000","last_fullsized":"2024-04-26T22:00:07.807120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259509+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:00:08.259509+0000","last_peered":"2024-04-26T22:00:08.259509+0000","last_clean":"2024-04-26T22:00:08.259509+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:00:08.259509+0000","last_undegraded":"2024-04-26T22:00:08.259509+0000","last_fullsized":"2024-04-26T22:00:08.259509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258732+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:00:08.258732+0000","last_peered":"2024-04-26T22:00:08.258732+0000","last_clean":"2024-04-26T22:00:08.258732+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:00:08.258732+0000","last_undegraded":"2024-04-26T22:00:08.258732+0000","last_fullsized":"2024-04-26T22:00:08.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259712+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:00:08.259712+0000","last_peered":"2024-04-26T22:00:08.259712+0000","last_clean":"2024-04-26T22:00:08.259712+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:00:08.259712+0000","last_undegraded":"2024-04-26T22:00:08.259712+0000","last_fullsized":"2024-04-26T22:00:08.259712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259096+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:00:08.259096+0000","last_peered":"2024-04-26T22:00:08.259096+0000","last_clean":"2024-04-26T22:00:08.259096+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:00:08.259096+0000","last_undegraded":"2024-04-26T22:00:08.259096+0000","last_fullsized":"2024-04-26T22:00:08.259096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809484+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:00:07.809484+0000","last_peered":"2024-04-26T22:00:07.809484+0000","last_clean":"2024-04-26T22:00:07.809484+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:00:07.809484+0000","last_undegraded":"2024-04-26T22:00:07.809484+0000","last_fullsized":"2024-04-26T22:00:07.809484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805110+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:00:07.805110+0000","last_peered":"2024-04-26T22:00:07.805110+0000","last_clean":"2024-04-26T22:00:07.805110+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:00:07.805110+0000","last_undegraded":"2024-04-26T22:00:07.805110+0000","last_fullsized":"2024-04-26T22:00:07.805110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259597+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:00:08.259597+0000","last_peered":"2024-04-26T22:00:08.259597+0000","last_clean":"2024-04-26T22:00:08.259597+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:00:08.259597+0000","last_undegraded":"2024-04-26T22:00:08.259597+0000","last_fullsized":"2024-04-26T22:00:08.259597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805162+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:00:07.805162+0000","last_peered":"2024-04-26T22:00:07.805162+0000","last_clean":"2024-04-26T22:00:07.805162+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:00:07.805162+0000","last_undegraded":"2024-04-26T22:00:07.805162+0000","last_fullsized":"2024-04-26T22:00:07.805162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809351+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:00:07.809351+0000","last_peered":"2024-04-26T22:00:07.809351+0000","last_clean":"2024-04-26T22:00:07.809351+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:00:07.809351+0000","last_undegraded":"2024-04-26T22:00:07.809351+0000","last_fullsized":"2024-04-26T22:00:07.809351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804634+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:00:07.804634+0000","last_peered":"2024-04-26T22:00:07.804634+0000","last_clean":"2024-04-26T22:00:07.804634+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:00:07.804634+0000","last_undegraded":"2024-04-26T22:00:07.804634+0000","last_fullsized":"2024-04-26T22:00:07.804634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260303+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:00:08.260303+0000","last_peered":"2024-04-26T22:00:08.260303+0000","last_clean":"2024-04-26T22:00:08.260303+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:00:08.260303+0000","last_undegraded":"2024-04-26T22:00:08.260303+0000","last_fullsized":"2024-04-26T22:00:08.260303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809413+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:00:07.809413+0000","last_peered":"2024-04-26T22:00:07.809413+0000","last_clean":"2024-04-26T22:00:07.809413+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:00:07.809413+0000","last_undegraded":"2024-04-26T22:00:07.809413+0000","last_fullsized":"2024-04-26T22:00:07.809413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806749+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:00:07.806749+0000","last_peered":"2024-04-26T22:00:07.806749+0000","last_clean":"2024-04-26T22:00:07.806749+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:00:07.806749+0000","last_undegraded":"2024-04-26T22:00:07.806749+0000","last_fullsized":"2024-04-26T22:00:07.806749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258952+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:00:08.258952+0000","last_peered":"2024-04-26T22:00:08.258952+0000","last_clean":"2024-04-26T22:00:08.258952+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:00:08.258952+0000","last_undegraded":"2024-04-26T22:00:08.258952+0000","last_fullsized":"2024-04-26T22:00:08.258952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260587+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:00:08.260587+0000","last_peered":"2024-04-26T22:00:08.260587+0000","last_clean":"2024-04-26T22:00:08.260587+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:00:08.260587+0000","last_undegraded":"2024-04-26T22:00:08.260587+0000","last_fullsized":"2024-04-26T22:00:08.260587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804985+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:00:07.804985+0000","last_peered":"2024-04-26T22:00:07.804985+0000","last_clean":"2024-04-26T22:00:07.804985+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:00:07.804985+0000","last_undegraded":"2024-04-26T22:00:07.804985+0000","last_fullsized":"2024-04-26T22:00:07.804985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258395+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:00:08.258395+0000","last_peered":"2024-04-26T22:00:08.258395+0000","last_clean":"2024-04-26T22:00:08.258395+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:00:08.258395+0000","last_undegraded":"2024-04-26T22:00:08.258395+0000","last_fullsized":"2024-04-26T22:00:08.258395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804633+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:00:07.804633+0000","last_peered":"2024-04-26T22:00:07.804633+0000","last_clean":"2024-04-26T22:00:07.804633+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:00:07.804633+0000","last_undegraded":"2024-04-26T22:00:07.804633+0000","last_fullsized":"2024-04-26T22:00:07.804633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260145+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:00:08.260145+0000","last_peered":"2024-04-26T22:00:08.260145+0000","last_clean":"2024-04-26T22:00:08.260145+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:00:08.260145+0000","last_undegraded":"2024-04-26T22:00:08.260145+0000","last_fullsized":"2024-04-26T22:00:08.260145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811517+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:00:07.811517+0000","last_peered":"2024-04-26T22:00:07.811517+0000","last_clean":"2024-04-26T22:00:07.811517+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:00:07.811517+0000","last_undegraded":"2024-04-26T22:00:07.811517+0000","last_fullsized":"2024-04-26T22:00:07.811517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810390+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:00:07.810390+0000","last_peered":"2024-04-26T22:00:07.810390+0000","last_clean":"2024-04-26T22:00:07.810390+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:00:07.810390+0000","last_undegraded":"2024-04-26T22:00:07.810390+0000","last_fullsized":"2024-04-26T22:00:07.810390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259149+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:00:08.259149+0000","last_peered":"2024-04-26T22:00:08.259149+0000","last_clean":"2024-04-26T22:00:08.259149+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:00:08.259149+0000","last_undegraded":"2024-04-26T22:00:08.259149+0000","last_fullsized":"2024-04-26T22:00:08.259149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259269+0000","last_change":"2024-04-26T21:59:57.651473+0000","last_active":"2024-04-26T22:00:08.259269+0000","last_peered":"2024-04-26T22:00:08.259269+0000","last_clean":"2024-04-26T22:00:08.259269+0000","last_became_active":"2024-04-26T21:59:57.651097+0000","last_became_peered":"2024-04-26T21:59:57.651097+0000","last_unstale":"2024-04-26T22:00:08.259269+0000","last_undegraded":"2024-04-26T22:00:08.259269+0000","last_fullsized":"2024-04-26T22:00:08.259269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804632+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:00:07.804632+0000","last_peered":"2024-04-26T22:00:07.804632+0000","last_clean":"2024-04-26T22:00:07.804632+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:00:07.804632+0000","last_undegraded":"2024-04-26T22:00:07.804632+0000","last_fullsized":"2024-04-26T22:00:07.804632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807126+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:00:07.807126+0000","last_peered":"2024-04-26T22:00:07.807126+0000","last_clean":"2024-04-26T22:00:07.807126+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:00:07.807126+0000","last_undegraded":"2024-04-26T22:00:07.807126+0000","last_fullsized":"2024-04-26T22:00:07.807126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260076+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:00:08.260076+0000","last_peered":"2024-04-26T22:00:08.260076+0000","last_clean":"2024-04-26T22:00:08.260076+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:00:08.260076+0000","last_undegraded":"2024-04-26T22:00:08.260076+0000","last_fullsized":"2024-04-26T22:00:08.260076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809033+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:00:07.809033+0000","last_peered":"2024-04-26T22:00:07.809033+0000","last_clean":"2024-04-26T22:00:07.809033+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:00:07.809033+0000","last_undegraded":"2024-04-26T22:00:07.809033+0000","last_fullsized":"2024-04-26T22:00:07.809033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809813+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:00:07.809813+0000","last_peered":"2024-04-26T22:00:07.809813+0000","last_clean":"2024-04-26T22:00:07.809813+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:00:07.809813+0000","last_undegraded":"2024-04-26T22:00:07.809813+0000","last_fullsized":"2024-04-26T22:00:07.809813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259172+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:00:08.259172+0000","last_peered":"2024-04-26T22:00:08.259172+0000","last_clean":"2024-04-26T22:00:08.259172+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:00:08.259172+0000","last_undegraded":"2024-04-26T22:00:08.259172+0000","last_fullsized":"2024-04-26T22:00:08.259172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259833+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:00:08.259833+0000","last_peered":"2024-04-26T22:00:08.259833+0000","last_clean":"2024-04-26T22:00:08.259833+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:00:08.259833+0000","last_undegraded":"2024-04-26T22:00:08.259833+0000","last_fullsized":"2024-04-26T22:00:08.259833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259553+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:00:08.259553+0000","last_peered":"2024-04-26T22:00:08.259553+0000","last_clean":"2024-04-26T22:00:08.259553+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:00:08.259553+0000","last_undegraded":"2024-04-26T22:00:08.259553+0000","last_fullsized":"2024-04-26T22:00:08.259553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259476+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:00:08.259476+0000","last_peered":"2024-04-26T22:00:08.259476+0000","last_clean":"2024-04-26T22:00:08.259476+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:00:08.259476+0000","last_undegraded":"2024-04-26T22:00:08.259476+0000","last_fullsized":"2024-04-26T22:00:08.259476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258944+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:00:08.258944+0000","last_peered":"2024-04-26T22:00:08.258944+0000","last_clean":"2024-04-26T22:00:08.258944+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:00:08.258944+0000","last_undegraded":"2024-04-26T22:00:08.258944+0000","last_fullsized":"2024-04-26T22:00:08.258944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258960+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:00:08.258960+0000","last_peered":"2024-04-26T22:00:08.258960+0000","last_clean":"2024-04-26T22:00:08.258960+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:00:08.258960+0000","last_undegraded":"2024-04-26T22:00:08.258960+0000","last_fullsized":"2024-04-26T22:00:08.258960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807277+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:00:07.807277+0000","last_peered":"2024-04-26T22:00:07.807277+0000","last_clean":"2024-04-26T22:00:07.807277+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:00:07.807277+0000","last_undegraded":"2024-04-26T22:00:07.807277+0000","last_fullsized":"2024-04-26T22:00:07.807277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260533+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:00:08.260533+0000","last_peered":"2024-04-26T22:00:08.260533+0000","last_clean":"2024-04-26T22:00:08.260533+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:00:08.260533+0000","last_undegraded":"2024-04-26T22:00:08.260533+0000","last_fullsized":"2024-04-26T22:00:08.260533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258763+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:00:08.258763+0000","last_peered":"2024-04-26T22:00:08.258763+0000","last_clean":"2024-04-26T22:00:08.258763+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:00:08.258763+0000","last_undegraded":"2024-04-26T22:00:08.258763+0000","last_fullsized":"2024-04-26T22:00:08.258763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259422+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:00:08.259422+0000","last_peered":"2024-04-26T22:00:08.259422+0000","last_clean":"2024-04-26T22:00:08.259422+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:00:08.259422+0000","last_undegraded":"2024-04-26T22:00:08.259422+0000","last_fullsized":"2024-04-26T22:00:08.259422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:29.054882+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:00:29.054882+0000","last_peered":"2024-04-26T22:00:29.054882+0000","last_clean":"2024-04-26T22:00:29.054882+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:00:29.054882+0000","last_undegraded":"2024-04-26T22:00:29.054882+0000","last_fullsized":"2024-04-26T22:00:29.054882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808687+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:00:07.808687+0000","last_peered":"2024-04-26T22:00:07.808687+0000","last_clean":"2024-04-26T22:00:07.808687+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:00:07.808687+0000","last_undegraded":"2024-04-26T22:00:07.808687+0000","last_fullsized":"2024-04-26T22:00:07.808687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809069+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:00:07.809069+0000","last_peered":"2024-04-26T22:00:07.809069+0000","last_clean":"2024-04-26T22:00:07.809069+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:00:07.809069+0000","last_undegraded":"2024-04-26T22:00:07.809069+0000","last_fullsized":"2024-04-26T22:00:07.809069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811208+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:00:07.811208+0000","last_peered":"2024-04-26T22:00:07.811208+0000","last_clean":"2024-04-26T22:00:07.811208+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:00:07.811208+0000","last_undegraded":"2024-04-26T22:00:07.811208+0000","last_fullsized":"2024-04-26T22:00:07.811208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809379+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:00:07.809379+0000","last_peered":"2024-04-26T22:00:07.809379+0000","last_clean":"2024-04-26T22:00:07.809379+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:00:07.809379+0000","last_undegraded":"2024-04-26T22:00:07.809379+0000","last_fullsized":"2024-04-26T22:00:07.809379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810817+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:00:07.810817+0000","last_peered":"2024-04-26T22:00:07.810817+0000","last_clean":"2024-04-26T22:00:07.810817+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:00:07.810817+0000","last_undegraded":"2024-04-26T22:00:07.810817+0000","last_fullsized":"2024-04-26T22:00:07.810817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809672+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:00:07.809672+0000","last_peered":"2024-04-26T22:00:07.809672+0000","last_clean":"2024-04-26T22:00:07.809672+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:00:07.809672+0000","last_undegraded":"2024-04-26T22:00:07.809672+0000","last_fullsized":"2024-04-26T22:00:07.809672+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806996+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:00:07.806996+0000","last_peered":"2024-04-26T22:00:07.806996+0000","last_clean":"2024-04-26T22:00:07.806996+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:00:07.806996+0000","last_undegraded":"2024-04-26T22:00:07.806996+0000","last_fullsized":"2024-04-26T22:00:07.806996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260337+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:00:08.260337+0000","last_peered":"2024-04-26T22:00:08.260337+0000","last_clean":"2024-04-26T22:00:08.260337+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:00:08.260337+0000","last_undegraded":"2024-04-26T22:00:08.260337+0000","last_fullsized":"2024-04-26T22:00:08.260337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258455+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:00:08.258455+0000","last_peered":"2024-04-26T22:00:08.258455+0000","last_clean":"2024-04-26T22:00:08.258455+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:00:08.258455+0000","last_undegraded":"2024-04-26T22:00:08.258455+0000","last_fullsized":"2024-04-26T22:00:08.258455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258795+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:00:08.258795+0000","last_peered":"2024-04-26T22:00:08.258795+0000","last_clean":"2024-04-26T22:00:08.258795+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:00:08.258795+0000","last_undegraded":"2024-04-26T22:00:08.258795+0000","last_fullsized":"2024-04-26T22:00:08.258795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260300+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:00:08.260300+0000","last_peered":"2024-04-26T22:00:08.260300+0000","last_clean":"2024-04-26T22:00:08.260300+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:00:08.260300+0000","last_undegraded":"2024-04-26T22:00:08.260300+0000","last_fullsized":"2024-04-26T22:00:08.260300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810156+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:00:07.810156+0000","last_peered":"2024-04-26T22:00:07.810156+0000","last_clean":"2024-04-26T22:00:07.810156+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:00:07.810156+0000","last_undegraded":"2024-04-26T22:00:07.810156+0000","last_fullsized":"2024-04-26T22:00:07.810156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258298+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:00:08.258298+0000","last_peered":"2024-04-26T22:00:08.258298+0000","last_clean":"2024-04-26T22:00:08.258298+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:00:08.258298+0000","last_undegraded":"2024-04-26T22:00:08.258298+0000","last_fullsized":"2024-04-26T22:00:08.258298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810045+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:00:07.810045+0000","last_peered":"2024-04-26T22:00:07.810045+0000","last_clean":"2024-04-26T22:00:07.810045+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:00:07.810045+0000","last_undegraded":"2024-04-26T22:00:07.810045+0000","last_fullsized":"2024-04-26T22:00:07.810045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806977+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:00:07.806977+0000","last_peered":"2024-04-26T22:00:07.806977+0000","last_clean":"2024-04-26T22:00:07.806977+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:00:07.806977+0000","last_undegraded":"2024-04-26T22:00:07.806977+0000","last_fullsized":"2024-04-26T22:00:07.806977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808696+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:00:07.808696+0000","last_peered":"2024-04-26T22:00:07.808696+0000","last_clean":"2024-04-26T22:00:07.808696+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:00:07.808696+0000","last_undegraded":"2024-04-26T22:00:07.808696+0000","last_fullsized":"2024-04-26T22:00:07.808696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258734+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:00:08.258734+0000","last_peered":"2024-04-26T22:00:08.258734+0000","last_clean":"2024-04-26T22:00:08.258734+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:00:08.258734+0000","last_undegraded":"2024-04-26T22:00:08.258734+0000","last_fullsized":"2024-04-26T22:00:08.258734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807031+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:00:07.807031+0000","last_peered":"2024-04-26T22:00:07.807031+0000","last_clean":"2024-04-26T22:00:07.807031+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:00:07.807031+0000","last_undegraded":"2024-04-26T22:00:07.807031+0000","last_fullsized":"2024-04-26T22:00:07.807031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810076+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:00:07.810076+0000","last_peered":"2024-04-26T22:00:07.810076+0000","last_clean":"2024-04-26T22:00:07.810076+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:00:07.810076+0000","last_undegraded":"2024-04-26T22:00:07.810076+0000","last_fullsized":"2024-04-26T22:00:07.810076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258591+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:00:08.258591+0000","last_peered":"2024-04-26T22:00:08.258591+0000","last_clean":"2024-04-26T22:00:08.258591+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:00:08.258591+0000","last_undegraded":"2024-04-26T22:00:08.258591+0000","last_fullsized":"2024-04-26T22:00:08.258591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258584+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:00:08.258584+0000","last_peered":"2024-04-26T22:00:08.258584+0000","last_clean":"2024-04-26T22:00:08.258584+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:00:08.258584+0000","last_undegraded":"2024-04-26T22:00:08.258584+0000","last_fullsized":"2024-04-26T22:00:08.258584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809306+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:00:07.809306+0000","last_peered":"2024-04-26T22:00:07.809306+0000","last_clean":"2024-04-26T22:00:07.809306+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:00:07.809306+0000","last_undegraded":"2024-04-26T22:00:07.809306+0000","last_fullsized":"2024-04-26T22:00:07.809306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809535+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:00:07.809535+0000","last_peered":"2024-04-26T22:00:07.809535+0000","last_clean":"2024-04-26T22:00:07.809535+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:00:07.809535+0000","last_undegraded":"2024-04-26T22:00:07.809535+0000","last_fullsized":"2024-04-26T22:00:07.809535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807197+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:00:07.807197+0000","last_peered":"2024-04-26T22:00:07.807197+0000","last_clean":"2024-04-26T22:00:07.807197+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:00:07.807197+0000","last_undegraded":"2024-04-26T22:00:07.807197+0000","last_fullsized":"2024-04-26T22:00:07.807197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804772+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:00:07.804772+0000","last_peered":"2024-04-26T22:00:07.804772+0000","last_clean":"2024-04-26T22:00:07.804772+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:00:07.804772+0000","last_undegraded":"2024-04-26T22:00:07.804772+0000","last_fullsized":"2024-04-26T22:00:07.804772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810747+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:00:07.810747+0000","last_peered":"2024-04-26T22:00:07.810747+0000","last_clean":"2024-04-26T22:00:07.810747+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:00:07.810747+0000","last_undegraded":"2024-04-26T22:00:07.810747+0000","last_fullsized":"2024-04-26T22:00:07.810747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258906+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:00:08.258906+0000","last_peered":"2024-04-26T22:00:08.258906+0000","last_clean":"2024-04-26T22:00:08.258906+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:00:08.258906+0000","last_undegraded":"2024-04-26T22:00:08.258906+0000","last_fullsized":"2024-04-26T22:00:08.258906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810890+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:00:07.810890+0000","last_peered":"2024-04-26T22:00:07.810890+0000","last_clean":"2024-04-26T22:00:07.810890+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:00:07.810890+0000","last_undegraded":"2024-04-26T22:00:07.810890+0000","last_fullsized":"2024-04-26T22:00:07.810890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259027+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:00:08.259027+0000","last_peered":"2024-04-26T22:00:08.259027+0000","last_clean":"2024-04-26T22:00:08.259027+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:00:08.259027+0000","last_undegraded":"2024-04-26T22:00:08.259027+0000","last_fullsized":"2024-04-26T22:00:08.259027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259239+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:00:08.259239+0000","last_peered":"2024-04-26T22:00:08.259239+0000","last_clean":"2024-04-26T22:00:08.259239+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:00:08.259239+0000","last_undegraded":"2024-04-26T22:00:08.259239+0000","last_fullsized":"2024-04-26T22:00:08.259239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809020+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:00:07.809020+0000","last_peered":"2024-04-26T22:00:07.809020+0000","last_clean":"2024-04-26T22:00:07.809020+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:00:07.809020+0000","last_undegraded":"2024-04-26T22:00:07.809020+0000","last_fullsized":"2024-04-26T22:00:07.809020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809227+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:00:07.809227+0000","last_peered":"2024-04-26T22:00:07.809227+0000","last_clean":"2024-04-26T22:00:07.809227+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:00:07.809227+0000","last_undegraded":"2024-04-26T22:00:07.809227+0000","last_fullsized":"2024-04-26T22:00:07.809227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804523+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:00:07.804523+0000","last_peered":"2024-04-26T22:00:07.804523+0000","last_clean":"2024-04-26T22:00:07.804523+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:00:07.804523+0000","last_undegraded":"2024-04-26T22:00:07.804523+0000","last_fullsized":"2024-04-26T22:00:07.804523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259903+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:00:08.259903+0000","last_peered":"2024-04-26T22:00:08.259903+0000","last_clean":"2024-04-26T22:00:08.259903+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:00:08.259903+0000","last_undegraded":"2024-04-26T22:00:08.259903+0000","last_fullsized":"2024-04-26T22:00:08.259903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259389+0000","last_change":"2024-04-26T21:59:57.654396+0000","last_active":"2024-04-26T22:00:08.259389+0000","last_peered":"2024-04-26T22:00:08.259389+0000","last_clean":"2024-04-26T22:00:08.259389+0000","last_became_active":"2024-04-26T21:59:57.654140+0000","last_became_peered":"2024-04-26T21:59:57.654140+0000","last_unstale":"2024-04-26T22:00:08.259389+0000","last_undegraded":"2024-04-26T22:00:08.259389+0000","last_fullsized":"2024-04-26T22:00:08.259389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809323+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:00:07.809323+0000","last_peered":"2024-04-26T22:00:07.809323+0000","last_clean":"2024-04-26T22:00:07.809323+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:00:07.809323+0000","last_undegraded":"2024-04-26T22:00:07.809323+0000","last_fullsized":"2024-04-26T22:00:07.809323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807134+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:00:07.807134+0000","last_peered":"2024-04-26T22:00:07.807134+0000","last_clean":"2024-04-26T22:00:07.807134+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:00:07.807134+0000","last_undegraded":"2024-04-26T22:00:07.807134+0000","last_fullsized":"2024-04-26T22:00:07.807134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258697+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:00:08.258697+0000","last_peered":"2024-04-26T22:00:08.258697+0000","last_clean":"2024-04-26T22:00:08.258697+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:00:08.258697+0000","last_undegraded":"2024-04-26T22:00:08.258697+0000","last_fullsized":"2024-04-26T22:00:08.258697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259157+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:00:08.259157+0000","last_peered":"2024-04-26T22:00:08.259157+0000","last_clean":"2024-04-26T22:00:08.259157+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:00:08.259157+0000","last_undegraded":"2024-04-26T22:00:08.259157+0000","last_fullsized":"2024-04-26T22:00:08.259157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810720+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:00:07.810720+0000","last_peered":"2024-04-26T22:00:07.810720+0000","last_clean":"2024-04-26T22:00:07.810720+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:00:07.810720+0000","last_undegraded":"2024-04-26T22:00:07.810720+0000","last_fullsized":"2024-04-26T22:00:07.810720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809360+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:00:07.809360+0000","last_peered":"2024-04-26T22:00:07.809360+0000","last_clean":"2024-04-26T22:00:07.809360+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:00:07.809360+0000","last_undegraded":"2024-04-26T22:00:07.809360+0000","last_fullsized":"2024-04-26T22:00:07.809360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258777+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:00:08.258777+0000","last_peered":"2024-04-26T22:00:08.258777+0000","last_clean":"2024-04-26T22:00:08.258777+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:00:08.258777+0000","last_undegraded":"2024-04-26T22:00:08.258777+0000","last_fullsized":"2024-04-26T22:00:08.258777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809835+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:00:07.809835+0000","last_peered":"2024-04-26T22:00:07.809835+0000","last_clean":"2024-04-26T22:00:07.809835+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:00:07.809835+0000","last_undegraded":"2024-04-26T22:00:07.809835+0000","last_fullsized":"2024-04-26T22:00:07.809835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809845+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:00:07.809845+0000","last_peered":"2024-04-26T22:00:07.809845+0000","last_clean":"2024-04-26T22:00:07.809845+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:00:07.809845+0000","last_undegraded":"2024-04-26T22:00:07.809845+0000","last_fullsized":"2024-04-26T22:00:07.809845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259204+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:00:08.259204+0000","last_peered":"2024-04-26T22:00:08.259204+0000","last_clean":"2024-04-26T22:00:08.259204+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:00:08.259204+0000","last_undegraded":"2024-04-26T22:00:08.259204+0000","last_fullsized":"2024-04-26T22:00:08.259204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259213+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:00:08.259213+0000","last_peered":"2024-04-26T22:00:08.259213+0000","last_clean":"2024-04-26T22:00:08.259213+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:00:08.259213+0000","last_undegraded":"2024-04-26T22:00:08.259213+0000","last_fullsized":"2024-04-26T22:00:08.259213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258628+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:00:08.258628+0000","last_peered":"2024-04-26T22:00:08.258628+0000","last_clean":"2024-04-26T22:00:08.258628+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:00:08.258628+0000","last_undegraded":"2024-04-26T22:00:08.258628+0000","last_fullsized":"2024-04-26T22:00:08.258628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260885+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:00:08.260885+0000","last_peered":"2024-04-26T22:00:08.260885+0000","last_clean":"2024-04-26T22:00:08.260885+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:00:08.260885+0000","last_undegraded":"2024-04-26T22:00:08.260885+0000","last_fullsized":"2024-04-26T22:00:08.260885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258481+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:00:08.258481+0000","last_peered":"2024-04-26T22:00:08.258481+0000","last_clean":"2024-04-26T22:00:08.258481+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:00:08.258481+0000","last_undegraded":"2024-04-26T22:00:08.258481+0000","last_fullsized":"2024-04-26T22:00:08.258481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805239+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:00:07.805239+0000","last_peered":"2024-04-26T22:00:07.805239+0000","last_clean":"2024-04-26T22:00:07.805239+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:00:07.805239+0000","last_undegraded":"2024-04-26T22:00:07.805239+0000","last_fullsized":"2024-04-26T22:00:07.805239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259356+0000","last_change":"2024-04-26T21:59:57.654894+0000","last_active":"2024-04-26T22:00:08.259356+0000","last_peered":"2024-04-26T22:00:08.259356+0000","last_clean":"2024-04-26T22:00:08.259356+0000","last_became_active":"2024-04-26T21:59:57.654582+0000","last_became_peered":"2024-04-26T21:59:57.654582+0000","last_unstale":"2024-04-26T22:00:08.259356+0000","last_undegraded":"2024-04-26T22:00:08.259356+0000","last_fullsized":"2024-04-26T22:00:08.259356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:02:57.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258970+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:00:08.258970+0000","last_peered":"2024-04-26T22:00:08.258970+0000","last_clean":"2024-04-26T22:00:08.258970+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:00:08.258970+0000","last_undegraded":"2024-04-26T22:00:08.258970+0000","last_fullsized":"2024-04-26T22:00:08.258970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.807041+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:00:07.807041+0000","last_peered":"2024-04-26T22:00:07.807041+0000","last_clean":"2024-04-26T22:00:07.807041+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:00:07.807041+0000","last_undegraded":"2024-04-26T22:00:07.807041+0000","last_fullsized":"2024-04-26T22:00:07.807041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810749+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:00:07.810749+0000","last_peered":"2024-04-26T22:00:07.810749+0000","last_clean":"2024-04-26T22:00:07.810749+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:00:07.810749+0000","last_undegraded":"2024-04-26T22:00:07.810749+0000","last_fullsized":"2024-04-26T22:00:07.810749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809319+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:00:07.809319+0000","last_peered":"2024-04-26T22:00:07.809319+0000","last_clean":"2024-04-26T22:00:07.809319+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:00:07.809319+0000","last_undegraded":"2024-04-26T22:00:07.809319+0000","last_fullsized":"2024-04-26T22:00:07.809319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810691+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:00:07.810691+0000","last_peered":"2024-04-26T22:00:07.810691+0000","last_clean":"2024-04-26T22:00:07.810691+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:00:07.810691+0000","last_undegraded":"2024-04-26T22:00:07.810691+0000","last_fullsized":"2024-04-26T22:00:07.810691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259755+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:00:08.259755+0000","last_peered":"2024-04-26T22:00:08.259755+0000","last_clean":"2024-04-26T22:00:08.259755+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:00:08.259755+0000","last_undegraded":"2024-04-26T22:00:08.259755+0000","last_fullsized":"2024-04-26T22:00:08.259755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258429+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:00:08.258429+0000","last_peered":"2024-04-26T22:00:08.258429+0000","last_clean":"2024-04-26T22:00:08.258429+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:00:08.258429+0000","last_undegraded":"2024-04-26T22:00:08.258429+0000","last_fullsized":"2024-04-26T22:00:08.258429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804869+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:00:07.804869+0000","last_peered":"2024-04-26T22:00:07.804869+0000","last_clean":"2024-04-26T22:00:07.804869+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:00:07.804869+0000","last_undegraded":"2024-04-26T22:00:07.804869+0000","last_fullsized":"2024-04-26T22:00:07.804869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806704+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:00:07.806704+0000","last_peered":"2024-04-26T22:00:07.806704+0000","last_clean":"2024-04-26T22:00:07.806704+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:00:07.806704+0000","last_undegraded":"2024-04-26T22:00:07.806704+0000","last_fullsized":"2024-04-26T22:00:07.806704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260884+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:00:08.260884+0000","last_peered":"2024-04-26T22:00:08.260884+0000","last_clean":"2024-04-26T22:00:08.260884+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:00:08.260884+0000","last_undegraded":"2024-04-26T22:00:08.260884+0000","last_fullsized":"2024-04-26T22:00:08.260884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809192+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:00:07.809192+0000","last_peered":"2024-04-26T22:00:07.809192+0000","last_clean":"2024-04-26T22:00:07.809192+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:00:07.809192+0000","last_undegraded":"2024-04-26T22:00:07.809192+0000","last_fullsized":"2024-04-26T22:00:07.809192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258522+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:00:08.258522+0000","last_peered":"2024-04-26T22:00:08.258522+0000","last_clean":"2024-04-26T22:00:08.258522+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:00:08.258522+0000","last_undegraded":"2024-04-26T22:00:08.258522+0000","last_fullsized":"2024-04-26T22:00:08.258522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811341+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:00:07.811341+0000","last_peered":"2024-04-26T22:00:07.811341+0000","last_clean":"2024-04-26T22:00:07.811341+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:00:07.811341+0000","last_undegraded":"2024-04-26T22:00:07.811341+0000","last_fullsized":"2024-04-26T22:00:07.811341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808778+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:00:07.808778+0000","last_peered":"2024-04-26T22:00:07.808778+0000","last_clean":"2024-04-26T22:00:07.808778+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:00:07.808778+0000","last_undegraded":"2024-04-26T22:00:07.808778+0000","last_fullsized":"2024-04-26T22:00:07.808778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260186+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:00:08.260186+0000","last_peered":"2024-04-26T22:00:08.260186+0000","last_clean":"2024-04-26T22:00:08.260186+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:00:08.260186+0000","last_undegraded":"2024-04-26T22:00:08.260186+0000","last_fullsized":"2024-04-26T22:00:08.260186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806699+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:00:07.806699+0000","last_peered":"2024-04-26T22:00:07.806699+0000","last_clean":"2024-04-26T22:00:07.806699+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:00:07.806699+0000","last_undegraded":"2024-04-26T22:00:07.806699+0000","last_fullsized":"2024-04-26T22:00:07.806699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259500+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:00:08.259500+0000","last_peered":"2024-04-26T22:00:08.259500+0000","last_clean":"2024-04-26T22:00:08.259500+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:00:08.259500+0000","last_undegraded":"2024-04-26T22:00:08.259500+0000","last_fullsized":"2024-04-26T22:00:08.259500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258695+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:00:08.258695+0000","last_peered":"2024-04-26T22:00:08.258695+0000","last_clean":"2024-04-26T22:00:08.258695+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:00:08.258695+0000","last_undegraded":"2024-04-26T22:00:08.258695+0000","last_fullsized":"2024-04-26T22:00:08.258695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258606+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:00:08.258606+0000","last_peered":"2024-04-26T22:00:08.258606+0000","last_clean":"2024-04-26T22:00:08.258606+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:00:08.258606+0000","last_undegraded":"2024-04-26T22:00:08.258606+0000","last_fullsized":"2024-04-26T22:00:08.258606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.811137+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:00:07.811137+0000","last_peered":"2024-04-26T22:00:07.811137+0000","last_clean":"2024-04-26T22:00:07.811137+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:00:07.811137+0000","last_undegraded":"2024-04-26T22:00:07.811137+0000","last_fullsized":"2024-04-26T22:00:07.811137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.804824+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:00:07.804824+0000","last_peered":"2024-04-26T22:00:07.804824+0000","last_clean":"2024-04-26T22:00:07.804824+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:00:07.804824+0000","last_undegraded":"2024-04-26T22:00:07.804824+0000","last_fullsized":"2024-04-26T22:00:07.804824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809375+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:00:07.809375+0000","last_peered":"2024-04-26T22:00:07.809375+0000","last_clean":"2024-04-26T22:00:07.809375+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:00:07.809375+0000","last_undegraded":"2024-04-26T22:00:07.809375+0000","last_fullsized":"2024-04-26T22:00:07.809375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259209+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:00:08.259209+0000","last_peered":"2024-04-26T22:00:08.259209+0000","last_clean":"2024-04-26T22:00:08.259209+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:00:08.259209+0000","last_undegraded":"2024-04-26T22:00:08.259209+0000","last_fullsized":"2024-04-26T22:00:08.259209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809804+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:00:07.809804+0000","last_peered":"2024-04-26T22:00:07.809804+0000","last_clean":"2024-04-26T22:00:07.809804+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:00:07.809804+0000","last_undegraded":"2024-04-26T22:00:07.809804+0000","last_fullsized":"2024-04-26T22:00:07.809804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258541+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:00:08.258541+0000","last_peered":"2024-04-26T22:00:08.258541+0000","last_clean":"2024-04-26T22:00:08.258541+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:00:08.258541+0000","last_undegraded":"2024-04-26T22:00:08.258541+0000","last_fullsized":"2024-04-26T22:00:08.258541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260489+0000","last_change":"2024-04-26T21:59:57.655888+0000","last_active":"2024-04-26T22:00:08.260489+0000","last_peered":"2024-04-26T22:00:08.260489+0000","last_clean":"2024-04-26T22:00:08.260489+0000","last_became_active":"2024-04-26T21:59:57.655580+0000","last_became_peered":"2024-04-26T21:59:57.655580+0000","last_unstale":"2024-04-26T22:00:08.260489+0000","last_undegraded":"2024-04-26T22:00:08.260489+0000","last_fullsized":"2024-04-26T22:00:08.260489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:37.332047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259075+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:00:08.259075+0000","last_peered":"2024-04-26T22:00:08.259075+0000","last_clean":"2024-04-26T22:00:08.259075+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:00:08.259075+0000","last_undegraded":"2024-04-26T22:00:08.259075+0000","last_fullsized":"2024-04-26T22:00:08.259075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258434+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:00:08.258434+0000","last_peered":"2024-04-26T22:00:08.258434+0000","last_clean":"2024-04-26T22:00:08.258434+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:00:08.258434+0000","last_undegraded":"2024-04-26T22:00:08.258434+0000","last_fullsized":"2024-04-26T22:00:08.258434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.810023+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:00:07.810023+0000","last_peered":"2024-04-26T22:00:07.810023+0000","last_clean":"2024-04-26T22:00:07.810023+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:00:07.810023+0000","last_undegraded":"2024-04-26T22:00:07.810023+0000","last_fullsized":"2024-04-26T22:00:07.810023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259045+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:00:08.259045+0000","last_peered":"2024-04-26T22:00:08.259045+0000","last_clean":"2024-04-26T22:00:08.259045+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:00:08.259045+0000","last_undegraded":"2024-04-26T22:00:08.259045+0000","last_fullsized":"2024-04-26T22:00:08.259045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.808947+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:00:07.808947+0000","last_peered":"2024-04-26T22:00:07.808947+0000","last_clean":"2024-04-26T22:00:07.808947+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:00:07.808947+0000","last_undegraded":"2024-04-26T22:00:07.808947+0000","last_fullsized":"2024-04-26T22:00:07.808947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.259259+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:00:08.259259+0000","last_peered":"2024-04-26T22:00:08.259259+0000","last_clean":"2024-04-26T22:00:08.259259+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:00:08.259259+0000","last_undegraded":"2024-04-26T22:00:08.259259+0000","last_fullsized":"2024-04-26T22:00:08.259259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260479+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:00:08.260479+0000","last_peered":"2024-04-26T22:00:08.260479+0000","last_clean":"2024-04-26T22:00:08.260479+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:00:08.260479+0000","last_undegraded":"2024-04-26T22:00:08.260479+0000","last_fullsized":"2024-04-26T22:00:08.260479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258807+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:00:08.258807+0000","last_peered":"2024-04-26T22:00:08.258807+0000","last_clean":"2024-04-26T22:00:08.258807+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:00:08.258807+0000","last_undegraded":"2024-04-26T22:00:08.258807+0000","last_fullsized":"2024-04-26T22:00:08.258807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258509+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:00:08.258509+0000","last_peered":"2024-04-26T22:00:08.258509+0000","last_clean":"2024-04-26T22:00:08.258509+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:00:08.258509+0000","last_undegraded":"2024-04-26T22:00:08.258509+0000","last_fullsized":"2024-04-26T22:00:08.258509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260065+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:00:08.260065+0000","last_peered":"2024-04-26T22:00:08.260065+0000","last_clean":"2024-04-26T22:00:08.260065+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:00:08.260065+0000","last_undegraded":"2024-04-26T22:00:08.260065+0000","last_fullsized":"2024-04-26T22:00:08.260065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.805072+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:00:07.805072+0000","last_peered":"2024-04-26T22:00:07.805072+0000","last_clean":"2024-04-26T22:00:07.805072+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:00:07.805072+0000","last_undegraded":"2024-04-26T22:00:07.805072+0000","last_fullsized":"2024-04-26T22:00:07.805072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.806889+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:00:07.806889+0000","last_peered":"2024-04-26T22:00:07.806889+0000","last_clean":"2024-04-26T22:00:07.806889+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:00:07.806889+0000","last_undegraded":"2024-04-26T22:00:07.806889+0000","last_fullsized":"2024-04-26T22:00:07.806889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258594+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:00:08.258594+0000","last_peered":"2024-04-26T22:00:08.258594+0000","last_clean":"2024-04-26T22:00:08.258594+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:00:08.258594+0000","last_undegraded":"2024-04-26T22:00:08.258594+0000","last_fullsized":"2024-04-26T22:00:08.258594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:07.809080+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:00:07.809080+0000","last_peered":"2024-04-26T22:00:07.809080+0000","last_clean":"2024-04-26T22:00:07.809080+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:00:07.809080+0000","last_undegraded":"2024-04-26T22:00:07.809080+0000","last_fullsized":"2024-04-26T22:00:07.809080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.258893+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:00:08.258893+0000","last_peered":"2024-04-26T22:00:08.258893+0000","last_clean":"2024-04-26T22:00:08.258893+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:00:08.258893+0000","last_undegraded":"2024-04-26T22:00:08.258893+0000","last_fullsized":"2024-04-26T22:00:08.258893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T22:00:08.260393+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:00:08.260393+0000","last_peered":"2024-04-26T22:00:08.260393+0000","last_clean":"2024-04-26T22:00:08.260393+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:00:08.260393+0000","last_undegraded":"2024-04-26T22:00:08.260393+0000","last_fullsized":"2024-04-26T22:00:08.260393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":102763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:00:36.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:36.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.622+0000 7fca50823640 1 Processor -- start 2024-04-26T22:00:36.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.623+0000 7fca50823640 1 -- start start 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.623+0000 7fca50823640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 0x7fca4c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.623+0000 7fca50823640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4c156c10 con 0x7fca4c0c5050 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.623+0000 7fca50823640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4c0c5570 con 0x7fca4c0c2b80 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.623+0000 7fca50823640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c15a8b0 con 0x7fca4c13feb0 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fca4c0c5050 0x7fca4c15a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39294/0 (socket says 172.21.15.135:39294) 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 -- 172.21.15.135:0/1729884196 learned_addr learned my addr 172.21.15.135:0/1729884196 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4a59c640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 0x7fca4c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595289182 0 0) 0x7fca4c156c10 con 0x7fca4c0c5050 2024-04-26T22:00:36.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca34003520 con 0x7fca4c0c5050 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2748265496 0 0) 0x7fca34003520 con 0x7fca4c0c5050 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 msgr2=0x7fca4c1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 0x7fca4c1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 >> v1:172.21.15.139:6789/0 conn(0x7fca4c0c2b80 legacy=0x7fca4c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0c5850 con 0x7fca4c0c5050 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fca400019f0 con 0x7fca4c0c5050 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 >> v1:172.21.15.135:6789/0 conn(0x7fca4c0c5050 legacy=0x7fca4c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 0x7fca340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca34003f60 0x7fca34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34004d40 0x7fca340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c15a8b0 con 0x7fca34003f60 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca3400bb50 con 0x7fca34004d40 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fca3400bd50 con 0x7fca4c13feb0 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca34003f60 0x7fca34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4a59c640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 0x7fca340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca4ad9d640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34004d40 0x7fca340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 -- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 msgr2=0x7fca340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c13feb0 0x7fca340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 -- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34004d40 msgr2=0x7fca340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.624+0000 7fca49d9b640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34004d40 0x7fca340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.625+0000 7fca49d9b640 1 -- 172.21.15.135:0/1729884196 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca34003520 con 0x7fca34003f60 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.625+0000 7fca49d9b640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca34003f60 0x7fca34004800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca4000a860 tx=0x7fca34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1855a8b55e17ff94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.625+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4000a480 con 0x7fca34003f60 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.625+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca4002f070 con 0x7fca34003f60 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.625+0000 7fca4959a640 1 -- 172.21.15.135:0/1729884196 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4002bce0 con 0x7fca34003f60 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca50823640 1 -- 172.21.15.135:0/1729884196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca34003f60 msgr2=0x7fca34004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca50823640 1 --2- 172.21.15.135:0/1729884196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca34003f60 0x7fca34004800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca4000a860 tx=0x7fca34005830 comp rx=0 tx=0).stop 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca4ad9d640 1 -- 172.21.15.135:0/1729884196 reap_dead start 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca50823640 1 -- 172.21.15.135:0/1729884196 shutdown_connections 2024-04-26T22:00:36.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca50823640 1 -- 172.21.15.135:0/1729884196 >> 172.21.15.135:0/1729884196 conn(0x7fca4c0c5ce0 msgr2=0x7fca4c0c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:36.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca50823640 1 -- 172.21.15.135:0/1729884196 shutdown_connections 2024-04-26T22:00:36.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.626+0000 7fca50823640 1 -- 172.21.15.135:0/1729884196 wait complete. 2024-04-26T22:00:36.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.627+0000 7fca50823640 1 Processor -- start 2024-04-26T22:00:36.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.627+0000 7fca50823640 1 -- start start 2024-04-26T22:00:36.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca50823640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34003f60 0x7fca4c073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca50823640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c1565f0 0x7fca4c13bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca50823640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 0x7fca4c13c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca50823640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c06da30 con 0x7fca4c13ce90 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca50823640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c157240 con 0x7fca34003f60 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca50823640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c06dbb0 con 0x7fca4c1565f0 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 0x7fca4c13c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 0x7fca4c13c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41208/0 (socket says 172.21.15.135:41208) 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 -- 172.21.15.135:0/3651154918 learned_addr learned my addr 172.21.15.135:0/3651154918 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4a59c640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34003f60 0x7fca4c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca49d9b640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c1565f0 0x7fca4c13bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 -- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c1565f0 msgr2=0x7fca4c13bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c1565f0 0x7fca4c13bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 -- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34003f60 msgr2=0x7fca4c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34003f60 0x7fca4c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 -- 172.21.15.135:0/3651154918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c1f09f0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca49d9b640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c1565f0 0x7fca4c13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca4ad9d640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 0x7fca4c13c1d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca4c06e650 tx=0x7fca3c00f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.628+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0052e0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.629+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c018070 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.629+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c1555b0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.629+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003830 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.629+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1f0cd0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.630+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c0c34d0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.630+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca3c0039d0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.630+0000 7fca3affd640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca2c06c8f0 0x7fca2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.630+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7fca3c08b8f0 con 0x7fca4c13ce90 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.630+0000 7fca4a59c640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca2c06c8f0 0x7fca2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.631+0000 7fca4a59c640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca2c06c8f0 0x7fca2c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca44009bc0 tx=0x7fca44009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.635+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca4c0c34d0 con 0x7fca4c13ce90 2024-04-26T22:00:36.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.811+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca4c06d7e0 con 0x7fca4c13ce90 2024-04-26T22:00:36.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.813+0000 7fca3affd640 1 -- 172.21.15.135:0/3651154918 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+20391 (secure 0 0 0) 0x7fca4c06d7e0 con 0x7fca4c13ce90 2024-04-26T22:00:36.813 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:36.813 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:07.800851+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca2c06c8f0 msgr2=0x7fca2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca2c06c8f0 0x7fca2c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca44009bc0 tx=0x7fca44009190 comp rx=0 tx=0).stop 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 msgr2=0x7fca4c13c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 0x7fca4c13c1d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca4c06e650 tx=0x7fca3c00f8a0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 shutdown_connections 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca2c06c8f0 0x7fca2c06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca4c13ce90 0x7fca4c13c1d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca4c1565f0 0x7fca4c13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 --2- 172.21.15.135:0/3651154918 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca34003f60 0x7fca4c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 >> 172.21.15.135:0/3651154918 conn(0x7fca4c0c5ce0 msgr2=0x7fca4c1575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 shutdown_connections 2024-04-26T22:00:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.815+0000 7fca50823640 1 -- 172.21.15.135:0/3651154918 wait complete. 2024-04-26T22:00:36.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:36.826 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:36.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c59869640 1 Processor -- start 2024-04-26T22:00:36.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c59869640 1 -- start start 2024-04-26T22:00:36.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c59869640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5c54156bb0 0x7f5c5415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c59869640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540c8df0 con 0x7f5c540ca5a0 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c59869640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c54154540 con 0x7f5c54154160 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c59869640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5415a6e0 con 0x7f5c54156bb0 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.926+0000 7f5c537fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5c54156bb0 0x7f5c5415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c527fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5c540ca5a0 0x7f5c540c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39308/0 (socket says 172.21.15.135:39308) 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c537fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5c54156bb0 0x7f5c5415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41224/0 (socket says 172.21.15.135:41224) 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c527fc640 1 -- 172.21.15.135:0/2777710315 learned_addr learned my addr 172.21.15.135:0/2777710315 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c537fe640 1 -- 172.21.15.135:0/2777710315 >> v1:172.21.15.139:6789/0 conn(0x7f5c54154160 legacy=0x7f5c540c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c537fe640 1 -- 172.21.15.135:0/2777710315 >> v1:172.21.15.135:6789/0 conn(0x7f5c540ca5a0 legacy=0x7f5c540c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c537fe640 1 -- 172.21.15.135:0/2777710315 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5415a910 con 0x7f5c54156bb0 2024-04-26T22:00:36.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c51ffb640 1 -- 172.21.15.135:0/2777710315 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716410496 0 0) 0x7f5c48001110 con 0x7f5c540ca5a0 2024-04-26T22:00:36.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.927+0000 7f5c537fe640 1 --2- 172.21.15.135:0/2777710315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54156bb0 0x7f5c5415a1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c540af040 tx=0x7f5c4c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6359996811683e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c51ffb640 1 -- 172.21.15.135:0/2777710315 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c02ae60 con 0x7f5c54156bb0 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c51ffb640 1 -- 172.21.15.135:0/2777710315 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c007740 con 0x7f5c54156bb0 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c51ffb640 1 -- 172.21.15.135:0/2777710315 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c007b10 con 0x7f5c54156bb0 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 -- 172.21.15.135:0/2777710315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54156bb0 msgr2=0x7f5c5415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 --2- 172.21.15.135:0/2777710315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54156bb0 0x7f5c5415a1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c540af040 tx=0x7f5c4c02a680 comp rx=0 tx=0).stop 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 -- 172.21.15.135:0/2777710315 shutdown_connections 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 --2- 172.21.15.135:0/2777710315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54156bb0 0x7f5c5415a1a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 -- 172.21.15.135:0/2777710315 >> 172.21.15.135:0/2777710315 conn(0x7f5c540ae010 msgr2=0x7f5c540ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 -- 172.21.15.135:0/2777710315 shutdown_connections 2024-04-26T22:00:36.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.928+0000 7f5c59869640 1 -- 172.21.15.135:0/2777710315 wait complete. 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.929+0000 7f5c59869640 1 Processor -- start 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.929+0000 7f5c59869640 1 -- start start 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c59869640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ca5a0 0x7f5c541ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c59869640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 0x7f5c541ed520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c59869640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c54156bb0 0x7f5c5414b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c59869640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5415aed0 con 0x7f5c54154160 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c59869640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540c9100 con 0x7f5c540ca5a0 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c59869640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c540c7510 con 0x7f5c54156bb0 2024-04-26T22:00:36.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c527fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 0x7f5c541ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c527fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 0x7f5c541ed520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41240/0 (socket says 172.21.15.135:41240) 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c527fc640 1 -- 172.21.15.135:0/945659642 learned_addr learned my addr 172.21.15.135:0/945659642 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c52ffd640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ca5a0 0x7f5c541ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.930+0000 7f5c537fe640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c54156bb0 0x7f5c5414b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.931+0000 7f5c527fc640 1 -- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c54156bb0 msgr2=0x7f5c5414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.931+0000 7f5c527fc640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c54156bb0 0x7f5c5414b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.931+0000 7f5c527fc640 1 -- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ca5a0 msgr2=0x7f5c541ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.931+0000 7f5c527fc640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ca5a0 0x7f5c541ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.931+0000 7f5c527fc640 1 -- 172.21.15.135:0/945659642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54157fa0 con 0x7f5c54154160 2024-04-26T22:00:36.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.931+0000 7f5c527fc640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 0x7f5c541ed520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c780 tx=0x7f5c4800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.935+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48018070 con 0x7f5c54154160 2024-04-26T22:00:36.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.935+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c480072d0 con 0x7f5c54154160 2024-04-26T22:00:36.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.935+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c480047e0 con 0x7f5c54154160 2024-04-26T22:00:36.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.935+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c54158280 con 0x7f5c54154160 2024-04-26T22:00:36.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.936+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c54158840 con 0x7f5c54154160 2024-04-26T22:00:36.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.937+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c54076650 con 0x7f5c54154160 2024-04-26T22:00:36.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.937+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c48004980 con 0x7f5c54154160 2024-04-26T22:00:36.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.937+0000 7f5c3bfff640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c4006c8f0 0x7f5c4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:36.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.937+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f5c4808b250 con 0x7f5c54154160 2024-04-26T22:00:36.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.937+0000 7f5c52ffd640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c4006c8f0 0x7f5c4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:36.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.938+0000 7f5c52ffd640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c4006c8f0 0x7f5c4006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004f60 tx=0x7f5c44007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:36.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:36.941+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c540c8df0 con 0x7f5c54154160 2024-04-26T22:00:37.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.121+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5c541579e0 con 0x7f5c54154160 2024-04-26T22:00:37.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.122+0000 7f5c3bfff640 1 -- 172.21.15.135:0/945659642 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f5c48058c70 con 0x7f5c54154160 2024-04-26T22:00:37.122 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c4006c8f0 msgr2=0x7f5c4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c4006c8f0 0x7f5c4006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004f60 tx=0x7f5c44007e30 comp rx=0 tx=0).stop 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 msgr2=0x7f5c541ed520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 0x7f5c541ed520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c780 tx=0x7f5c4800cc40 comp rx=0 tx=0).stop 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 shutdown_connections 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c4006c8f0 0x7f5c4006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c54156bb0 0x7f5c5414b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c54154160 0x7f5c541ed520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 --2- 172.21.15.135:0/945659642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ca5a0 0x7f5c541ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 >> 172.21.15.135:0/945659642 conn(0x7f5c540ae010 msgr2=0x7f5c540c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 shutdown_connections 2024-04-26T22:00:37.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.124+0000 7f5c59869640 1 -- 172.21.15.135:0/945659642 wait complete. 2024-04-26T22:00:37.135 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:37.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.234+0000 7f82798da640 1 Processor -- start 2024-04-26T22:00:37.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82798da640 1 -- start start 2024-04-26T22:00:37.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82798da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82740ca5a0 0x7f8274156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82798da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f827415a1a0 0x7f82740c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82798da640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82740c8df0 con 0x7f8274154160 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82798da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f82740c9100 con 0x7f82740ca5a0 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82798da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8274154540 con 0x7f827415a1a0 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82727fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82740ca5a0 0x7f8274156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82727fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82740ca5a0 0x7f8274156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41242/0 (socket says 172.21.15.135:41242) 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.235+0000 7f82727fc640 1 -- 172.21.15.135:0/2777487837 learned_addr learned my addr 172.21.15.135:0/2777487837 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.236+0000 7f82737fe640 1 --2- 172.21.15.135:0/2777487837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f827415a1a0 0x7f82740c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.236+0000 7f82727fc640 1 -- 172.21.15.135:0/2777487837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f827415a1a0 msgr2=0x7f82740c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.236+0000 7f82727fc640 1 --2- 172.21.15.135:0/2777487837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f827415a1a0 0x7f82740c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.236+0000 7f82727fc640 1 -- 172.21.15.135:0/2777487837 >> v1:172.21.15.139:6789/0 conn(0x7f8274154160 legacy=0x7f82740c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.236+0000 7f82727fc640 1 -- 172.21.15.135:0/2777487837 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827415a910 con 0x7f82740ca5a0 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.236+0000 7f82727fc640 1 --2- 172.21.15.135:0/2777487837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82740ca5a0 0x7f8274156bb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8268009870 tx=0x7f826802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdca3e375a83c46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f8271ffb640 1 -- 172.21.15.135:0/2777487837 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268037070 con 0x7f82740ca5a0 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f8271ffb640 1 -- 172.21.15.135:0/2777487837 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8268032440 con 0x7f82740ca5a0 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f8271ffb640 1 -- 172.21.15.135:0/2777487837 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268032810 con 0x7f82740ca5a0 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 -- 172.21.15.135:0/2777487837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82740ca5a0 msgr2=0x7f8274156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 --2- 172.21.15.135:0/2777487837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82740ca5a0 0x7f8274156bb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8268009870 tx=0x7f826802a210 comp rx=0 tx=0).stop 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 -- 172.21.15.135:0/2777487837 shutdown_connections 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 --2- 172.21.15.135:0/2777487837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f827415a1a0 0x7f82740c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 --2- 172.21.15.135:0/2777487837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82740ca5a0 0x7f8274156bb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 -- 172.21.15.135:0/2777487837 >> 172.21.15.135:0/2777487837 conn(0x7f82740ae010 msgr2=0x7f82740ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 -- 172.21.15.135:0/2777487837 shutdown_connections 2024-04-26T22:00:37.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.237+0000 7f82798da640 1 -- 172.21.15.135:0/2777487837 wait complete. 2024-04-26T22:00:37.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.238+0000 7f82798da640 1 Processor -- start 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.238+0000 7f82798da640 1 -- start start 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82798da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82740ca5a0 0x7f82741493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82798da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8274154160 0x7f8274158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82798da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 0x7f8274158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82798da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f827415a5f0 con 0x7f827415a1a0 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82798da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f827415aed0 con 0x7f8274154160 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82798da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f82740c7510 con 0x7f82740ca5a0 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 0x7f8274158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 0x7f8274158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41254/0 (socket says 172.21.15.135:41254) 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 -- 172.21.15.135:0/1460146454 learned_addr learned my addr 172.21.15.135:0/1460146454 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f8272ffd640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82740ca5a0 0x7f82741493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82727fc640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8274154160 0x7f8274158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 -- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82740ca5a0 msgr2=0x7f82741493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82740ca5a0 0x7f82741493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 -- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8274154160 msgr2=0x7f8274158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8274154160 0x7f8274158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.239+0000 7f82737fe640 1 -- 172.21.15.135:0/1460146454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82741477e0 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.240+0000 7f82737fe640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 0x7f8274158c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c550 tx=0x7f826c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.240+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c015070 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.240+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c004220 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.240+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827414b070 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.240+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c003610 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.240+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827414b630 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.241+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274076650 con 0x7f827415a1a0 2024-04-26T22:00:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.241+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f826c0037b0 con 0x7f827415a1a0 2024-04-26T22:00:37.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.242+0000 7f825bfff640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f826006c920 0x7f826006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.242+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f826c08baa0 con 0x7f827415a1a0 2024-04-26T22:00:37.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.242+0000 7f8272ffd640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f826006c920 0x7f826006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.242+0000 7f8272ffd640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f826006c920 0x7f826006ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8274069930 tx=0x7f8264003d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.246+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f826c0595c0 con 0x7f827415a1a0 2024-04-26T22:00:37.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.418+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f82740507a0 con 0x7f827415a1a0 2024-04-26T22:00:37.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.419+0000 7f825bfff640 1 -- 172.21.15.135:0/1460146454 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) ==== 99+0+11 (secure 0 0 0) 0x7f826c05d8c0 con 0x7f827415a1a0 2024-04-26T22:00:37.419 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:37.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f826006c920 msgr2=0x7f826006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f826006c920 0x7f826006ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8274069930 tx=0x7f8264003d90 comp rx=0 tx=0).stop 2024-04-26T22:00:37.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 msgr2=0x7f8274158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 0x7f8274158c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c550 tx=0x7f826c00ca10 comp rx=0 tx=0).stop 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 shutdown_connections 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f826006c920 0x7f826006ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827415a1a0 0x7f8274158c40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8274154160 0x7f8274158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 --2- 172.21.15.135:0/1460146454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82740ca5a0 0x7f82741493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 >> 172.21.15.135:0/1460146454 conn(0x7f82740ae010 msgr2=0x7f827414eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 shutdown_connections 2024-04-26T22:00:37.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.421+0000 7f82798da640 1 -- 172.21.15.135:0/1460146454 wait complete. 2024-04-26T22:00:37.432 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:37.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.532+0000 7f48fe8bd640 1 Processor -- start 2024-04-26T22:00:37.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.532+0000 7f48fe8bd640 1 -- start start 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48fe8bd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48f80c3060 0x7f48f80720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48fe8bd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f8154ba0 con 0x7f48f80c40e0 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48fe8bd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f8154e80 con 0x7f48f80c46e0 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48fe8bd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f48f8155160 con 0x7f48f80c3060 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48f80c3060 0x7f48f80720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f77fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f48f80c40e0 0x7f48f8151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39322/0 (socket says 172.21.15.135:39322) 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f77fe640 1 -- 172.21.15.135:0/1138289283 learned_addr learned my addr 172.21.15.135:0/1138289283 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f7fff640 1 --2- 172.21.15.135:0/1138289283 >> v2:172.21.15.135:3300/0 conn(0x7f48f80c3060 0x7f48f80720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41260/0 (socket says 172.21.15.135:41260) 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f7fff640 1 -- 172.21.15.135:0/1138289283 >> v1:172.21.15.139:6789/0 conn(0x7f48f80c46e0 legacy=0x7f48f80c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f7fff640 1 -- 172.21.15.135:0/1138289283 >> v1:172.21.15.135:6789/0 conn(0x7f48f80c40e0 legacy=0x7f48f8151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f7fff640 1 -- 172.21.15.135:0/1138289283 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f8155440 con 0x7f48f80c3060 2024-04-26T22:00:37.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.533+0000 7f48f6ffd640 1 -- 172.21.15.135:0/1138289283 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865508170 0 0) 0x7f48f8154ba0 con 0x7f48f80c40e0 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48f7fff640 1 --2- 172.21.15.135:0/1138289283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f80720a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097a0 tx=0x7f48e802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5779f9f8b42e0d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48f6ffd640 1 -- 172.21.15.135:0/1138289283 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8037070 con 0x7f48f80c3060 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48f6ffd640 1 -- 172.21.15.135:0/1138289283 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e802ae50 con 0x7f48f80c3060 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48f6ffd640 1 -- 172.21.15.135:0/1138289283 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8032660 con 0x7f48f80c3060 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 -- 172.21.15.135:0/1138289283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 msgr2=0x7f48f80720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/1138289283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f80720a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097a0 tx=0x7f48e802a140 comp rx=0 tx=0).stop 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 -- 172.21.15.135:0/1138289283 shutdown_connections 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/1138289283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f80720a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 -- 172.21.15.135:0/1138289283 >> 172.21.15.135:0/1138289283 conn(0x7f48f806c310 msgr2=0x7f48f806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 -- 172.21.15.135:0/1138289283 shutdown_connections 2024-04-26T22:00:37.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.534+0000 7f48fe8bd640 1 -- 172.21.15.135:0/1138289283 wait complete. 2024-04-26T22:00:37.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.535+0000 7f48fe8bd640 1 Processor -- start 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 -- start start 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48f80c40e0 0x7f48f81ede80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48f80c46e0 0x7f48f81ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80c3aa0 con 0x7f48f80c3060 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80c5280 con 0x7f48f80c40e0 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48fe8bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f48f80c3440 con 0x7f48f80c46e0 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48f7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48f7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f81584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41272/0 (socket says 172.21.15.135:41272) 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.536+0000 7f48f7fff640 1 -- 172.21.15.135:0/4029356080 learned_addr learned my addr 172.21.15.135:0/4029356080 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f77fe640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48f80c40e0 0x7f48f81ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48fce5a640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48f80c46e0 0x7f48f81ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f7fff640 1 -- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48f80c46e0 msgr2=0x7f48f81ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f7fff640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48f80c46e0 0x7f48f81ee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f7fff640 1 -- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48f80c40e0 msgr2=0x7f48f81ede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f7fff640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48f80c40e0 0x7f48f81ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f7fff640 1 -- 172.21.15.135:0/4029356080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f81472c0 con 0x7f48f80c3060 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f7fff640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f81584a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097a0 tx=0x7f48e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8042070 con 0x7f48f80c3060 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e803e670 con 0x7f48f80c3060 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8037040 con 0x7f48f80c3060 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f81589e0 con 0x7f48f80c3060 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.537+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f81561d0 con 0x7f48f80c3060 2024-04-26T22:00:37.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.538+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f80507a0 con 0x7f48f80c3060 2024-04-26T22:00:37.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.538+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48e8045070 con 0x7f48f80c3060 2024-04-26T22:00:37.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.539+0000 7f48f4ff9640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f48d806c8f0 0x7f48d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.539+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f48e80ac640 con 0x7f48f80c3060 2024-04-26T22:00:37.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.539+0000 7f48f77fe640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f48d806c8f0 0x7f48d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.539+0000 7f48f77fe640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f48d806c8f0 0x7f48d806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48e0007fd0 tx=0x7f48e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.543+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48e807a7d0 con 0x7f48f80c3060 2024-04-26T22:00:37.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.722+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f48f8069550 con 0x7f48f80c3060 2024-04-26T22:00:37.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.722+0000 7f48f4ff9640 1 -- 172.21.15.135:0/4029356080 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) ==== 95+0+11 (secure 0 0 0) 0x7f48e807a0d0 con 0x7f48f80c3060 2024-04-26T22:00:37.723 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f48d806c8f0 msgr2=0x7f48d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f48d806c8f0 0x7f48d806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48e0007fd0 tx=0x7f48e0009040 comp rx=0 tx=0).stop 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 msgr2=0x7f48f81584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f81584a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097a0 tx=0x7f48e80046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 shutdown_connections 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f48d806c8f0 0x7f48d806edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48f80c46e0 0x7f48f81ee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48f80c40e0 0x7f48f81ede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 --2- 172.21.15.135:0/4029356080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f80c3060 0x7f48f81584a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 >> 172.21.15.135:0/4029356080 conn(0x7f48f806c310 msgr2=0x7f48f8151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.724+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 shutdown_connections 2024-04-26T22:00:37.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.725+0000 7f48fe8bd640 1 -- 172.21.15.135:0/4029356080 wait complete. 2024-04-26T22:00:37.735 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:37.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.837+0000 7f0ac7d22640 1 Processor -- start 2024-04-26T22:00:37.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac7d22640 1 -- start start 2024-04-26T22:00:37.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac7d22640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0ac0154160 0x7f0ac0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac7d22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac7d22640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac00c8df0 con 0x7f0ac00ca5a0 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac7d22640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac00c9100 con 0x7f0ac0154160 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac7d22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac0154540 con 0x7f0ac015a1a0 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac5abe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0ac0154160 0x7f0ac0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac52bd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f0ac00ca5a0 0x7f0ac00c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39324/0 (socket says 172.21.15.135:39324) 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.838+0000 7f0ac52bd640 1 -- 172.21.15.135:0/2886083882 learned_addr learned my addr 172.21.15.135:0/2886083882 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac5abe640 1 -- 172.21.15.135:0/2886083882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 msgr2=0x7f0ac00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac62bf640 1 --2- 172.21.15.135:0/2886083882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac5abe640 1 --2- 172.21.15.135:0/2886083882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac00c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac5abe640 1 -- 172.21.15.135:0/2886083882 >> v1:172.21.15.135:6789/0 conn(0x7f0ac00ca5a0 legacy=0x7f0ac00c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac5abe640 1 -- 172.21.15.135:0/2886083882 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac015a910 con 0x7f0ac0154160 2024-04-26T22:00:37.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac4abc640 1 -- 172.21.15.135:0/2886083882 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066859731 0 0) 0x7f0ab0001110 con 0x7f0ac00ca5a0 2024-04-26T22:00:37.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac5abe640 1 --2- 172.21.15.135:0/2886083882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0156bb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0097a0 tx=0x7f0abc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88979f261246caa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.839+0000 7f0ac4abc640 1 -- 172.21.15.135:0/2886083882 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc037070 con 0x7f0ac0154160 2024-04-26T22:00:37.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac4abc640 1 -- 172.21.15.135:0/2886083882 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc032440 con 0x7f0ac0154160 2024-04-26T22:00:37.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac4abc640 1 -- 172.21.15.135:0/2886083882 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc032810 con 0x7f0ac0154160 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2886083882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 msgr2=0x7f0ac0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2886083882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0156bb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0097a0 tx=0x7f0abc02a180 comp rx=0 tx=0).stop 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2886083882 shutdown_connections 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2886083882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2886083882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0156bb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2886083882 >> 172.21.15.135:0/2886083882 conn(0x7f0ac00ae010 msgr2=0x7f0ac00ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2886083882 shutdown_connections 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.840+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2886083882 wait complete. 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.841+0000 7f0ac7d22640 1 Processor -- start 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.841+0000 7f0ac7d22640 1 -- start start 2024-04-26T22:00:37.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac7d22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac00ca5a0 0x7f0ac01ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac7d22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac7d22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac7d22640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac015a5f0 con 0x7f0ac0154160 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac7d22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac015aed0 con 0x7f0ac00ca5a0 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac7d22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac00c7510 con 0x7f0ac015a1a0 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41282/0 (socket says 172.21.15.135:41282) 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 -- 172.21.15.135:0/2479543184 learned_addr learned my addr 172.21.15.135:0/2479543184 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac5abe640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac00ca5a0 0x7f0ac01ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac62bf640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 -- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 msgr2=0x7f0ac014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac014aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 -- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac00ca5a0 msgr2=0x7f0ac01ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac00ca5a0 0x7f0ac01ecb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 -- 172.21.15.135:0/2479543184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01ed0d0 con 0x7f0ac0154160 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.842+0000 7f0ac52bd640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0159260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c780 tx=0x7f0ab000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.843+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0018070 con 0x7f0ac0154160 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.843+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab00072d0 con 0x7f0ac0154160 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.843+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00047e0 con 0x7f0ac0154160 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.843+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac01ed370 con 0x7f0ac0154160 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.843+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01ed930 con 0x7f0ac0154160 2024-04-26T22:00:37.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.843+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac01539d0 con 0x7f0ac0154160 2024-04-26T22:00:37.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.844+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ab001b070 con 0x7f0ac0154160 2024-04-26T22:00:37.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.844+0000 7f0ab67fc640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0aa406c8f0 0x7f0aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.844+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6521+0+0 (secure 0 0 0) 0x7f0ab008b160 con 0x7f0ac0154160 2024-04-26T22:00:37.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.844+0000 7f0ac5abe640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0aa406c8f0 0x7f0aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.845+0000 7f0ac5abe640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0aa406c8f0 0x7f0aa406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0abc034000 tx=0x7f0abc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:37.849+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ac00c8df0 con 0x7f0ac0154160 2024-04-26T22:00:38.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.031+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f0ac0076650 con 0x7f0ac0154160 2024-04-26T22:00:38.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.031+0000 7f0ab67fc640 1 -- 172.21.15.135:0/2479543184 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v25) ==== 99+0+11 (secure 0 0 0) 0x7f0ab00592f0 con 0x7f0ac0154160 2024-04-26T22:00:38.031 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:38.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0aa406c8f0 msgr2=0x7f0aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0aa406c8f0 0x7f0aa406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0abc034000 tx=0x7f0abc02e000 comp rx=0 tx=0).stop 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 msgr2=0x7f0ac0159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0159260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c780 tx=0x7f0ab000cc40 comp rx=0 tx=0).stop 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 shutdown_connections 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0aa406c8f0 0x7f0aa406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac015a1a0 0x7f0ac014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ac0154160 0x7f0ac0159260 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 --2- 172.21.15.135:0/2479543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac00ca5a0 0x7f0ac01ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.033+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 >> 172.21.15.135:0/2479543184 conn(0x7f0ac00ae010 msgr2=0x7f0ac00cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.034+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 shutdown_connections 2024-04-26T22:00:38.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.034+0000 7f0ac7d22640 1 -- 172.21.15.135:0/2479543184 wait complete. 2024-04-26T22:00:38.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:38.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.146+0000 7fd69b1b7640 1 Processor -- start 2024-04-26T22:00:38.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.146+0000 7fd69b1b7640 1 -- start start 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.146+0000 7fd69b1b7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6940720a0 0x7fd6940c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.146+0000 7fd69b1b7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6940c5330 con 0x7fd6940c3060 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.146+0000 7fd69b1b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd694154ba0 con 0x7fd6940c40e0 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.146+0000 7fd69b1b7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd694154e80 con 0x7fd6940720a0 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd6940c3060 0x7fd694151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39330/0 (socket says 172.21.15.135:39330) 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 -- 172.21.15.135:0/4036435001 learned_addr learned my addr 172.21.15.135:0/4036435001 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125140112 0 0) 0x7fd6940c5330 con 0x7fd6940c3060 2024-04-26T22:00:38.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd684003520 con 0x7fd6940c3060 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd699754640 1 --2- 172.21.15.135:0/4036435001 >> v2:172.21.15.139:3300/0 conn(0x7fd6940720a0 0x7fd6940c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2384751710 0 0) 0x7fd684003520 con 0x7fd6940c3060 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 >> v2:172.21.15.139:3300/0 conn(0x7fd6940720a0 msgr2=0x7fd6940c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 --2- 172.21.15.135:0/4036435001 >> v2:172.21.15.139:3300/0 conn(0x7fd6940720a0 0x7fd6940c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 >> v1:172.21.15.139:6789/0 conn(0x7fd6940c40e0 legacy=0x7fd6940c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd694155160 con 0x7fd6940c3060 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fd688001670 con 0x7fd6940c3060 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 >> v1:172.21.15.135:6789/0 conn(0x7fd6940c3060 legacy=0x7fd694151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6840039b0 0x7fd684003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd684004800 0x7fd684004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840051c0 0x7fd68400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd694154e80 con 0x7fd6840039b0 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd68400c160 con 0x7fd684004800 2024-04-26T22:00:38.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd68400c360 con 0x7fd6840051c0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6840039b0 0x7fd684003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd693fff640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd684004800 0x7fd684004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd699754640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840051c0 0x7fd68400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 -- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840051c0 msgr2=0x7fd68400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840051c0 0x7fd68400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 -- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd684004800 msgr2=0x7fd684004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.147+0000 7fd698f53640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd684004800 0x7fd684004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd698f53640 1 -- 172.21.15.135:0/4036435001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd684003520 con 0x7fd6840039b0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd699754640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840051c0 0x7fd68400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd699754640 1 -- 172.21.15.135:0/4036435001 reap_dead start 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd698f53640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6840039b0 0x7fd684003dd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c210 tx=0x7fd684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffce33ea7655cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688007d30 con 0x7fd6840039b0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd688005af0 con 0x7fd6840039b0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.148+0000 7fd6937fe640 1 -- 172.21.15.135:0/4036435001 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688007070 con 0x7fd6840039b0 2024-04-26T22:00:38.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 -- 172.21.15.135:0/4036435001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6840039b0 msgr2=0x7fd684003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6840039b0 0x7fd684003dd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c210 tx=0x7fd684005830 comp rx=0 tx=0).stop 2024-04-26T22:00:38.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 -- 172.21.15.135:0/4036435001 shutdown_connections 2024-04-26T22:00:38.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/4036435001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6840039b0 0x7fd684003dd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 -- 172.21.15.135:0/4036435001 >> 172.21.15.135:0/4036435001 conn(0x7fd69406c310 msgr2=0x7fd69406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 -- 172.21.15.135:0/4036435001 shutdown_connections 2024-04-26T22:00:38.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.149+0000 7fd69b1b7640 1 -- 172.21.15.135:0/4036435001 wait complete. 2024-04-26T22:00:38.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.150+0000 7fd69b1b7640 1 Processor -- start 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.150+0000 7fd69b1b7640 1 -- start start 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840039b0 0x7fd69414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6941494f0 0x7fd6941e1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 0x7fd6941e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6940c3aa0 con 0x7fd694149a40 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd694155720 con 0x7fd6941494f0 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6940c3440 con 0x7fd6840039b0 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 0x7fd6941e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 0x7fd6941e1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41306/0 (socket says 172.21.15.135:41306) 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 -- 172.21.15.135:0/2540684255 learned_addr learned my addr 172.21.15.135:0/2540684255 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd693fff640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6941494f0 0x7fd6941e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd698f53640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840039b0 0x7fd69414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 -- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840039b0 msgr2=0x7fd69414a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840039b0 0x7fd69414a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 -- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6941494f0 msgr2=0x7fd6941e1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6941494f0 0x7fd6941e1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 -- 172.21.15.135:0/2540684255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69414f390 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd699754640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 0x7fd6941e1a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd6940c2660 tx=0x7fd68c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c01a030 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c010da0 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69414e3a0 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.151+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c003bb0 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.152+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6941eb350 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.152+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6941eb550 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.153+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd68c01d070 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.153+0000 7fd6917fa640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd67806c920 0x7fd67806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.153+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6666+0+0 (secure 0 0 0) 0x7fd68c08c2b0 con 0x7fd694149a40 2024-04-26T22:00:38.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.153+0000 7fd698f53640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd67806c920 0x7fd67806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.154+0000 7fd698f53640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd67806c920 0x7fd67806ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd688032000 tx=0x7fd68802d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.158+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd68c059d40 con 0x7fd694149a40 2024-04-26T22:00:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.330+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fd69414e690 con 0x7fd694149a40 2024-04-26T22:00:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.331+0000 7fd6917fa640 1 -- 172.21.15.135:0/2540684255 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v25) ==== 95+0+11 (secure 0 0 0) 0x7fd68c05e040 con 0x7fd694149a40 2024-04-26T22:00:38.331 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd67806c920 msgr2=0x7fd67806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd67806c920 0x7fd67806ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd688032000 tx=0x7fd68802d000 comp rx=0 tx=0).stop 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 msgr2=0x7fd6941e1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 0x7fd6941e1a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd6940c2660 tx=0x7fd68c010590 comp rx=0 tx=0).stop 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 shutdown_connections 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd67806c920 0x7fd67806ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd694149a40 0x7fd6941e1a60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6941494f0 0x7fd6941e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 --2- 172.21.15.135:0/2540684255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6840039b0 0x7fd69414a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 >> 172.21.15.135:0/2540684255 conn(0x7fd69406c310 msgr2=0x7fd69406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 shutdown_connections 2024-04-26T22:00:38.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.333+0000 7fd69b1b7640 1 -- 172.21.15.135:0/2540684255 wait complete. 2024-04-26T22:00:38.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:38.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.440+0000 7fcfabfe4640 1 Processor -- start 2024-04-26T22:00:38.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.441+0000 7fcfabfe4640 1 -- start start 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.441+0000 7fcfabfe4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa40c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.441+0000 7fcfabfe4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcfa40c7160 0x7fcfa4151c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.441+0000 7fcfabfe4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa40a7e30 con 0x7fcfa40c2b90 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.441+0000 7fcfabfe4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa40a7be0 con 0x7fcfa40c7160 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.441+0000 7fcfabfe4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa40701e0 con 0x7fcfa406f6c0 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcfa40c7160 0x7fcfa4151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa9d80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa40c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcfa40c7160 0x7fcfa4151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40376/0 (socket says 172.21.15.135:40376) 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfaa581640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcfa40c2b90 0x7fcfa4158070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54256/0 (socket says 172.21.15.135:54256) 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 -- 172.21.15.135:0/544049116 learned_addr learned my addr 172.21.15.135:0/544049116 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 -- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 msgr2=0x7fcfa40c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 --2- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa40c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 -- 172.21.15.135:0/544049116 >> v1:172.21.15.139:6789/0 conn(0x7fcfa40c2b90 legacy=0x7fcfa4158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa8d7e640 1 -- 172.21.15.135:0/544049116 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271391343 0 0) 0x7fcfa40a7e30 con 0x7fcfa40c2b90 2024-04-26T22:00:38.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.442+0000 7fcfa957f640 1 -- 172.21.15.135:0/544049116 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa406ff00 con 0x7fcfa40c7160 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.443+0000 7fcfa957f640 1 --2- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c7160 0x7fcfa4151c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009870 tx=0x7fcfa002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183433c23593d834 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.443+0000 7fcfa8d7e640 1 -- 172.21.15.135:0/544049116 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0037070 con 0x7fcfa40c7160 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.443+0000 7fcfa8d7e640 1 -- 172.21.15.135:0/544049116 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa0032440 con 0x7fcfa40c7160 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.443+0000 7fcfa8d7e640 1 -- 172.21.15.135:0/544049116 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0032810 con 0x7fcfa40c7160 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.443+0000 7fcfabfe4640 1 -- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c7160 msgr2=0x7fcfa4151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.443+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c7160 0x7fcfa4151c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009870 tx=0x7fcfa002a1c0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.444+0000 7fcfabfe4640 1 -- 172.21.15.135:0/544049116 shutdown_connections 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.444+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c7160 0x7fcfa4151c60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.444+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/544049116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa40c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.444+0000 7fcfabfe4640 1 -- 172.21.15.135:0/544049116 >> 172.21.15.135:0/544049116 conn(0x7fcfa40c8900 msgr2=0x7fcfa40c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.444+0000 7fcfabfe4640 1 -- 172.21.15.135:0/544049116 shutdown_connections 2024-04-26T22:00:38.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.444+0000 7fcfabfe4640 1 -- 172.21.15.135:0/544049116 wait complete. 2024-04-26T22:00:38.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 Processor -- start 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 -- start start 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa41ea320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c2b90 0x7fcfa414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 0x7fcfa40a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa40a8340 con 0x7fcfa40c7160 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4068ef0 con 0x7fcfa40c2b90 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.445+0000 7fcfabfe4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa406faa0 con 0x7fcfa406f6c0 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 0x7fcfa40a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 0x7fcfa40a9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41318/0 (socket says 172.21.15.135:41318) 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfa9d80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa41ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 -- 172.21.15.135:0/683516220 learned_addr learned my addr 172.21.15.135:0/683516220 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfa957f640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c2b90 0x7fcfa414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 -- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 msgr2=0x7fcfa41ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa41ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 -- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c2b90 msgr2=0x7fcfa414a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c2b90 0x7fcfa414a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 -- 172.21.15.135:0/683516220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa41560c0 con 0x7fcfa40c7160 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfaa581640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 0x7fcfa40a9dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c7d0 tx=0x7fcf9c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c018070 con 0x7fcfa40c7160 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c00a640 con 0x7fcfa40c7160 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa41563a0 con 0x7fcfa40c7160 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00aa70 con 0x7fcfa40c7160 2024-04-26T22:00:38.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.446+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa414ad10 con 0x7fcfa40c7160 2024-04-26T22:00:38.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.447+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa4074e60 con 0x7fcfa40c7160 2024-04-26T22:00:38.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.448+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf9c00ac10 con 0x7fcfa40c7160 2024-04-26T22:00:38.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.448+0000 7fcf967fc640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcf8806c920 0x7fcf8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.448+0000 7fcfa9d80640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcf8806c920 0x7fcf8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.448+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6666+0+0 (secure 0 0 0) 0x7fcf9c08b440 con 0x7fcfa40c7160 2024-04-26T22:00:38.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.449+0000 7fcfa9d80640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcf8806c920 0x7fcf8806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4154e60 tx=0x7fcf98009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.453+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf9c058de0 con 0x7fcfa40c7160 2024-04-26T22:00:38.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.666+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfa40765e0 con 0x7fcfa40c7160 2024-04-26T22:00:38.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.666+0000 7fcf967fc640 1 -- 172.21.15.135:0/683516220 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7fcf9c05d0e0 con 0x7fcfa40c7160 2024-04-26T22:00:38.666 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:38.666 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:38.667 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:38.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcf8806c920 msgr2=0x7fcf8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcf8806c920 0x7fcf8806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4154e60 tx=0x7fcf98009190 comp rx=0 tx=0).stop 2024-04-26T22:00:38.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 msgr2=0x7fcfa40a9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 0x7fcfa40a9dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c7d0 tx=0x7fcf9c00cc90 comp rx=0 tx=0).stop 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 shutdown_connections 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcf8806c920 0x7fcf8806ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa40c7160 0x7fcfa40a9dc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfa40c2b90 0x7fcfa414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 --2- 172.21.15.135:0/683516220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfa406f6c0 0x7fcfa41ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 >> 172.21.15.135:0/683516220 conn(0x7fcfa40c8900 msgr2=0x7fcfa414c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.668+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 shutdown_connections 2024-04-26T22:00:38.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.669+0000 7fcfabfe4640 1 -- 172.21.15.135:0/683516220 wait complete. 2024-04-26T22:00:38.679 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:38.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 Processor -- start 2024-04-26T22:00:38.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 -- start start 2024-04-26T22:00:38.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f2013ddd0 0x7f9f20154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9f20070f70 0x7f9f200713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f200707e0 con 0x7f9f20071cc0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9f20071900 con 0x7f9f2013ddd0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f25207640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9f20155450 con 0x7f9f20070f70 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.777+0000 7f9f1ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f2013ddd0 0x7f9f20154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f2013ddd0 0x7f9f20154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55122/0 (socket says 172.21.15.135:55122) 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 -- 172.21.15.135:0/889997460 learned_addr learned my addr 172.21.15.135:0/889997460 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1f59e640 1 --2- 172.21.15.135:0/889997460 >> v2:172.21.15.139:3300/0 conn(0x7f9f20070f70 0x7f9f200713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 -- 172.21.15.135:0/889997460 >> v2:172.21.15.139:3300/0 conn(0x7f9f20070f70 msgr2=0x7f9f200713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 --2- 172.21.15.135:0/889997460 >> v2:172.21.15.139:3300/0 conn(0x7f9f20070f70 0x7f9f200713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 -- 172.21.15.135:0/889997460 >> v1:172.21.15.139:6789/0 conn(0x7f9f20071cc0 legacy=0x7f9f20073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 -- 172.21.15.135:0/889997460 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20155730 con 0x7f9f2013ddd0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.778+0000 7f9f1ed9d640 1 --2- 172.21.15.135:0/889997460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20154b00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9f20074fd0 tx=0x7f9f1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d1f197a951b8de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f1e59c640 1 -- 172.21.15.135:0/889997460 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037070 con 0x7f9f2013ddd0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f1e59c640 1 -- 172.21.15.135:0/889997460 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10032440 con 0x7f9f2013ddd0 2024-04-26T22:00:38.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f1e59c640 1 -- 172.21.15.135:0/889997460 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10032810 con 0x7f9f2013ddd0 2024-04-26T22:00:38.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 -- 172.21.15.135:0/889997460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 msgr2=0x7f9f20154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 --2- 172.21.15.135:0/889997460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20154b00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9f20074fd0 tx=0x7f9f1002a1a0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 -- 172.21.15.135:0/889997460 shutdown_connections 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 --2- 172.21.15.135:0/889997460 >> v2:172.21.15.139:3300/0 conn(0x7f9f20070f70 0x7f9f200713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 --2- 172.21.15.135:0/889997460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20154b00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 -- 172.21.15.135:0/889997460 >> 172.21.15.135:0/889997460 conn(0x7f9f200ca2f0 msgr2=0x7f9f200ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 -- 172.21.15.135:0/889997460 shutdown_connections 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.779+0000 7f9f25207640 1 -- 172.21.15.135:0/889997460 wait complete. 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.780+0000 7f9f25207640 1 Processor -- start 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.780+0000 7f9f25207640 1 -- start start 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f25207640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f20070f70 0x7f9f20136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f25207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f20071cc0 0x7f9f20137060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f25207640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f25207640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20073f90 con 0x7f9f2013ddd0 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f25207640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20073e10 con 0x7f9f20071cc0 2024-04-26T22:00:38.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f25207640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2013e1b0 con 0x7f9f20070f70 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f1f59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f1f59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20139ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55136/0 (socket says 172.21.15.135:55136) 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f1f59e640 1 -- 172.21.15.135:0/3684251510 learned_addr learned my addr 172.21.15.135:0/3684251510 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f1ed9d640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f20070f70 0x7f9f20136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.781+0000 7f9f17fff640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f20071cc0 0x7f9f20137060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f1f59e640 1 -- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f20070f70 msgr2=0x7f9f20136b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f1f59e640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f20070f70 0x7f9f20136b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f1f59e640 1 -- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f20071cc0 msgr2=0x7f9f20137060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f1f59e640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f20071cc0 0x7f9f20137060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f1f59e640 1 -- 172.21.15.135:0/3684251510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20151f50 con 0x7f9f2013ddd0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f1f59e640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20139ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800cc60 tx=0x7f9f1800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18003990 con 0x7f9f2013ddd0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f18003b30 con 0x7f9f2013ddd0 2024-04-26T22:00:38.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f20138540 con 0x7f9f2013ddd0 2024-04-26T22:00:38.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004020 con 0x7f9f2013ddd0 2024-04-26T22:00:38.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.782+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f20139020 con 0x7f9f2013ddd0 2024-04-26T22:00:38.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.783+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f200507a0 con 0x7f9f2013ddd0 2024-04-26T22:00:38.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.783+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f180041c0 con 0x7f9f2013ddd0 2024-04-26T22:00:38.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.784+0000 7f9f177fe640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f0006c920 0x7f9f0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.784+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6666+0+0 (secure 0 0 0) 0x7f9f1808c370 con 0x7f9f2013ddd0 2024-04-26T22:00:38.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.784+0000 7f9f1ed9d640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f0006c920 0x7f9f0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.784+0000 7f9f1ed9d640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f0006c920 0x7f9f0006ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f10034000 tx=0x7f9f1002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.788+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f18014030 con 0x7f9f2013ddd0 2024-04-26T22:00:38.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.972+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f20074110 con 0x7f9f2013ddd0 2024-04-26T22:00:38.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.974+0000 7f9f177fe640 1 -- 172.21.15.135:0/3684251510 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20688 (secure 0 0 0) 0x7f9f18059d10 con 0x7f9f2013ddd0 2024-04-26T22:00:38.974 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:38.974 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:38.936891+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f0006c920 msgr2=0x7f9f0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f0006c920 0x7f9f0006ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f10034000 tx=0x7f9f1002e000 comp rx=0 tx=0).stop 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 msgr2=0x7f9f20139ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20139ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800cc60 tx=0x7f9f1800a5c0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 shutdown_connections 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f0006c920 0x7f9f0006ede0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2013ddd0 0x7f9f20139ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f20071cc0 0x7f9f20137060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 --2- 172.21.15.135:0/3684251510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f20070f70 0x7f9f20136b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 >> 172.21.15.135:0/3684251510 conn(0x7f9f200ca2f0 msgr2=0x7f9f200abda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 shutdown_connections 2024-04-26T22:00:38.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:38.976+0000 7f9f25207640 1 -- 172.21.15.135:0/3684251510 wait complete. 2024-04-26T22:00:38.987 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:38.988 INFO:tasks.mds_thrash.fs.[a]:starting mds_do_thrash for fs a 2024-04-26T22:00:38.988 INFO:tasks.mds_thrash.fs.[a]:waiting for 34 secs before thrashing 2024-04-26T22:00:39.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.085+0000 7f2df97e7640 1 Processor -- start 2024-04-26T22:00:39.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.085+0000 7f2df97e7640 1 -- start start 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.085+0000 7f2df97e7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2df4156bb0 0x7f2df415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.085+0000 7f2df97e7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df40c8df0 con 0x7f2df40ca5a0 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.085+0000 7f2df97e7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df4154540 con 0x7f2df4154160 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.085+0000 7f2df97e7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2df415a6e0 con 0x7f2df4156bb0 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df27fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2df40ca5a0 0x7f2df40c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:40852/0 (socket says 172.21.15.135:40852) 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df27fc640 1 -- 172.21.15.135:0/2744106045 learned_addr learned my addr 172.21.15.135:0/2744106045 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709291642 0 0) 0x7f2de8001110 con 0x7f2df40ca5a0 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ddc003520 con 0x7f2df40ca5a0 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df37fe640 1 --2- 172.21.15.135:0/2744106045 >> v2:172.21.15.139:3300/0 conn(0x7f2df4156bb0 0x7f2df415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404522009 0 0) 0x7f2ddc003520 con 0x7f2df40ca5a0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 >> v2:172.21.15.139:3300/0 conn(0x7f2df4156bb0 msgr2=0x7f2df415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 --2- 172.21.15.135:0/2744106045 >> v2:172.21.15.139:3300/0 conn(0x7f2df4156bb0 0x7f2df415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 >> v1:172.21.15.139:6789/0 conn(0x7f2df4154160 legacy=0x7f2df40c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df415a910 con 0x7f2df40ca5a0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2de8001840 con 0x7f2df40ca5a0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 >> v1:172.21.15.135:6789/0 conn(0x7f2df40ca5a0 legacy=0x7f2df40c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2ddc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ddc004800 0x7f2ddc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc00c030 con 0x7f2ddc0039b0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc00c1e0 con 0x7f2ddc0051c0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2ddc00c3e0 con 0x7f2ddc004800 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2ddc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df27fc640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ddc004800 0x7f2ddc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df37fe640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df2ffd640 1 -- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ddc004800 msgr2=0x7f2ddc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ddc004800 0x7f2ddc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df2ffd640 1 -- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc0051c0 msgr2=0x7f2ddc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df37fe640 1 -- 172.21.15.135:0/2744106045 reap_dead start 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.086+0000 7f2df2ffd640 1 -- 172.21.15.135:0/2744106045 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc003520 con 0x7f2ddc0039b0 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.087+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2ddc003dd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001720 tx=0x7f2ddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90e0d218ecbb160 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.087+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4001370 con 0x7f2ddc0039b0 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.087+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de40052d0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.087+0000 7f2df1ffb640 1 -- 172.21.15.135:0/2744106045 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de402ab90 con 0x7f2ddc0039b0 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 -- 172.21.15.135:0/2744106045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 msgr2=0x7f2ddc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2ddc003dd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001720 tx=0x7f2ddc005830 comp rx=0 tx=0).stop 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 -- 172.21.15.135:0/2744106045 shutdown_connections 2024-04-26T22:00:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 --2- 172.21.15.135:0/2744106045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2ddc003dd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 -- 172.21.15.135:0/2744106045 >> 172.21.15.135:0/2744106045 conn(0x7f2df40ae010 msgr2=0x7f2df40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 -- 172.21.15.135:0/2744106045 shutdown_connections 2024-04-26T22:00:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.088+0000 7f2df97e7640 1 -- 172.21.15.135:0/2744106045 wait complete. 2024-04-26T22:00:39.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.089+0000 7f2df97e7640 1 Processor -- start 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.089+0000 7f2df97e7640 1 -- start start 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2df414b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2df4159ca0 0x7f2df40cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2df40cb820 0x7f2df41efa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2df415aed0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40c9100 con 0x7f2df4159ca0 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2df415a1a0 con 0x7f2df40cb820 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2df414b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2df414b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55156/0 (socket says 172.21.15.135:55156) 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 -- 172.21.15.135:0/1484101977 learned_addr learned my addr 172.21.15.135:0/1484101977 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df27fc640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2df4159ca0 0x7f2df40cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 -- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2df40cb820 msgr2=0x7f2df41efa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df37fe640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2df40cb820 0x7f2df41efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2df40cb820 0x7f2df41efa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 -- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2df4159ca0 msgr2=0x7f2df40cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2df4159ca0 0x7f2df40cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 -- 172.21.15.135:0/1484101977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df4158de0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df27fc640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2df4159ca0 0x7f2df40cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df2ffd640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2df414b810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001840 tx=0x7f2de4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4007a50 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4041070 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df40cc5d0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.090+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4005480 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.091+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df40cadc0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.092+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df4076650 con 0x7f2ddc0039b0 2024-04-26T22:00:39.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.092+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2de40067d0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.092+0000 7f2ddb7fe640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2de006c920 0x7f2de006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.092+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f2de40ac060 con 0x7f2ddc0039b0 2024-04-26T22:00:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.092+0000 7f2df27fc640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2de006c920 0x7f2de006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.093+0000 7f2df27fc640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2de006c920 0x7f2de006ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2de80013d0 tx=0x7f2de8001320 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.096+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2de4007680 con 0x7f2ddc0039b0 2024-04-26T22:00:39.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.278+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2df40507a0 con 0x7f2ddc0039b0 2024-04-26T22:00:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.279+0000 7f2ddb7fe640 1 -- 172.21.15.135:0/1484101977 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20688 (secure 0 0 0) 0x7f2de407a160 con 0x7f2ddc0039b0 2024-04-26T22:00:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:39.279 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:38.936891+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2de006c920 msgr2=0x7f2de006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2de006c920 0x7f2de006ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2de80013d0 tx=0x7f2de8001320 comp rx=0 tx=0).stop 2024-04-26T22:00:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 msgr2=0x7f2df414b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2df414b810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001840 tx=0x7f2de4001120 comp rx=0 tx=0).stop 2024-04-26T22:00:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 shutdown_connections 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2de006c920 0x7f2de006ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2df40cb820 0x7f2df41efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2df4159ca0 0x7f2df40cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 --2- 172.21.15.135:0/1484101977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ddc0039b0 0x7f2df414b810 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 >> 172.21.15.135:0/1484101977 conn(0x7f2df40ae010 msgr2=0x7f2df4156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 shutdown_connections 2024-04-26T22:00:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.281+0000 7f2df97e7640 1 -- 172.21.15.135:0/1484101977 wait complete. 2024-04-26T22:00:39.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:00:39.293 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:00:39.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.392+0000 7f91e93a5640 1 Processor -- start 2024-04-26T22:00:39.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.392+0000 7f91e93a5640 1 -- start start 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e93a5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f91e413ddd0 0x7f91e4154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e93a5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91e4071cc0 0x7f91e4073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e93a5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e40707e0 con 0x7f91e4073c40 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e93a5640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f91e4155450 con 0x7f91e413ddd0 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e93a5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f91e4155730 con 0x7f91e4071cc0 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f91e413ddd0 0x7f91e4154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f91e413ddd0 0x7f91e4154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55168/0 (socket says 172.21.15.135:55168) 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 -- 172.21.15.135:0/1626728560 learned_addr learned my addr 172.21.15.135:0/1626728560 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 -- 172.21.15.135:0/1626728560 >> v2:172.21.15.139:3300/0 conn(0x7f91e4071cc0 msgr2=0x7f91e4073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 --2- 172.21.15.135:0/1626728560 >> v2:172.21.15.139:3300/0 conn(0x7f91e4071cc0 0x7f91e4073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 -- 172.21.15.135:0/1626728560 >> v1:172.21.15.139:6789/0 conn(0x7f91e4073c40 legacy=0x7f91e4070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.393+0000 7f91e2ffd640 1 -- 172.21.15.135:0/1626728560 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e4155a10 con 0x7f91e413ddd0 2024-04-26T22:00:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.394+0000 7f91e2ffd640 1 --2- 172.21.15.135:0/1626728560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e413ddd0 0x7f91e4154b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f91e40cb3b0 tx=0x7f91d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36931182087950db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.394+0000 7f91e1ffb640 1 -- 172.21.15.135:0/1626728560 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4036070 con 0x7f91e413ddd0 2024-04-26T22:00:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.394+0000 7f91e1ffb640 1 -- 172.21.15.135:0/1626728560 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d4031600 con 0x7f91e413ddd0 2024-04-26T22:00:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.394+0000 7f91e1ffb640 1 -- 172.21.15.135:0/1626728560 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d40319d0 con 0x7f91e413ddd0 2024-04-26T22:00:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 -- 172.21.15.135:0/1626728560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e413ddd0 msgr2=0x7f91e4154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 --2- 172.21.15.135:0/1626728560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e413ddd0 0x7f91e4154b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f91e40cb3b0 tx=0x7f91d402a660 comp rx=0 tx=0).stop 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 -- 172.21.15.135:0/1626728560 shutdown_connections 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 --2- 172.21.15.135:0/1626728560 >> v2:172.21.15.139:3300/0 conn(0x7f91e4071cc0 0x7f91e4073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 --2- 172.21.15.135:0/1626728560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e413ddd0 0x7f91e4154b00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 -- 172.21.15.135:0/1626728560 >> 172.21.15.135:0/1626728560 conn(0x7f91e40ca2f0 msgr2=0x7f91e40ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 -- 172.21.15.135:0/1626728560 shutdown_connections 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.395+0000 7f91e93a5640 1 -- 172.21.15.135:0/1626728560 wait complete. 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.396+0000 7f91e93a5640 1 Processor -- start 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.396+0000 7f91e93a5640 1 -- start start 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91e4071cc0 0x7f91e4151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 0x7f91e4153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91e413ddd0 0x7f91e40ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91e41551c0 con 0x7f91e4073c40 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91e4155040 con 0x7f91e4071cc0 2024-04-26T22:00:39.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91e40ab470 con 0x7f91e413ddd0 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 0x7f91e4153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 0x7f91e4153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55172/0 (socket says 172.21.15.135:55172) 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 -- 172.21.15.135:0/839674476 learned_addr learned my addr 172.21.15.135:0/839674476 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e2ffd640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91e4071cc0 0x7f91e4151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e37fe640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91e413ddd0 0x7f91e40ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 -- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91e413ddd0 msgr2=0x7f91e40ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91e413ddd0 0x7f91e40ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 -- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91e4071cc0 msgr2=0x7f91e4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91e4071cc0 0x7f91e4151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 -- 172.21.15.135:0/839674476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e4151b20 con 0x7f91e4073c40 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e27fc640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 0x7f91e4153c30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f91d800b440 tx=0x7f91d800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8015020 con 0x7f91e4073c40 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d8011890 con 0x7f91e4073c40 2024-04-26T22:00:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8011c80 con 0x7f91e4073c40 2024-04-26T22:00:39.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.397+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e41eb150 con 0x7f91e4073c40 2024-04-26T22:00:39.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.398+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e4154170 con 0x7f91e4073c40 2024-04-26T22:00:39.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.399+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e4076650 con 0x7f91e4073c40 2024-04-26T22:00:39.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.399+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d8018020 con 0x7f91e4073c40 2024-04-26T22:00:39.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.399+0000 7f91cb7fe640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f91d006c8f0 0x7f91d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.400+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f91d808be70 con 0x7f91e4073c40 2024-04-26T22:00:39.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.400+0000 7f91e2ffd640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f91d006c8f0 0x7f91d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.400+0000 7f91e2ffd640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f91d006c8f0 0x7f91d006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f91d4033000 tx=0x7f91d4009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.404+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91d801f020 con 0x7f91e4073c40 2024-04-26T22:00:39.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.578+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91e40507a0 con 0x7f91e4073c40 2024-04-26T22:00:39.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.578+0000 7f91cb7fe640 1 -- 172.21.15.135:0/839674476 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f91d800d040 con 0x7f91e4073c40 2024-04-26T22:00:39.578 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:00:39.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f91d006c8f0 msgr2=0x7f91d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f91d006c8f0 0x7f91d006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f91d4033000 tx=0x7f91d4009000 comp rx=0 tx=0).stop 2024-04-26T22:00:39.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 msgr2=0x7f91e4153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 0x7f91e4153c30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f91d800b440 tx=0x7f91d800b900 comp rx=0 tx=0).stop 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 shutdown_connections 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f91d006c8f0 0x7f91d006edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91e413ddd0 0x7f91e40ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91e4073c40 0x7f91e4153c30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 --2- 172.21.15.135:0/839674476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91e4071cc0 0x7f91e4151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 >> 172.21.15.135:0/839674476 conn(0x7f91e40ca2f0 msgr2=0x7f91e4070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 shutdown_connections 2024-04-26T22:00:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.580+0000 7f91e93a5640 1 -- 172.21.15.135:0/839674476 wait complete. 2024-04-26T22:00:39.591 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:00:39.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.691+0000 7f13b93af640 1 Processor -- start 2024-04-26T22:00:39.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.691+0000 7f13b93af640 1 -- start start 2024-04-26T22:00:39.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b93af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13b414eb50 0x7f13b40c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b93af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13b4158980 0x7f13b406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b93af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b4154a90 con 0x7f13b406f8a0 2024-04-26T22:00:39.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b93af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f13b40c5080 con 0x7f13b4158980 2024-04-26T22:00:39.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b93af640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f13b4155930 con 0x7f13b414eb50 2024-04-26T22:00:39.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13b4158980 0x7f13b406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13b4158980 0x7f13b406d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55182/0 (socket says 172.21.15.135:55182) 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b37fe640 1 -- 172.21.15.135:0/2683287814 learned_addr learned my addr 172.21.15.135:0/2683287814 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.692+0000 7f13b2ffd640 1 --2- 172.21.15.135:0/2683287814 >> v2:172.21.15.139:3300/0 conn(0x7f13b414eb50 0x7f13b40c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b37fe640 1 -- 172.21.15.135:0/2683287814 >> v2:172.21.15.139:3300/0 conn(0x7f13b414eb50 msgr2=0x7f13b40c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b37fe640 1 --2- 172.21.15.135:0/2683287814 >> v2:172.21.15.139:3300/0 conn(0x7f13b414eb50 0x7f13b40c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b37fe640 1 -- 172.21.15.135:0/2683287814 >> v1:172.21.15.139:6789/0 conn(0x7f13b406f8a0 legacy=0x7f13b4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b37fe640 1 -- 172.21.15.135:0/2683287814 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b406ff90 con 0x7f13b4158980 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b37fe640 1 --2- 172.21.15.135:0/2683287814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b4158980 0x7f13b406d320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f13b4076790 tx=0x7f13ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8e673cbfff8495 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b1ffb640 1 -- 172.21.15.135:0/2683287814 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac037070 con 0x7f13b4158980 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b1ffb640 1 -- 172.21.15.135:0/2683287814 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac032440 con 0x7f13b4158980 2024-04-26T22:00:39.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.693+0000 7f13b1ffb640 1 -- 172.21.15.135:0/2683287814 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac032810 con 0x7f13b4158980 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 -- 172.21.15.135:0/2683287814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b4158980 msgr2=0x7f13b406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 --2- 172.21.15.135:0/2683287814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b4158980 0x7f13b406d320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f13b4076790 tx=0x7f13ac02a230 comp rx=0 tx=0).stop 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 -- 172.21.15.135:0/2683287814 shutdown_connections 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 --2- 172.21.15.135:0/2683287814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b4158980 0x7f13b406d320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 --2- 172.21.15.135:0/2683287814 >> v2:172.21.15.139:3300/0 conn(0x7f13b414eb50 0x7f13b40c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 -- 172.21.15.135:0/2683287814 >> 172.21.15.135:0/2683287814 conn(0x7f13b40c5d70 msgr2=0x7f13b40c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 -- 172.21.15.135:0/2683287814 shutdown_connections 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.694+0000 7f13b93af640 1 -- 172.21.15.135:0/2683287814 wait complete. 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.695+0000 7f13b93af640 1 Processor -- start 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.695+0000 7f13b93af640 1 -- start start 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b93af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 0x7f13b41df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b93af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b414eb50 0x7f13b414bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b93af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13b4158980 0x7f13b414c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b93af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4158db0 con 0x7f13b406f8a0 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b93af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4154d00 con 0x7f13b414eb50 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b93af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13b4070550 con 0x7f13b4158980 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 0x7f13b41df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 0x7f13b41df8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55196/0 (socket says 172.21.15.135:55196) 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 -- 172.21.15.135:0/832745294 learned_addr learned my addr 172.21.15.135:0/832745294 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b37fe640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13b4158980 0x7f13b414c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b27fc640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b414eb50 0x7f13b414bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 -- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13b4158980 msgr2=0x7f13b414c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13b4158980 0x7f13b414c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 -- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b414eb50 msgr2=0x7f13b414bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b414eb50 0x7f13b414bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.696+0000 7f13b2ffd640 1 -- 172.21.15.135:0/832745294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4151f40 con 0x7f13b406f8a0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.697+0000 7f13b2ffd640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 0x7f13b41df8b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13a4007e40 tx=0x7f13a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.697+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4014620 con 0x7f13b406f8a0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.697+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a4019070 con 0x7f13b406f8a0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.697+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b41ee0d0 con 0x7f13b406f8a0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.697+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4014a00 con 0x7f13b406f8a0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.697+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41ee690 con 0x7f13b406f8a0 2024-04-26T22:00:39.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.698+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b4074b80 con 0x7f13b406f8a0 2024-04-26T22:00:39.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.698+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13a4014ba0 con 0x7f13b406f8a0 2024-04-26T22:00:39.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.698+0000 7f139b7fe640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f13a006c8f0 0x7f13a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.699+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f13a408c5d0 con 0x7f13b406f8a0 2024-04-26T22:00:39.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.699+0000 7f13b27fc640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f13a006c8f0 0x7f13a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.699+0000 7f13b27fc640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f13a006c8f0 0x7f13a006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f13b4152ad0 tx=0x7f13a8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.702 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.703+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f13a405a080 con 0x7f13b406f8a0 2024-04-26T22:00:39.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.878+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f13b40507a0 con 0x7f13b406f8a0 2024-04-26T22:00:39.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.878+0000 7f139b7fe640 1 -- 172.21.15.135:0/832745294 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f13a405e380 con 0x7f13b406f8a0 2024-04-26T22:00:39.878 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:39.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f13a006c8f0 msgr2=0x7f13a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f13a006c8f0 0x7f13a006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f13b4152ad0 tx=0x7f13a8009060 comp rx=0 tx=0).stop 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 msgr2=0x7f13b41df8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 0x7f13b41df8b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13a4007e40 tx=0x7f13a400e4b0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 shutdown_connections 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f13a006c8f0 0x7f13a006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13b4158980 0x7f13b414c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b414eb50 0x7f13b414bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 --2- 172.21.15.135:0/832745294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13b406f8a0 0x7f13b41df8b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 >> 172.21.15.135:0/832745294 conn(0x7f13b40c5d70 msgr2=0x7f13b40c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 shutdown_connections 2024-04-26T22:00:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.880+0000 7f13b93af640 1 -- 172.21.15.135:0/832745294 wait complete. 2024-04-26T22:00:39.891 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:00:39.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.991+0000 7fd1c70fc640 1 Processor -- start 2024-04-26T22:00:39.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.991+0000 7fd1c70fc640 1 -- start start 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c70fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd1c01510c0 0x7fd1c0072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c70fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd1c00c4160 0x7fd1c015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c70fc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1c00c4b50 con 0x7fd1c00734d0 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c70fc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c00c50d0 con 0x7fd1c00c4160 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c70fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c01514a0 con 0x7fd1c01510c0 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c5699640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd1c00c4160 0x7fd1c015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1bffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd1c00734d0 0x7fd1c006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:40858/0 (socket says 172.21.15.135:40858) 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1bffff640 1 -- 172.21.15.135:0/583732208 learned_addr learned my addr 172.21.15.135:0/583732208 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.992+0000 7fd1c5699640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd1c00c4160 0x7fd1c015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55198/0 (socket says 172.21.15.135:55198) 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1c4e98640 1 --2- 172.21.15.135:0/583732208 >> v2:172.21.15.139:3300/0 conn(0x7fd1c01510c0 0x7fd1c0072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1c5699640 1 -- 172.21.15.135:0/583732208 >> v2:172.21.15.139:3300/0 conn(0x7fd1c01510c0 msgr2=0x7fd1c0072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1c5699640 1 --2- 172.21.15.135:0/583732208 >> v2:172.21.15.139:3300/0 conn(0x7fd1c01510c0 0x7fd1c0072f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1c5699640 1 -- 172.21.15.135:0/583732208 >> v1:172.21.15.135:6789/0 conn(0x7fd1c00734d0 legacy=0x7fd1c006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1c5699640 1 -- 172.21.15.135:0/583732208 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c0154920 con 0x7fd1c00c4160 2024-04-26T22:00:39.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1bf7fe640 1 -- 172.21.15.135:0/583732208 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1705372681 0 0) 0x7fd1c00c4b50 con 0x7fd1c00734d0 2024-04-26T22:00:39.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1c5699640 1 --2- 172.21.15.135:0/583732208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c015e400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00c5250 tx=0x7fd1b802a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10b2673c1e8ea44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1bf7fe640 1 -- 172.21.15.135:0/583732208 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8007740 con 0x7fd1c00c4160 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1bf7fe640 1 -- 172.21.15.135:0/583732208 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b80078e0 con 0x7fd1c00c4160 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.993+0000 7fd1bf7fe640 1 -- 172.21.15.135:0/583732208 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8007cb0 con 0x7fd1c00c4160 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 -- 172.21.15.135:0/583732208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 msgr2=0x7fd1c015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/583732208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c015e400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00c5250 tx=0x7fd1b802a540 comp rx=0 tx=0).stop 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 -- 172.21.15.135:0/583732208 shutdown_connections 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/583732208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c015e400 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/583732208 >> v2:172.21.15.139:3300/0 conn(0x7fd1c01510c0 0x7fd1c0072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 -- 172.21.15.135:0/583732208 >> 172.21.15.135:0/583732208 conn(0x7fd1c00c5ce0 msgr2=0x7fd1c00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 -- 172.21.15.135:0/583732208 shutdown_connections 2024-04-26T22:00:39.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.994+0000 7fd1c70fc640 1 -- 172.21.15.135:0/583732208 wait complete. 2024-04-26T22:00:39.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.995+0000 7fd1c70fc640 1 Processor -- start 2024-04-26T22:00:39.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.995+0000 7fd1c70fc640 1 -- start start 2024-04-26T22:00:39.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c70fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c0152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c70fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c01510c0 0x7fd1c0075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c70fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1c00aef40 0x7fd1c01e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c70fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c00c5920 con 0x7fd1c00c4160 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c70fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c00c5400 con 0x7fd1c00aef40 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c70fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c006f500 con 0x7fd1c01510c0 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c0152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c0152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55200/0 (socket says 172.21.15.135:55200) 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 -- 172.21.15.135:0/3415030521 learned_addr learned my addr 172.21.15.135:0/3415030521 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c5699640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1c00aef40 0x7fd1c01e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1bffff640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c01510c0 0x7fd1c0075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 -- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c01510c0 msgr2=0x7fd1c0075960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c01510c0 0x7fd1c0075960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 -- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1c00aef40 msgr2=0x7fd1c01e74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1c00aef40 0x7fd1c01e74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 -- 172.21.15.135:0/3415030521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c01ee3a0 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1bffff640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c01510c0 0x7fd1c0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.996+0000 7fd1c4e98640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c0152fb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c7f0 tx=0x7fd1b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.997+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0015070 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.997+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0004220 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.997+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c01ee680 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.997+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0007620 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.997+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c01eb9f0 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.998+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c00507a0 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.998+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1b00077c0 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.998+0000 7fd1bd7fa640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd1ac06c920 0x7fd1ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.998+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7fd1b008bc50 con 0x7fd1c00c4160 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.998+0000 7fd1bffff640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd1ac06c920 0x7fd1ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:39.999+0000 7fd1bffff640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd1ac06c920 0x7fd1ac06ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00af590 tx=0x7fd1b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.003+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1b0059d50 con 0x7fd1c00c4160 2024-04-26T22:00:40.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.180+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7fd1c00c3ed0 con 0x7fd1c00c4160 2024-04-26T22:00:40.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.181+0000 7fd1bd7fa640 1 -- 172.21.15.135:0/3415030521 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7fd1b0059650 con 0x7fd1c00c4160 2024-04-26T22:00:40.181 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd1ac06c920 msgr2=0x7fd1ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd1ac06c920 0x7fd1ac06ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00af590 tx=0x7fd1b4009040 comp rx=0 tx=0).stop 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 msgr2=0x7fd1c0152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c0152fb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c7f0 tx=0x7fd1b000ccb0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 shutdown_connections 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd1ac06c920 0x7fd1ac06ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1c00aef40 0x7fd1c01e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c01510c0 0x7fd1c0075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 --2- 172.21.15.135:0/3415030521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1c00c4160 0x7fd1c0152fb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 >> 172.21.15.135:0/3415030521 conn(0x7fd1c00c5ce0 msgr2=0x7fd1c00680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 shutdown_connections 2024-04-26T22:00:40.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.183+0000 7fd1c70fc640 1 -- 172.21.15.135:0/3415030521 wait complete. 2024-04-26T22:00:40.194 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:00:40.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.294+0000 7f2666171640 1 Processor -- start 2024-04-26T22:00:40.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.294+0000 7f2666171640 1 -- start start 2024-04-26T22:00:40.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f2666171640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f26601541d0 0x7f26600c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f2666171640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f266015a3c0 0x7f266015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f2666171640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26600c3fc0 con 0x7f2660070490 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f2666171640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f26600c3500 con 0x7f266015a3c0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f2666171640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f26600c37e0 con 0x7f26601541d0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f266015a3c0 0x7f266015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f266015a3c0 0x7f266015a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55212/0 (socket says 172.21.15.135:55212) 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 -- 172.21.15.135:0/956833335 learned_addr learned my addr 172.21.15.135:0/956833335 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265f7fe640 1 --2- 172.21.15.135:0/956833335 >> v2:172.21.15.139:3300/0 conn(0x7f26601541d0 0x7f26600c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 -- 172.21.15.135:0/956833335 >> v2:172.21.15.139:3300/0 conn(0x7f26601541d0 msgr2=0x7f26600c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 --2- 172.21.15.135:0/956833335 >> v2:172.21.15.139:3300/0 conn(0x7f26601541d0 0x7f26600c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 -- 172.21.15.135:0/956833335 >> v1:172.21.15.139:6789/0 conn(0x7f2660070490 legacy=0x7f26600c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.295+0000 7f265ffff640 1 -- 172.21.15.135:0/956833335 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26600c3ac0 con 0x7f266015a3c0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.296+0000 7f265ffff640 1 --2- 172.21.15.135:0/956833335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f266015a3c0 0x7f266015a7e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f26600c2f80 tx=0x7f265802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8981c97d29a14c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.296+0000 7f265e7fc640 1 -- 172.21.15.135:0/956833335 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658036070 con 0x7f266015a3c0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.296+0000 7f265e7fc640 1 -- 172.21.15.135:0/956833335 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658031600 con 0x7f266015a3c0 2024-04-26T22:00:40.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.296+0000 7f265e7fc640 1 -- 172.21.15.135:0/956833335 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580319d0 con 0x7f266015a3c0 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.296+0000 7f2666171640 1 -- 172.21.15.135:0/956833335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f266015a3c0 msgr2=0x7f266015a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.296+0000 7f2666171640 1 --2- 172.21.15.135:0/956833335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f266015a3c0 0x7f266015a7e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f26600c2f80 tx=0x7f265802a660 comp rx=0 tx=0).stop 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.297+0000 7f2666171640 1 -- 172.21.15.135:0/956833335 shutdown_connections 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.297+0000 7f2666171640 1 --2- 172.21.15.135:0/956833335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f266015a3c0 0x7f266015a7e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.297+0000 7f2666171640 1 --2- 172.21.15.135:0/956833335 >> v2:172.21.15.139:3300/0 conn(0x7f26601541d0 0x7f26600c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.297+0000 7f2666171640 1 -- 172.21.15.135:0/956833335 >> 172.21.15.135:0/956833335 conn(0x7f26600c8c40 msgr2=0x7f26600c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.297+0000 7f2666171640 1 -- 172.21.15.135:0/956833335 shutdown_connections 2024-04-26T22:00:40.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.297+0000 7f2666171640 1 -- 172.21.15.135:0/956833335 wait complete. 2024-04-26T22:00:40.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 Processor -- start 2024-04-26T22:00:40.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 -- start start 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 0x7f2660159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26601541d0 0x7f26601e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f266015a3c0 0x7f26600ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26600c7cb0 con 0x7f2660070490 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26600c7b30 con 0x7f266015a3c0 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.298+0000 7f2666171640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f26600c42a0 con 0x7f26601541d0 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 0x7f2660159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 0x7f2660159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55222/0 (socket says 172.21.15.135:55222) 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 -- 172.21.15.135:0/4012385333 learned_addr learned my addr 172.21.15.135:0/4012385333 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265ffff640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f266015a3c0 0x7f26600ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265effd640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26601541d0 0x7f26601e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 -- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26601541d0 msgr2=0x7f26601e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26601541d0 0x7f26601e3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 -- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f266015a3c0 msgr2=0x7f26600ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f266015a3c0 0x7f26600ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 -- 172.21.15.135:0/4012385333 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601efcb0 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265ffff640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f266015a3c0 0x7f26600ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f265f7fe640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 0x7f2660159bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f265000d830 tx=0x7f265000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000b6c0 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2650019070 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000ba80 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26601eff90 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.299+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601f0550 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.300+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2660076690 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.301+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f265001e460 con 0x7f2660070490 2024-04-26T22:00:40.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.301+0000 7f2647fff640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f264c06c8f0 0x7f264c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.301+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f265008c560 con 0x7f2660070490 2024-04-26T22:00:40.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.301+0000 7f265effd640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f264c06c8f0 0x7f264c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.302+0000 7f265effd640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f264c06c8f0 0x7f264c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2654003d90 tx=0x7f2654009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.305+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f265005a660 con 0x7f2660070490 2024-04-26T22:00:40.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.481+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f2660157ac0 con 0x7f2660070490 2024-04-26T22:00:40.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.481+0000 7f2647fff640 1 -- 172.21.15.135:0/4012385333 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) ==== 99+0+11 (secure 0 0 0) 0x7f2650059f60 con 0x7f2660070490 2024-04-26T22:00:40.481 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:00:40.483 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f264c06c8f0 msgr2=0x7f264c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f264c06c8f0 0x7f264c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2654003d90 tx=0x7f2654009040 comp rx=0 tx=0).stop 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 msgr2=0x7f2660159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 0x7f2660159bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f265000d830 tx=0x7f265000dcf0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 shutdown_connections 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f264c06c8f0 0x7f264c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f266015a3c0 0x7f26600ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26601541d0 0x7f26601e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 --2- 172.21.15.135:0/4012385333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2660070490 0x7f2660159bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 >> 172.21.15.135:0/4012385333 conn(0x7f26600c8c40 msgr2=0x7f26600680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 shutdown_connections 2024-04-26T22:00:40.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.483+0000 7f2666171640 1 -- 172.21.15.135:0/4012385333 wait complete. 2024-04-26T22:00:40.494 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:00:40.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.594+0000 7fd4565f0640 1 Processor -- start 2024-04-26T22:00:40.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.594+0000 7fd4565f0640 1 -- start start 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd4565f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 0x7fd4501576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd4565f0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd450156c10 con 0x7fd4500c2b80 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd4565f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4500c5570 con 0x7fd4500c5050 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd4565f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd45015a8b0 con 0x7fd45013feb0 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd454b8d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd4500c2b80 0x7fd45006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:40864/0 (socket says 172.21.15.135:40864) 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd454b8d640 1 -- 172.21.15.135:0/2147460526 learned_addr learned my addr 172.21.15.135:0/2147460526 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44ffff640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 0x7fd4501576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2966484096 0 0) 0x7fd450156c10 con 0x7fd4500c2b80 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd438003520 con 0x7fd4500c2b80 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3703858762 0 0) 0x7fd438003520 con 0x7fd4500c2b80 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 msgr2=0x7fd4501576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 0x7fd4501576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 >> v1:172.21.15.139:6789/0 conn(0x7fd4500c5050 legacy=0x7fd45015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4500c5850 con 0x7fd4500c2b80 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fd4480019f0 con 0x7fd4500c2b80 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 >> v1:172.21.15.135:6789/0 conn(0x7fd4500c2b80 legacy=0x7fd45006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 0x7fd4380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438003f20 0x7fd438004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd438004d40 0x7fd4380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd45015a8b0 con 0x7fd438004d40 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd43800bad0 con 0x7fd438003f20 2024-04-26T22:00:40.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.595+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd43800bcd0 con 0x7fd45013feb0 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd438004d40 0x7fd4380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd44f7fe640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438003f20 0x7fd438004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd44ffff640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 0x7fd4380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 -- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 msgr2=0x7fd4380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd45013feb0 0x7fd4380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 -- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438003f20 msgr2=0x7fd438004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd438003f20 0x7fd438004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 -- 172.21.15.135:0/2147460526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438003520 con 0x7fd438004d40 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.596+0000 7fd454b8d640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd438004d40 0x7fd4380051c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a800 tx=0x7fd438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea918666c5db0fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44800a450 con 0x7fd438004d40 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44802f070 con 0x7fd438004d40 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd44effd640 1 -- 172.21.15.135:0/2147460526 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44802bc90 con 0x7fd438004d40 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd4565f0640 1 -- 172.21.15.135:0/2147460526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd438004d40 msgr2=0x7fd4380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd4565f0640 1 --2- 172.21.15.135:0/2147460526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd438004d40 0x7fd4380051c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a800 tx=0x7fd438005830 comp rx=0 tx=0).stop 2024-04-26T22:00:40.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd454b8d640 1 -- 172.21.15.135:0/2147460526 reap_dead start 2024-04-26T22:00:40.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd4565f0640 1 -- 172.21.15.135:0/2147460526 shutdown_connections 2024-04-26T22:00:40.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd4565f0640 1 -- 172.21.15.135:0/2147460526 >> 172.21.15.135:0/2147460526 conn(0x7fd4500c5ce0 msgr2=0x7fd4500c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd4565f0640 1 -- 172.21.15.135:0/2147460526 shutdown_connections 2024-04-26T22:00:40.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.597+0000 7fd4565f0640 1 -- 172.21.15.135:0/2147460526 wait complete. 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.598+0000 7fd4565f0640 1 Processor -- start 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.598+0000 7fd4565f0640 1 -- start start 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd4565f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4501ecd00 0x7fd450159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd4565f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 0x7fd450155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd4565f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4501586f0 0x7fd450156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd4565f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd45006da30 con 0x7fd450154070 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd4565f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd450157240 con 0x7fd4501ecd00 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd4565f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd45006dbb0 con 0x7fd4501586f0 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 0x7fd450155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 0x7fd450155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55226/0 (socket says 172.21.15.135:55226) 2024-04-26T22:00:40.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd44f7fe640 1 -- 172.21.15.135:0/4074195750 learned_addr learned my addr 172.21.15.135:0/4074195750 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd44ffff640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4501ecd00 0x7fd450159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd454b8d640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4501586f0 0x7fd450156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd44f7fe640 1 -- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4501586f0 msgr2=0x7fd450156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.599+0000 7fd44f7fe640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4501586f0 0x7fd450156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44f7fe640 1 -- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4501ecd00 msgr2=0x7fd450159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44f7fe640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4501ecd00 0x7fd450159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44f7fe640 1 -- 172.21.15.135:0/4074195750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4501ed540 con 0x7fd450154070 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44ffff640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4501ecd00 0x7fd450159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44f7fe640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 0x7fd450155cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd44400f7c0 tx=0x7fd44400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4440052e0 con 0x7fd450154070 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd444018070 con 0x7fd450154070 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44400d910 con 0x7fd450154070 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4501ed820 con 0x7fd450154070 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.600+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4500adbd0 con 0x7fd450154070 2024-04-26T22:00:40.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.601+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4500c3160 con 0x7fd450154070 2024-04-26T22:00:40.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.601+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd44400dab0 con 0x7fd450154070 2024-04-26T22:00:40.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.602+0000 7fd44cff9640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd43c06c920 0x7fd43c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.602+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7fd44408be10 con 0x7fd450154070 2024-04-26T22:00:40.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.602+0000 7fd44ffff640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd43c06c920 0x7fd43c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.602+0000 7fd44ffff640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd43c06c920 0x7fd43c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd440009bc0 tx=0x7fd440008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.606+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4500c3160 con 0x7fd450154070 2024-04-26T22:00:40.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.782+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fd4500c3d60 con 0x7fd450154070 2024-04-26T22:00:40.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.782+0000 7fd44cff9640 1 -- 172.21.15.135:0/4074195750 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) ==== 95+0+11 (secure 0 0 0) 0x7fd4500c3d60 con 0x7fd450154070 2024-04-26T22:00:40.782 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd43c06c920 msgr2=0x7fd43c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd43c06c920 0x7fd43c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd440009bc0 tx=0x7fd440008040 comp rx=0 tx=0).stop 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 msgr2=0x7fd450155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 0x7fd450155cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd44400f7c0 tx=0x7fd44400fc80 comp rx=0 tx=0).stop 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 shutdown_connections 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd43c06c920 0x7fd43c06ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4501586f0 0x7fd450156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd450154070 0x7fd450155cc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 --2- 172.21.15.135:0/4074195750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4501ecd00 0x7fd450159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 >> 172.21.15.135:0/4074195750 conn(0x7fd4500c5ce0 msgr2=0x7fd4501575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 shutdown_connections 2024-04-26T22:00:40.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.784+0000 7fd4565f0640 1 -- 172.21.15.135:0/4074195750 wait complete. 2024-04-26T22:00:40.795 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:40.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.891+0000 7fc24ded1640 1 Processor -- start 2024-04-26T22:00:40.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc24ded1640 1 -- start start 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc24ded1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2480ca5a0 0x7fc248156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc24ded1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc24815a1a0 0x7fc2480c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc24ded1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2480c8df0 con 0x7fc248154160 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc24ded1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc2480c9100 con 0x7fc2480ca5a0 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc24ded1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc248154540 con 0x7fc24815a1a0 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc246ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2480ca5a0 0x7fc248156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc246ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2480ca5a0 0x7fc248156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55230/0 (socket says 172.21.15.135:55230) 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.892+0000 7fc246ffd640 1 -- 172.21.15.135:0/3684105258 learned_addr learned my addr 172.21.15.135:0/3684105258 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:40.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.893+0000 7fc247fff640 1 --2- 172.21.15.135:0/3684105258 >> v2:172.21.15.139:3300/0 conn(0x7fc24815a1a0 0x7fc2480c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.893+0000 7fc246ffd640 1 -- 172.21.15.135:0/3684105258 >> v2:172.21.15.139:3300/0 conn(0x7fc24815a1a0 msgr2=0x7fc2480c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.893+0000 7fc246ffd640 1 --2- 172.21.15.135:0/3684105258 >> v2:172.21.15.139:3300/0 conn(0x7fc24815a1a0 0x7fc2480c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.893+0000 7fc246ffd640 1 -- 172.21.15.135:0/3684105258 >> v1:172.21.15.139:6789/0 conn(0x7fc248154160 legacy=0x7fc2480c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.893+0000 7fc246ffd640 1 -- 172.21.15.135:0/3684105258 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24815a910 con 0x7fc2480ca5a0 2024-04-26T22:00:40.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.893+0000 7fc246ffd640 1 --2- 172.21.15.135:0/3684105258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2480ca5a0 0x7fc248156bb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc2480c9280 tx=0x7fc23c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b968a9c1a7222374 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc2467fc640 1 -- 172.21.15.135:0/3684105258 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c037070 con 0x7fc2480ca5a0 2024-04-26T22:00:40.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc2467fc640 1 -- 172.21.15.135:0/3684105258 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c032440 con 0x7fc2480ca5a0 2024-04-26T22:00:40.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc2467fc640 1 -- 172.21.15.135:0/3684105258 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c032810 con 0x7fc2480ca5a0 2024-04-26T22:00:40.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 -- 172.21.15.135:0/3684105258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2480ca5a0 msgr2=0x7fc248156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 --2- 172.21.15.135:0/3684105258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2480ca5a0 0x7fc248156bb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc2480c9280 tx=0x7fc23c02a210 comp rx=0 tx=0).stop 2024-04-26T22:00:40.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 -- 172.21.15.135:0/3684105258 shutdown_connections 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 --2- 172.21.15.135:0/3684105258 >> v2:172.21.15.139:3300/0 conn(0x7fc24815a1a0 0x7fc2480c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 --2- 172.21.15.135:0/3684105258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2480ca5a0 0x7fc248156bb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 -- 172.21.15.135:0/3684105258 >> 172.21.15.135:0/3684105258 conn(0x7fc2480ae010 msgr2=0x7fc2480ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 -- 172.21.15.135:0/3684105258 shutdown_connections 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.894+0000 7fc24ded1640 1 -- 172.21.15.135:0/3684105258 wait complete. 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.895+0000 7fc24ded1640 1 Processor -- start 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.895+0000 7fc24ded1640 1 -- start start 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc24ded1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2480ca5a0 0x7fc248159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc24ded1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 0x7fc248150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc24ded1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc24815a1a0 0x7fc2481f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc24ded1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc24815a5b0 con 0x7fc248154160 2024-04-26T22:00:40.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc24ded1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc24815aed0 con 0x7fc2480ca5a0 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc24ded1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc2480c7510 con 0x7fc24815a1a0 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 0x7fc248150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 0x7fc248150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55236/0 (socket says 172.21.15.135:55236) 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 -- 172.21.15.135:0/391419179 learned_addr learned my addr 172.21.15.135:0/391419179 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc2477fe640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2480ca5a0 0x7fc248159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc247fff640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc24815a1a0 0x7fc2481f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 -- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc24815a1a0 msgr2=0x7fc2481f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc24815a1a0 0x7fc2481f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 -- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2480ca5a0 msgr2=0x7fc248159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.896+0000 7fc246ffd640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2480ca5a0 0x7fc248159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc246ffd640 1 -- 172.21.15.135:0/391419179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc248153430 con 0x7fc248154160 2024-04-26T22:00:40.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc246ffd640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 0x7fc248150940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000c00 tx=0x7fc23c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c041020 con 0x7fc248154160 2024-04-26T22:00:40.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0306c0 con 0x7fc248154160 2024-04-26T22:00:40.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2481ed8f0 con 0x7fc248154160 2024-04-26T22:00:40.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c037040 con 0x7fc248154160 2024-04-26T22:00:40.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.897+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc248152f60 con 0x7fc248154160 2024-04-26T22:00:40.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.898+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc248153930 con 0x7fc248154160 2024-04-26T22:00:40.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.898+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc23c044020 con 0x7fc248154160 2024-04-26T22:00:40.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.899+0000 7fc22ffff640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc23406c8f0 0x7fc23406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.899+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7fc23c0ac650 con 0x7fc248154160 2024-04-26T22:00:40.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.899+0000 7fc2477fe640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc23406c8f0 0x7fc23406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.899+0000 7fc2477fe640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc23406c8f0 0x7fc23406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc238004f60 tx=0x7fc238007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:40.903+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc248153930 con 0x7fc248154160 2024-04-26T22:00:41.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.115+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc248076650 con 0x7fc248154160 2024-04-26T22:00:41.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.115+0000 7fc22ffff640 1 -- 172.21.15.135:0/391419179 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7fc23c04b030 con 0x7fc248154160 2024-04-26T22:00:41.115 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:41.116 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:00:41.116 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:00:41.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc23406c8f0 msgr2=0x7fc23406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc23406c8f0 0x7fc23406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc238004f60 tx=0x7fc238007e30 comp rx=0 tx=0).stop 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 msgr2=0x7fc248150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 0x7fc248150940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000c00 tx=0x7fc23c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 shutdown_connections 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc23406c8f0 0x7fc23406edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc24815a1a0 0x7fc2481f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc248154160 0x7fc248150940 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 --2- 172.21.15.135:0/391419179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2480ca5a0 0x7fc248159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 >> 172.21.15.135:0/391419179 conn(0x7fc2480ae010 msgr2=0x7fc2480c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 shutdown_connections 2024-04-26T22:00:41.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.117+0000 7fc24ded1640 1 -- 172.21.15.135:0/391419179 wait complete. 2024-04-26T22:00:41.128 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:00:41.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 Processor -- start 2024-04-26T22:00:41.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 -- start start 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15500c4820 0x7f1550073a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500ab630 0x7f1550073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1550156f60 con 0x7f15500aecf0 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1550157240 con 0x7f15500c4820 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.227+0000 7f1557a99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1550157520 con 0x7f15500ab630 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555034640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15500c4820 0x7f1550073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555835640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f15500aecf0 0x7f15500c3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:40874/0 (socket says 172.21.15.135:40874) 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555835640 1 -- 172.21.15.135:0/4084568168 learned_addr learned my addr 172.21.15.135:0/4084568168 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555034640 1 -- 172.21.15.135:0/4084568168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500ab630 msgr2=0x7f1550073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1556036640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500ab630 0x7f1550073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555034640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500ab630 0x7f1550073fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555034640 1 -- 172.21.15.135:0/4084568168 >> v1:172.21.15.135:6789/0 conn(0x7f15500aecf0 legacy=0x7f15500c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1554833640 1 -- 172.21.15.135:0/4084568168 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394351397 0 0) 0x7f1550156f60 con 0x7f15500aecf0 2024-04-26T22:00:41.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1555034640 1 -- 172.21.15.135:0/4084568168 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1550157800 con 0x7f15500c4820 2024-04-26T22:00:41.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.228+0000 7f1556036640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500ab630 0x7f1550073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1555034640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500c4820 0x7f1550073a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097a0 tx=0x7f154c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eacb78fa02006 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1554833640 1 -- 172.21.15.135:0/4084568168 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c037070 con 0x7f15500c4820 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1554833640 1 -- 172.21.15.135:0/4084568168 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c02ae50 con 0x7f15500c4820 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1554833640 1 -- 172.21.15.135:0/4084568168 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c032660 con 0x7f15500c4820 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 -- 172.21.15.135:0/4084568168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500c4820 msgr2=0x7f1550073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500c4820 0x7f1550073a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f154c0097a0 tx=0x7f154c02a100 comp rx=0 tx=0).stop 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 -- 172.21.15.135:0/4084568168 shutdown_connections 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500ab630 0x7f1550073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 --2- 172.21.15.135:0/4084568168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500c4820 0x7f1550073a70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 -- 172.21.15.135:0/4084568168 >> 172.21.15.135:0/4084568168 conn(0x7f15500c6ed0 msgr2=0x7f15500c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 -- 172.21.15.135:0/4084568168 shutdown_connections 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.229+0000 7f1557a99640 1 -- 172.21.15.135:0/4084568168 wait complete. 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.230+0000 7f1557a99640 1 Processor -- start 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.230+0000 7f1557a99640 1 -- start start 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1557a99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 0x7f15500c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1557a99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15500aecf0 0x7f155014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1557a99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500c4820 0x7f15501489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1557a99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15500c3f10 con 0x7f15500ab630 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1557a99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1550156c10 con 0x7f15500aecf0 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1557a99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f15500c4090 con 0x7f15500c4820 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1555835640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 0x7f15500c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1555835640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 0x7f15500c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55256/0 (socket says 172.21.15.135:55256) 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1555835640 1 -- 172.21.15.135:0/735299778 learned_addr learned my addr 172.21.15.135:0/735299778 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1555835640 1 -- 172.21.15.135:0/735299778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500c4820 msgr2=0x7f15501489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.231+0000 7f1555835640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500c4820 0x7f15501489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f1555835640 1 -- 172.21.15.135:0/735299778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15500aecf0 msgr2=0x7f155014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f1555835640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15500aecf0 0x7f155014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f1555835640 1 -- 172.21.15.135:0/735299778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15500c9510 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f1555835640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 0x7f15500c89a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f154000c500 tx=0x7f154000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540018070 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1540007580 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540004a20 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15500c97f0 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.232+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155015ae10 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.233+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15500c43a0 con 0x7f15500ab630 2024-04-26T22:00:41.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.233+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f154001b070 con 0x7f15500ab630 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.234+0000 7f15467fc640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153406c920 0x7f153406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.234+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f154008b1a0 con 0x7f15500ab630 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.234+0000 7f1555034640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153406c920 0x7f153406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.234+0000 7f1555034640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153406c920 0x7f153406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f154c034000 tx=0x7f154c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.238+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15500c43a0 con 0x7f15500ab630 2024-04-26T22:00:41.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.412+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15500507a0 con 0x7f15500ab630 2024-04-26T22:00:41.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.414+0000 7f15467fc640 1 -- 172.21.15.135:0/735299778 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+20688 (secure 0 0 0) 0x7f15400592a0 con 0x7f15500ab630 2024-04-26T22:00:41.414 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:00:41.414 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:00:38.936891+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153406c920 msgr2=0x7f153406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153406c920 0x7f153406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f154c034000 tx=0x7f154c02c000 comp rx=0 tx=0).stop 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 msgr2=0x7f15500c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 0x7f15500c89a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f154000c500 tx=0x7f154000c9c0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 shutdown_connections 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153406c920 0x7f153406ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15500c4820 0x7f15501489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15500aecf0 0x7f155014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 --2- 172.21.15.135:0/735299778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15500ab630 0x7f15500c89a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 >> 172.21.15.135:0/735299778 conn(0x7f15500c6ed0 msgr2=0x7f15500adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 shutdown_connections 2024-04-26T22:00:41.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:00:41.416+0000 7f1557a99640 1 -- 172.21.15.135:0/735299778 wait complete. 2024-04-26T22:00:41.427 DEBUG:tasks.mds_thrash:Yielding 2024-04-26T22:00:41.427 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T22:00:41.440 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:00:41.440 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T22:00:41.440 INFO:tasks.workunit:timeout=3h 2024-04-26T22:00:41.440 INFO:tasks.workunit:cleanup=True 2024-04-26T22:00:41.440 DEBUG:teuthology.orchestra.run.smithi135:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:41.441 INFO:tasks.mds_thrash.fs.[b]:starting mds_do_thrash for fs b 2024-04-26T22:00:41.441 INFO:tasks.mds_thrash.fs.[b]:waiting for 39 secs before thrashing 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-26 21:59:57.644042192 +0000 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-26 22:00:29.087261910 +0000 2024-04-26T22:00:41.456 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-26 21:59:57.644042192 +0000 2024-04-26T22:00:41.456 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T22:00:41.457 DEBUG:teuthology.orchestra.run.smithi135:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T22:00:41.524 DEBUG:teuthology.orchestra.run.smithi139:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:41.540 INFO:teuthology.orchestra.run.smithi139.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:41.540 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T22:00:41.540 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-26T22:00:41.540 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:00:41.541 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T22:00:41.541 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-26T22:00:41.541 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-26 22:00:06.802075865 +0000 2024-04-26T22:00:41.541 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-26 22:00:31.121520827 +0000 2024-04-26T22:00:41.541 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-26 22:00:06.802075865 +0000 2024-04-26T22:00:41.541 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-26T22:00:41.541 DEBUG:teuthology.orchestra.run.smithi139:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-26T22:00:41.611 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:00:41.612 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:00:41.627 INFO:tasks.workunit.client.0.smithi135.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T22:00:41.666 INFO:tasks.workunit.client.1.smithi139.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-26T22:00:57.773 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:00:57.776 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:01:12.989 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:13.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.102+0000 7f4c9b61d640 1 Processor -- start 2024-04-26T22:01:13.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c9b61d640 1 -- start start 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c9b61d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c940c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c9b61d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c940c52c0 0x7f4c941574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c9b61d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c940c3810 con 0x7f4c941570d0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c9b61d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4c940c4330 con 0x7f4c940c52c0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c9b61d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c940c3ab0 con 0x7f4c940ab3f0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c993b9640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4c941570d0 0x7f4c94150a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52328/0 (socket says 172.21.15.135:52328) 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c993b9640 1 -- 172.21.15.135:0/1585648233 learned_addr learned my addr 172.21.15.135:0/1585648233 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.103+0000 7f4c99bba640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c940c52c0 0x7f4c941574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c98bb8640 1 --2- 172.21.15.135:0/1585648233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c940c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c8bfff640 1 -- 172.21.15.135:0/1585648233 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239212089 0 0) 0x7f4c940c3810 con 0x7f4c941570d0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c8bfff640 1 -- 172.21.15.135:0/1585648233 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c80003520 con 0x7f4c941570d0 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c99bba640 1 -- 172.21.15.135:0/1585648233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 msgr2=0x7f4c940c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:13.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c99bba640 1 --2- 172.21.15.135:0/1585648233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c940c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c99bba640 1 -- 172.21.15.135:0/1585648233 >> v1:172.21.15.135:6789/0 conn(0x7f4c941570d0 legacy=0x7f4c94150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.104+0000 7f4c99bba640 1 -- 172.21.15.135:0/1585648233 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c940c4580 con 0x7f4c940c52c0 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c99bba640 1 --2- 172.21.15.135:0/1585648233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c941574b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4c940c44b0 tx=0x7f4c8c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbed696e6f97671c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c8bfff640 1 -- 172.21.15.135:0/1585648233 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c007740 con 0x7f4c940c52c0 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c8bfff640 1 -- 172.21.15.135:0/1585648233 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0078e0 con 0x7f4c940c52c0 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c8bfff640 1 -- 172.21.15.135:0/1585648233 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c007cb0 con 0x7f4c940c52c0 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 -- 172.21.15.135:0/1585648233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 msgr2=0x7f4c941574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/1585648233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c941574b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4c940c44b0 tx=0x7f4c8c02a530 comp rx=0 tx=0).stop 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 -- 172.21.15.135:0/1585648233 shutdown_connections 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/1585648233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c940c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/1585648233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c941574b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 -- 172.21.15.135:0/1585648233 >> 172.21.15.135:0/1585648233 conn(0x7f4c9406c310 msgr2=0x7f4c9406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 -- 172.21.15.135:0/1585648233 shutdown_connections 2024-04-26T22:01:13.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.105+0000 7f4c9b61d640 1 -- 172.21.15.135:0/1585648233 wait complete. 2024-04-26T22:01:13.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 Processor -- start 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 -- start start 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c94145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c94072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c941570d0 0x7f4c941ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941512e0 con 0x7f4c940c52c0 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94151160 con 0x7f4c941570d0 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c9b61d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c940aba60 con 0x7f4c940ab3f0 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c98bb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c94072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c98bb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c94072980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42338/0 (socket says 172.21.15.135:42338) 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.107+0000 7f4c98bb8640 1 -- 172.21.15.135:0/3687517520 learned_addr learned my addr 172.21.15.135:0/3687517520 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c993b9640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c94145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c99bba640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c941570d0 0x7f4c941ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c98bb8640 1 -- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 msgr2=0x7f4c94145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c98bb8640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c94145780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c98bb8640 1 -- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c941570d0 msgr2=0x7f4c941ee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c98bb8640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c941570d0 0x7f4c941ee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c98bb8640 1 -- 172.21.15.135:0/3687517520 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c94073fd0 con 0x7f4c940c52c0 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c98bb8640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c94072980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d800 tx=0x7f4c8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400b6c0 con 0x7f4c940c52c0 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c84019070 con 0x7f4c940c52c0 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c940c6080 con 0x7f4c940c52c0 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400ba80 con 0x7f4c940c52c0 2024-04-26T22:01:13.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.108+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c94074900 con 0x7f4c940c52c0 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.109+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c940507a0 con 0x7f4c940c52c0 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.110+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c8400bc20 con 0x7f4c940c52c0 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.110+0000 7f4c89ffb640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c7806c8f0 0x7f4c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.110+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f4c8408c390 con 0x7f4c940c52c0 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.110+0000 7f4c993b9640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c7806c8f0 0x7f4c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.110+0000 7f4c993b9640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c7806c8f0 0x7f4c7806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c90005880 tx=0x7f4c90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.114+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c8401c360 con 0x7f4c940c52c0 2024-04-26T22:01:13.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.324+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c940c4110 con 0x7f4c940c52c0 2024-04-26T22:01:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.325+0000 7f4c89ffb640 1 -- 172.21.15.135:0/3687517520 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f4c8405a490 con 0x7f4c940c52c0 2024-04-26T22:01:13.325 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:13.325 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c7806c8f0 msgr2=0x7f4c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c7806c8f0 0x7f4c7806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c90005880 tx=0x7f4c90009040 comp rx=0 tx=0).stop 2024-04-26T22:01:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 msgr2=0x7f4c94072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c94072980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d800 tx=0x7f4c8400dcc0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 shutdown_connections 2024-04-26T22:01:13.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4c7806c8f0 0x7f4c7806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c941570d0 0x7f4c941ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c940c52c0 0x7f4c94072980 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 --2- 172.21.15.135:0/3687517520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c940ab3f0 0x7f4c94145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 >> 172.21.15.135:0/3687517520 conn(0x7f4c9406c310 msgr2=0x7f4c9406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 shutdown_connections 2024-04-26T22:01:13.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.327+0000 7f4c9b61d640 1 -- 172.21.15.135:0/3687517520 wait complete. 2024-04-26T22:01:13.338 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-04-26T22:01:13.338 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:01:13.338 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:01:13.348 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:01:13.349 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.487+0000 7f32a22c4640 1 Processor -- start 2024-04-26T22:01:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.487+0000 7f32a22c4640 1 -- start start 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a22c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f329c0c4250 0x7f329c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a22c4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f329c0c3cd0 0x7f329c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a22c4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329c0ad010 con 0x7f329c0acc30 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a22c4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f329c0c8ba0 con 0x7f329c0c3cd0 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a22c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f329c156e50 con 0x7f329c0c4250 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a0861640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f329c0c3cd0 0x7f329c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f329bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f329c0acc30 0x7f329c15abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:52342/0 (socket says 172.21.15.135:52342) 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f329bfff640 1 -- 172.21.15.135:0/774484061 learned_addr learned my addr 172.21.15.135:0/774484061 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f329b7fe640 1 --2- 172.21.15.135:0/774484061 >> v2:172.21.15.139:3300/0 conn(0x7f329c0c4250 0x7f329c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a0861640 1 -- 172.21.15.135:0/774484061 >> v2:172.21.15.139:3300/0 conn(0x7f329c0c4250 msgr2=0x7f329c0c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a0861640 1 --2- 172.21.15.135:0/774484061 >> v2:172.21.15.139:3300/0 conn(0x7f329c0c4250 0x7f329c0c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a0861640 1 -- 172.21.15.135:0/774484061 >> v1:172.21.15.135:6789/0 conn(0x7f329c0acc30 legacy=0x7f329c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.488+0000 7f32a0861640 1 -- 172.21.15.135:0/774484061 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c0c8300 con 0x7f329c0c3cd0 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.489+0000 7f32a0861640 1 --2- 172.21.15.135:0/774484061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0c3cd0 0x7f329c1572e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f329c071a50 tx=0x7f329402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ae3930cba1a797 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.489+0000 7f329affd640 1 -- 172.21.15.135:0/774484061 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294007740 con 0x7f329c0c3cd0 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.489+0000 7f329affd640 1 -- 172.21.15.135:0/774484061 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32940078e0 con 0x7f329c0c3cd0 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.489+0000 7f329affd640 1 -- 172.21.15.135:0/774484061 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294007cb0 con 0x7f329c0c3cd0 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 -- 172.21.15.135:0/774484061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0c3cd0 msgr2=0x7f329c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 --2- 172.21.15.135:0/774484061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0c3cd0 0x7f329c1572e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f329c071a50 tx=0x7f329402a5f0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 -- 172.21.15.135:0/774484061 shutdown_connections 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 --2- 172.21.15.135:0/774484061 >> v2:172.21.15.139:3300/0 conn(0x7f329c0c4250 0x7f329c0c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 --2- 172.21.15.135:0/774484061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0c3cd0 0x7f329c1572e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 -- 172.21.15.135:0/774484061 >> 172.21.15.135:0/774484061 conn(0x7f329c0c6e60 msgr2=0x7f329c0c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 -- 172.21.15.135:0/774484061 shutdown_connections 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.490+0000 7f32a22c4640 1 -- 172.21.15.135:0/774484061 wait complete. 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.491+0000 7f32a22c4640 1 Processor -- start 2024-04-26T22:01:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.491+0000 7f32a22c4640 1 -- start start 2024-04-26T22:01:13.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a22c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 0x7f329c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a22c4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f329c0c3cd0 0x7f329c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a22c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329c0c4250 0x7f329c070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a22c4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0abec0 con 0x7f329c0acc30 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a22c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f329c0c88c0 con 0x7f329c0c4250 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a22c4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f329c0ac040 con 0x7f329c0c3cd0 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 0x7f329c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 0x7f329c06e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42360/0 (socket says 172.21.15.135:42360) 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 -- 172.21.15.135:0/4156695385 learned_addr learned my addr 172.21.15.135:0/4156695385 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329b7fe640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f329c0c3cd0 0x7f329c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a0861640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329c0c4250 0x7f329c070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 -- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f329c0c3cd0 msgr2=0x7f329c06e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f329c0c3cd0 0x7f329c06e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 -- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329c0c4250 msgr2=0x7f329c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329c0c4250 0x7f329c070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 -- 172.21.15.135:0/4156695385 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c151ae0 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f32a0861640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329c0c4250 0x7f329c070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f329bfff640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 0x7f329c06e0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f328c00c430 tx=0x7f328c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.492+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c017070 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.493+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328c005ce0 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.493+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c1ecf40 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.493+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0072f0 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.493+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c15a2f0 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.493+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c15a0e0 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.494+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f328c01a070 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.494+0000 7f3298ff9640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f328806c920 0x7f328806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.494+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f328c08b280 con 0x7f329c0acc30 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.494+0000 7f329b7fe640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f328806c920 0x7f328806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.495+0000 7f329b7fe640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f328806c920 0x7f328806ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f329c0c4690 tx=0x7f3290008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.499+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f328c058ca0 con 0x7f329c0acc30 2024-04-26T22:01:13.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.715+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f329c0c46f0 con 0x7f329c0acc30 2024-04-26T22:01:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.715+0000 7f3298ff9640 1 -- 172.21.15.135:0/4156695385 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f328c05cfa0 con 0x7f329c0acc30 2024-04-26T22:01:13.715 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:13.715 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f328806c920 msgr2=0x7f328806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f328806c920 0x7f328806ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f329c0c4690 tx=0x7f3290008040 comp rx=0 tx=0).stop 2024-04-26T22:01:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 msgr2=0x7f329c06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 0x7f329c06e0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f328c00c430 tx=0x7f328c00c8f0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 shutdown_connections 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f328806c920 0x7f328806ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329c0c4250 0x7f329c070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f329c0c3cd0 0x7f329c06e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 --2- 172.21.15.135:0/4156695385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f329c0acc30 0x7f329c06e0e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 >> 172.21.15.135:0/4156695385 conn(0x7f329c0c6e60 msgr2=0x7f329c0c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 shutdown_connections 2024-04-26T22:01:13.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:13.717+0000 7f32a22c4640 1 -- 172.21.15.135:0/4156695385 wait complete. 2024-04-26T22:01:13.728 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:01:15.728 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:15.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.838+0000 7f30d7093640 1 Processor -- start 2024-04-26T22:01:15.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.838+0000 7f30d7093640 1 -- start start 2024-04-26T22:01:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d7093640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f30d0154160 0x7f30d0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d7093640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d7093640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d00c8df0 con 0x7f30d00ca5a0 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d7093640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00c9100 con 0x7f30d0154160 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d7093640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f30d0154540 con 0x7f30d015a1a0 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f30d0154160 0x7f30d0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f30d0154160 0x7f30d0156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42366/0 (socket says 172.21.15.135:42366) 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 -- 172.21.15.135:0/4105478783 learned_addr learned my addr 172.21.15.135:0/4105478783 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d5630640 1 --2- 172.21.15.135:0/4105478783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 -- 172.21.15.135:0/4105478783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 msgr2=0x7f30d00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 --2- 172.21.15.135:0/4105478783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d00c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 -- 172.21.15.135:0/4105478783 >> v1:172.21.15.139:6789/0 conn(0x7f30d00ca5a0 legacy=0x7f30d00c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.839+0000 7f30d4e2f640 1 -- 172.21.15.135:0/4105478783 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d015a910 con 0x7f30d0154160 2024-04-26T22:01:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.840+0000 7f30d4e2f640 1 --2- 172.21.15.135:0/4105478783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d0154160 0x7f30d0156bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009870 tx=0x7f30c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2874f2cd8dfb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.840+0000 7f30cf7fe640 1 -- 172.21.15.135:0/4105478783 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4037070 con 0x7f30d0154160 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.840+0000 7f30cf7fe640 1 -- 172.21.15.135:0/4105478783 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c4032440 con 0x7f30d0154160 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.840+0000 7f30cf7fe640 1 -- 172.21.15.135:0/4105478783 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4032810 con 0x7f30d0154160 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 -- 172.21.15.135:0/4105478783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d0154160 msgr2=0x7f30d0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 --2- 172.21.15.135:0/4105478783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d0154160 0x7f30d0156bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009870 tx=0x7f30c402a210 comp rx=0 tx=0).stop 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 -- 172.21.15.135:0/4105478783 shutdown_connections 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 --2- 172.21.15.135:0/4105478783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 --2- 172.21.15.135:0/4105478783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d0154160 0x7f30d0156bb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 -- 172.21.15.135:0/4105478783 >> 172.21.15.135:0/4105478783 conn(0x7f30d00ae010 msgr2=0x7f30d00ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 -- 172.21.15.135:0/4105478783 shutdown_connections 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.841+0000 7f30d7093640 1 -- 172.21.15.135:0/4105478783 wait complete. 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.842+0000 7f30d7093640 1 Processor -- start 2024-04-26T22:01:15.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.842+0000 7f30d7093640 1 -- start start 2024-04-26T22:01:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d7093640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 0x7f30d00b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d7093640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d0154160 0x7f30d01526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d7093640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d0147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d7093640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30d015a5f0 con 0x7f30d00ca5a0 2024-04-26T22:01:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d7093640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f30d015aed0 con 0x7f30d0154160 2024-04-26T22:01:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d7093640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00c7510 con 0x7f30d015a1a0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 0x7f30d00b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 0x7f30d00b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42368/0 (socket says 172.21.15.135:42368) 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 -- 172.21.15.135:0/3374054620 learned_addr learned my addr 172.21.15.135:0/3374054620 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d5630640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30cffff640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d0154160 0x7f30d01526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 -- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 msgr2=0x7f30d0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d0147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 -- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d0154160 msgr2=0x7f30d01526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d0154160 0x7f30d01526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.843+0000 7f30d4e2f640 1 -- 172.21.15.135:0/3374054620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d01ed8f0 con 0x7f30d00ca5a0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.844+0000 7f30d4e2f640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 0x7f30d00b1bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009870 tx=0x7f30c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.844+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4041020 con 0x7f30d00ca5a0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.844+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c40306c0 con 0x7f30d00ca5a0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.844+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d0152f30 con 0x7f30d00ca5a0 2024-04-26T22:01:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.844+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4037040 con 0x7f30d00ca5a0 2024-04-26T22:01:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.844+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d01534f0 con 0x7f30d00ca5a0 2024-04-26T22:01:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.845+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d0076650 con 0x7f30d00ca5a0 2024-04-26T22:01:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.845+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30c4044020 con 0x7f30d00ca5a0 2024-04-26T22:01:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.846+0000 7f30cd7fa640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30b406c8f0 0x7f30b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.846+0000 7f30cffff640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30b406c8f0 0x7f30b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.846+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f30c40ac710 con 0x7f30d00ca5a0 2024-04-26T22:01:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.846+0000 7f30cffff640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30b406c8f0 0x7f30b406edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f30d014b9c0 tx=0x7f30bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:15.850+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30c404b030 con 0x7f30d00ca5a0 2024-04-26T22:01:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.064+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30d00507a0 con 0x7f30d00ca5a0 2024-04-26T22:01:16.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.064+0000 7f30cd7fa640 1 -- 172.21.15.135:0/3374054620 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f30d00c8df0 con 0x7f30d00ca5a0 2024-04-26T22:01:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:16.064 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:16.064 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30b406c8f0 msgr2=0x7f30b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30b406c8f0 0x7f30b406edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f30d014b9c0 tx=0x7f30bc009040 comp rx=0 tx=0).stop 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 msgr2=0x7f30d00b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 0x7f30d00b1bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009870 tx=0x7f30c40046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 shutdown_connections 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30b406c8f0 0x7f30b406edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d015a1a0 0x7f30d0147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:16.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d0154160 0x7f30d01526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:16.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 --2- 172.21.15.135:0/3374054620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30d00ca5a0 0x7f30d00b1bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:16.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 >> 172.21.15.135:0/3374054620 conn(0x7f30d00ae010 msgr2=0x7f30d00cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:16.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 shutdown_connections 2024-04-26T22:01:16.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:16.066+0000 7f30d7093640 1 -- 172.21.15.135:0/3374054620 wait complete. 2024-04-26T22:01:16.077 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:01:18.078 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:18.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.188+0000 7f910b22a640 1 Processor -- start 2024-04-26T22:01:18.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.188+0000 7f910b22a640 1 -- start start 2024-04-26T22:01:18.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f910b22a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91040ab3f0 0x7f91040c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f910b22a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f91041574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f910b22a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91040c3810 con 0x7f91041570d0 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f910b22a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f91040c56c0 con 0x7f91040ab3f0 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f910b22a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91040c4330 con 0x7f91040c52c0 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f91041574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f91041574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:47600/0 (socket says 172.21.15.135:47600) 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 -- 172.21.15.135:0/2956300942 learned_addr learned my addr 172.21.15.135:0/2956300942 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f9103fff640 1 --2- 172.21.15.135:0/2956300942 >> v2:172.21.15.139:3300/0 conn(0x7f91040ab3f0 0x7f91040c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 -- 172.21.15.135:0/2956300942 >> v2:172.21.15.139:3300/0 conn(0x7f91040ab3f0 msgr2=0x7f91040c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:18.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 --2- 172.21.15.135:0/2956300942 >> v2:172.21.15.139:3300/0 conn(0x7f91040ab3f0 0x7f91040c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 -- 172.21.15.135:0/2956300942 >> v1:172.21.15.139:6789/0 conn(0x7f91041570d0 legacy=0x7f9104150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f91097c7640 1 -- 172.21.15.135:0/2956300942 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91040c3ab0 con 0x7f91040c52c0 2024-04-26T22:01:18.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.189+0000 7f9103fff640 1 --2- 172.21.15.135:0/2956300942 >> v2:172.21.15.139:3300/0 conn(0x7f91040ab3f0 0x7f91040c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:18.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.190+0000 7f91097c7640 1 --2- 172.21.15.135:0/2956300942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f91041574b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f91040c44b0 tx=0x7f90fc02c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17140b7fedae2c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.190+0000 7f91037fe640 1 -- 172.21.15.135:0/2956300942 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc009600 con 0x7f91040c52c0 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.190+0000 7f91037fe640 1 -- 172.21.15.135:0/2956300942 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc0097a0 con 0x7f91040c52c0 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.190+0000 7f91037fe640 1 -- 172.21.15.135:0/2956300942 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc009b70 con 0x7f91040c52c0 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 -- 172.21.15.135:0/2956300942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 msgr2=0x7f91041574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 --2- 172.21.15.135:0/2956300942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f91041574b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f91040c44b0 tx=0x7f90fc02c7f0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 -- 172.21.15.135:0/2956300942 shutdown_connections 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 --2- 172.21.15.135:0/2956300942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f91041574b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 --2- 172.21.15.135:0/2956300942 >> v2:172.21.15.139:3300/0 conn(0x7f91040ab3f0 0x7f91040c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 -- 172.21.15.135:0/2956300942 >> 172.21.15.135:0/2956300942 conn(0x7f910406c310 msgr2=0x7f910406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 -- 172.21.15.135:0/2956300942 shutdown_connections 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.191+0000 7f910b22a640 1 -- 172.21.15.135:0/2956300942 wait complete. 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.192+0000 7f910b22a640 1 Processor -- start 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.192+0000 7f910b22a640 1 -- start start 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f910b22a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 0x7f91041ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f910b22a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f910414f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f910b22a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91041570d0 0x7f91041e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f910b22a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9104151160 con 0x7f91040ab3f0 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f910b22a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91040c4860 con 0x7f91041570d0 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f910b22a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91041512e0 con 0x7f91040c52c0 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 0x7f91041ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 0x7f91041ec3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42382/0 (socket says 172.21.15.135:42382) 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 -- 172.21.15.135:0/3116665375 learned_addr learned my addr 172.21.15.135:0/3116665375 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9103fff640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f910414f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f91097c7640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91041570d0 0x7f91041e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 -- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 msgr2=0x7f910414f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f910414f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 -- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91041570d0 msgr2=0x7f91041e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91041570d0 0x7f91041e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.193+0000 7f9108fc6640 1 -- 172.21.15.135:0/3116665375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91041ec910 con 0x7f91040ab3f0 2024-04-26T22:01:18.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.194+0000 7f9108fc6640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 0x7f91041ec3d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c3f0 tx=0x7f90f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.194+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80052d0 con 0x7f91040ab3f0 2024-04-26T22:01:18.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.194+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f8005490 con 0x7f91040ab3f0 2024-04-26T22:01:18.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.194+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8007260 con 0x7f91040ab3f0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.194+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9104151c10 con 0x7f91040ab3f0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.194+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041521d0 con 0x7f91040ab3f0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.195+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910406fa70 con 0x7f91040ab3f0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.195+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90f8007400 con 0x7f91040ab3f0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.196+0000 7f91017fa640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90e806c8f0 0x7f90e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.196+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f90f808b290 con 0x7f91040ab3f0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.196+0000 7f9103fff640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90e806c8f0 0x7f90e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.196+0000 7f9103fff640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90e806c8f0 0x7f90e806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009560 tx=0x7f90f0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.200+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90f8058d40 con 0x7f91040ab3f0 2024-04-26T22:01:18.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.412+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91040c4110 con 0x7f91040ab3f0 2024-04-26T22:01:18.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.412+0000 7f91017fa640 1 -- 172.21.15.135:0/3116665375 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f90f805d040 con 0x7f91040ab3f0 2024-04-26T22:01:18.413 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:18.413 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:18.413 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90e806c8f0 msgr2=0x7f90e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90e806c8f0 0x7f90e806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009560 tx=0x7f90f0009190 comp rx=0 tx=0).stop 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 msgr2=0x7f91041ec3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 0x7f91041ec3d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c3f0 tx=0x7f90f800c8b0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 shutdown_connections 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90e806c8f0 0x7f90e806edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91041570d0 0x7f91041e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91040c52c0 0x7f910414f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 --2- 172.21.15.135:0/3116665375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91040ab3f0 0x7f91041ec3d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 >> 172.21.15.135:0/3116665375 conn(0x7f910406c310 msgr2=0x7f910406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 shutdown_connections 2024-04-26T22:01:18.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:18.414+0000 7f910b22a640 1 -- 172.21.15.135:0/3116665375 wait complete. 2024-04-26T22:01:18.426 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:01:20.426 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:20.442 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:20.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.538+0000 7fead51c1640 1 Processor -- start 2024-04-26T22:01:20.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.538+0000 7fead51c1640 1 -- start start 2024-04-26T22:01:20.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7fead51c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead00ca5a0 0x7fead0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7fead51c1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead00c8df0 con 0x7fead015a1a0 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7fead51c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead0154540 con 0x7fead0154160 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7fead51c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fead00c9100 con 0x7fead00ca5a0 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feacf59e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fead015a1a0 0x7fead00c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44654/0 (socket says 172.21.15.135:44654) 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feacf59e640 1 -- 172.21.15.135:0/668138653 learned_addr learned my addr 172.21.15.135:0/668138653 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feace59c640 1 --2- 172.21.15.135:0/668138653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead00ca5a0 0x7fead0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3340800035 0 0) 0x7feac0001490 con 0x7fead015a1a0 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feab8003520 con 0x7fead015a1a0 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795501607 0 0) 0x7fead0154540 con 0x7fead0154160 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feac0001490 con 0x7fead0154160 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feace59c640 1 -- 172.21.15.135:0/668138653 >> v1:172.21.15.139:6789/0 conn(0x7fead0154160 legacy=0x7fead00c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.539+0000 7feace59c640 1 -- 172.21.15.135:0/668138653 >> v1:172.21.15.135:6789/0 conn(0x7fead015a1a0 legacy=0x7fead00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.540+0000 7feace59c640 1 -- 172.21.15.135:0/668138653 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead015a910 con 0x7fead00ca5a0 2024-04-26T22:01:20.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.540+0000 7feace59c640 1 --2- 172.21.15.135:0/668138653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead00ca5a0 0x7fead0156bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feac40097d0 tx=0x7feac402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6367a7294681fb0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:20.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.540+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4037070 con 0x7fead00ca5a0 2024-04-26T22:01:20.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.540+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac4032440 con 0x7fead00ca5a0 2024-04-26T22:01:20.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.540+0000 7feacdd9b640 1 -- 172.21.15.135:0/668138653 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40327f0 con 0x7fead00ca5a0 2024-04-26T22:01:20.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 -- 172.21.15.135:0/668138653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead00ca5a0 msgr2=0x7fead0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 --2- 172.21.15.135:0/668138653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead00ca5a0 0x7fead0156bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feac40097d0 tx=0x7feac402a210 comp rx=0 tx=0).stop 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 -- 172.21.15.135:0/668138653 shutdown_connections 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 --2- 172.21.15.135:0/668138653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead00ca5a0 0x7fead0156bb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 -- 172.21.15.135:0/668138653 >> 172.21.15.135:0/668138653 conn(0x7fead00ae010 msgr2=0x7fead00ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 -- 172.21.15.135:0/668138653 shutdown_connections 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.541+0000 7fead51c1640 1 -- 172.21.15.135:0/668138653 wait complete. 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 Processor -- start 2024-04-26T22:01:20.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 -- start start 2024-04-26T22:01:20.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fead00ca5a0 0x7fead0159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 0x7fead01efb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead015a1a0 0x7fead01f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fead015a630 con 0x7fead0154160 2024-04-26T22:01:20.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fead015aed0 con 0x7fead00ca5a0 2024-04-26T22:01:20.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7fead51c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fead00c7510 con 0x7fead015a1a0 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7feace59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 0x7fead01efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7feace59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 0x7fead01efb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51418/0 (socket says 172.21.15.135:51418) 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.543+0000 7feace59c640 1 -- 172.21.15.135:0/747649270 learned_addr learned my addr 172.21.15.135:0/747649270 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feaced9d640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fead00ca5a0 0x7fead0159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feacf59e640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead015a1a0 0x7fead01f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feace59c640 1 -- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead015a1a0 msgr2=0x7fead01f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feace59c640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead015a1a0 0x7fead01f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feace59c640 1 -- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fead00ca5a0 msgr2=0x7fead0159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feace59c640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fead00ca5a0 0x7fead0159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feace59c640 1 -- 172.21.15.135:0/747649270 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead014e660 con 0x7fead0154160 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feace59c640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 0x7fead01efb00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feac402a250 tx=0x7feac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4041020 con 0x7fead0154160 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac403d6e0 con 0x7fead0154160 2024-04-26T22:01:20.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead0149540 con 0x7fead0154160 2024-04-26T22:01:20.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4037040 con 0x7fead0154160 2024-04-26T22:01:20.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.544+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead00ad540 con 0x7fead0154160 2024-04-26T22:01:20.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.545+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead0076650 con 0x7fead0154160 2024-04-26T22:01:20.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.546+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feac4044020 con 0x7fead0154160 2024-04-26T22:01:20.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.546+0000 7feab77fe640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feabc06c920 0x7feabc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.546+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7feac40ac320 con 0x7fead0154160 2024-04-26T22:01:20.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.546+0000 7feaced9d640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feabc06c920 0x7feabc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.547+0000 7feaced9d640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feabc06c920 0x7feabc06ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feac80011e0 tx=0x7feac8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:20.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.551+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feac404b070 con 0x7fead0154160 2024-04-26T22:01:20.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.576+0000 7ff7fe714640 1 Processor -- start 2024-04-26T22:01:20.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.576+0000 7ff7fe714640 1 -- start start 2024-04-26T22:01:20.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7fe714640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7f815a280 0x7ff7f80c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7fe714640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f81585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7fe714640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f80c8a30 con 0x7ff7f813ffb0 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7fe714640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f8072be0 con 0x7ff7f815a280 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7fe714640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f80c9640 con 0x7ff7f8158180 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7f77fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7f815a280 0x7ff7f80c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7f77fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7f815a280 0x7ff7f80c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51430/0 (socket says 172.21.15.135:51430) 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7f77fe640 1 -- 172.21.15.135:0/3744575691 learned_addr learned my addr 172.21.15.135:0/3744575691 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7fccb1640 1 --2- 172.21.15.135:0/3744575691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f81585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7f77fe640 1 -- 172.21.15.135:0/3744575691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 msgr2=0x7ff7f81585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7f77fe640 1 --2- 172.21.15.135:0/3744575691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f81585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.577+0000 7ff7f77fe640 1 -- 172.21.15.135:0/3744575691 >> v1:172.21.15.139:6789/0 conn(0x7ff7f813ffb0 legacy=0x7ff7f8072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.578+0000 7ff7f77fe640 1 -- 172.21.15.135:0/3744575691 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f80c8d10 con 0x7ff7f815a280 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.578+0000 7ff7f77fe640 1 --2- 172.21.15.135:0/3744575691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f815a280 0x7ff7f80c82a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009870 tx=0x7ff7e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf290425df216d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.578+0000 7ff7f6ffd640 1 -- 172.21.15.135:0/3744575691 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8037070 con 0x7ff7f815a280 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.578+0000 7ff7f6ffd640 1 -- 172.21.15.135:0/3744575691 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e8032440 con 0x7ff7f815a280 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.578+0000 7ff7f6ffd640 1 -- 172.21.15.135:0/3744575691 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8032810 con 0x7ff7f815a280 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 -- 172.21.15.135:0/3744575691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f815a280 msgr2=0x7ff7f80c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 --2- 172.21.15.135:0/3744575691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f815a280 0x7ff7f80c82a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009870 tx=0x7ff7e802a1c0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 -- 172.21.15.135:0/3744575691 shutdown_connections 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 --2- 172.21.15.135:0/3744575691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f81585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 --2- 172.21.15.135:0/3744575691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f815a280 0x7ff7f80c82a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 -- 172.21.15.135:0/3744575691 >> 172.21.15.135:0/3744575691 conn(0x7ff7f806dc10 msgr2=0x7ff7f806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 -- 172.21.15.135:0/3744575691 shutdown_connections 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.579+0000 7ff7fe714640 1 -- 172.21.15.135:0/3744575691 wait complete. 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.580+0000 7ff7fe714640 1 Processor -- start 2024-04-26T22:01:20.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.580+0000 7ff7fe714640 1 -- start start 2024-04-26T22:01:20.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fe714640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 0x7ff7f813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fe714640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f8136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fe714640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f815a280 0x7ff7f81e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fe714640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f80c92d0 con 0x7ff7f813ffb0 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fe714640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f80c87e0 con 0x7ff7f815a280 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fe714640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f8073700 con 0x7ff7f8158180 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 0x7ff7f813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 0x7ff7f813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51440/0 (socket says 172.21.15.135:51440) 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 -- 172.21.15.135:0/1960001273 learned_addr learned my addr 172.21.15.135:0/1960001273 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f77fe640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f8136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fccb1640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f815a280 0x7ff7f81e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 -- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 msgr2=0x7ff7f8136f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f8136f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 -- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f815a280 msgr2=0x7ff7f81e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f815a280 0x7ff7f81e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 -- 172.21.15.135:0/1960001273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81ecc50 con 0x7ff7f813ffb0 2024-04-26T22:01:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7fccb1640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f815a280 0x7ff7f81e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.581+0000 7ff7f7fff640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 0x7ff7f813cd80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c430 tx=0x7ff7ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.582+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0052d0 con 0x7ff7f813ffb0 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.582+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec005490 con 0x7ff7f813ffb0 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.582+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec007260 con 0x7ff7f813ffb0 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.582+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f8135840 con 0x7ff7f813ffb0 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.582+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81e2f30 con 0x7ff7f813ffb0 2024-04-26T22:01:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.583+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f8076650 con 0x7ff7f813ffb0 2024-04-26T22:01:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.583+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7ec007400 con 0x7ff7f813ffb0 2024-04-26T22:01:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.584+0000 7ff7f4ff9640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff7e406c8f0 0x7ff7e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.584+0000 7ff7f77fe640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff7e406c8f0 0x7ff7e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.584+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7ff7ec08b330 con 0x7ff7f813ffb0 2024-04-26T22:01:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.584+0000 7ff7f77fe640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff7e406c8f0 0x7ff7e406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8034000 tx=0x7ff7e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:20.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.588+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7f81ecc50 con 0x7ff7f813ffb0 2024-04-26T22:01:20.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.772+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fead00cbcd0 con 0x7fead0154160 2024-04-26T22:01:20.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.772+0000 7feab77fe640 1 -- 172.21.15.135:0/747649270 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7fead014e660 con 0x7fead0154160 2024-04-26T22:01:20.772 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:20.772 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feabc06c920 msgr2=0x7feabc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feabc06c920 0x7feabc06ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feac80011e0 tx=0x7feac8009040 comp rx=0 tx=0).stop 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 msgr2=0x7fead01efb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 0x7fead01efb00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feac402a250 tx=0x7feac40046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 shutdown_connections 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7feabc06c920 0x7feabc06ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fead015a1a0 0x7fead01f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fead0154160 0x7fead01efb00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.774+0000 7fead51c1640 1 --2- 172.21.15.135:0/747649270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fead00ca5a0 0x7fead0159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.775+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 >> 172.21.15.135:0/747649270 conn(0x7fead00ae010 msgr2=0x7fead00cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.775+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 shutdown_connections 2024-04-26T22:01:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.775+0000 7fead51c1640 1 -- 172.21.15.135:0/747649270 wait complete. 2024-04-26T22:01:20.786 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:01:20.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.808+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f80cb100 con 0x7ff7f813ffb0 2024-04-26T22:01:20.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.808+0000 7ff7f4ff9640 1 -- 172.21.15.135:0/1960001273 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7ff7ec058de0 con 0x7ff7f813ffb0 2024-04-26T22:01:20.808 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:20.808 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:20.809 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:20.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff7e406c8f0 msgr2=0x7ff7e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff7e406c8f0 0x7ff7e406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8034000 tx=0x7ff7e802e000 comp rx=0 tx=0).stop 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 msgr2=0x7ff7f813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 0x7ff7f813cd80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c430 tx=0x7ff7ec00c8f0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 shutdown_connections 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff7e406c8f0 0x7ff7e406edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f815a280 0x7ff7f81e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7f8158180 0x7ff7f8136f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 --2- 172.21.15.135:0/1960001273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7f813ffb0 0x7ff7f813cd80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.810+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 >> 172.21.15.135:0/1960001273 conn(0x7ff7f806dc10 msgr2=0x7ff7f813dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.811+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 shutdown_connections 2024-04-26T22:01:20.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:20.811+0000 7ff7fe714640 1 -- 172.21.15.135:0/1960001273 wait complete. 2024-04-26T22:01:20.822 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-04-26T22:01:20.822 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:01:20.822 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:01:20.872 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:01:20.872 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.009+0000 7f1b3af24640 1 Processor -- start 2024-04-26T22:01:21.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.009+0000 7f1b3af24640 1 -- start start 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.010+0000 7f1b3af24640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b34154160 0x7f1b34156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.010+0000 7f1b3af24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3415a1a0 0x7f1b340c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.010+0000 7f1b3af24640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b340c8df0 con 0x7f1b340ca5a0 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.010+0000 7f1b3af24640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1b340c9100 con 0x7f1b34154160 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.010+0000 7f1b3af24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b34154540 con 0x7f1b3415a1a0 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.010+0000 7f1b394c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3415a1a0 0x7f1b340c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b34154160 0x7f1b34156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b34154160 0x7f1b34156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:37396/0 (socket says 172.21.15.135:37396) 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 -- 172.21.15.135:0/1083583199 learned_addr learned my addr 172.21.15.135:0/1083583199 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b33fff640 1 --1- 172.21.15.135:0/1083583199 >> v1:172.21.15.139:6789/0 conn(0x7f1b340ca5a0 0x7f1b340c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:48894/0 (socket says 172.21.15.135:48894) 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 -- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3415a1a0 msgr2=0x7f1b340c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:21.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 --2- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3415a1a0 0x7f1b340c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 -- 172.21.15.135:0/1083583199 >> v1:172.21.15.139:6789/0 conn(0x7f1b340ca5a0 legacy=0x7f1b340c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:21.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b38cc0640 1 -- 172.21.15.135:0/1083583199 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3415a910 con 0x7f1b34154160 2024-04-26T22:01:21.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b337fe640 1 -- 172.21.15.135:0/1083583199 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941509105 0 0) 0x7f1b28001110 con 0x7f1b340ca5a0 2024-04-26T22:01:21.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.011+0000 7f1b394c1640 1 --2- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3415a1a0 0x7f1b340c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b38cc0640 1 --2- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b34154160 0x7f1b34156bb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009870 tx=0x7f1b2402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d835f5a01c56069 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b337fe640 1 -- 172.21.15.135:0/1083583199 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24037070 con 0x7f1b34154160 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b337fe640 1 -- 172.21.15.135:0/1083583199 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b24032440 con 0x7f1b34154160 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b337fe640 1 -- 172.21.15.135:0/1083583199 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24032810 con 0x7f1b34154160 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 -- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b34154160 msgr2=0x7f1b34156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 --2- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b34154160 0x7f1b34156bb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009870 tx=0x7f1b2402a270 comp rx=0 tx=0).stop 2024-04-26T22:01:21.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 -- 172.21.15.135:0/1083583199 shutdown_connections 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 --2- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3415a1a0 0x7f1b340c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 --2- 172.21.15.135:0/1083583199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b34154160 0x7f1b34156bb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 -- 172.21.15.135:0/1083583199 >> 172.21.15.135:0/1083583199 conn(0x7f1b340ae010 msgr2=0x7f1b340ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 -- 172.21.15.135:0/1083583199 shutdown_connections 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.012+0000 7f1b3af24640 1 -- 172.21.15.135:0/1083583199 wait complete. 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.013+0000 7f1b3af24640 1 Processor -- start 2024-04-26T22:01:21.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.013+0000 7f1b3af24640 1 -- start start 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b3af24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b340ca5a0 0x7f1b34152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b3af24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 0x7f1b34153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b3af24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3415a1a0 0x7f1b3414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b3af24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3415a5f0 con 0x7f1b34154160 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b3af24640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3415aed0 con 0x7f1b3415a1a0 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b3af24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b340c7510 con 0x7f1b340ca5a0 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 0x7f1b34153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 0x7f1b34153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51452/0 (socket says 172.21.15.135:51452) 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.014+0000 7f1b33fff640 1 -- 172.21.15.135:0/3494053176 learned_addr learned my addr 172.21.15.135:0/3494053176 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b394c1640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3415a1a0 0x7f1b3414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b38cc0640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b340ca5a0 0x7f1b34152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b33fff640 1 -- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b340ca5a0 msgr2=0x7f1b34152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b33fff640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b340ca5a0 0x7f1b34152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b33fff640 1 -- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3415a1a0 msgr2=0x7f1b3414ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b33fff640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3415a1a0 0x7f1b3414ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b33fff640 1 -- 172.21.15.135:0/3494053176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3414c060 con 0x7f1b34154160 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b33fff640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 0x7f1b34153490 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c7b0 tx=0x7f1b2800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28018070 con 0x7f1b34154160 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b280072d0 con 0x7f1b34154160 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b341539d0 con 0x7f1b34154160 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280047e0 con 0x7f1b34154160 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.015+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3414d2f0 con 0x7f1b34154160 2024-04-26T22:01:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.016+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b34076650 con 0x7f1b34154160 2024-04-26T22:01:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.016+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b2801b070 con 0x7f1b34154160 2024-04-26T22:01:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.017+0000 7f1b317fa640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1b2006c8f0 0x7f1b2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.017+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f1b2808b1f0 con 0x7f1b34154160 2024-04-26T22:01:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.017+0000 7f1b38cc0640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1b2006c8f0 0x7f1b2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.017+0000 7f1b38cc0640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1b2006c8f0 0x7f1b2006edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b24034000 tx=0x7f1b2402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.021+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b340c8df0 con 0x7f1b34154160 2024-04-26T22:01:21.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.236+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b340507a0 con 0x7f1b34154160 2024-04-26T22:01:21.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.237+0000 7f1b317fa640 1 -- 172.21.15.135:0/3494053176 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f1b280592f0 con 0x7f1b34154160 2024-04-26T22:01:21.237 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:21.237 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:21.237 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.238+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1b2006c8f0 msgr2=0x7f1b2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.238+0000 7f1b3af24640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1b2006c8f0 0x7f1b2006edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b24034000 tx=0x7f1b2402e000 comp rx=0 tx=0).stop 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.238+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 msgr2=0x7f1b34153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.238+0000 7f1b3af24640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 0x7f1b34153490 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c7b0 tx=0x7f1b2800cc70 comp rx=0 tx=0).stop 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 shutdown_connections 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1b2006c8f0 0x7f1b2006edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3415a1a0 0x7f1b3414ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b34154160 0x7f1b34153490 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 --2- 172.21.15.135:0/3494053176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b340ca5a0 0x7f1b34152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:21.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 >> 172.21.15.135:0/3494053176 conn(0x7f1b340ae010 msgr2=0x7f1b340c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:21.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 shutdown_connections 2024-04-26T22:01:21.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:21.239+0000 7f1b3af24640 1 -- 172.21.15.135:0/3494053176 wait complete. 2024-04-26T22:01:21.250 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:22.787 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:22.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.896+0000 7f03561c9640 1 Processor -- start 2024-04-26T22:01:22.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.896+0000 7f03561c9640 1 -- start start 2024-04-26T22:01:22.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.896+0000 7f03561c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500aecf0 0x7f03500c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.896+0000 7f03561c9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f03500c4820 0x7f0350073a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f03561c9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03500c3d40 0x7f0350073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f03561c9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0350156f60 con 0x7f03500c3d40 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f03561c9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0350157240 con 0x7f03500c4820 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f03561c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0350157520 con 0x7f03500aecf0 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03500c3d40 0x7f0350073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03500c3d40 0x7f0350073fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51458/0 (socket says 172.21.15.135:51458) 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 -- 172.21.15.135:0/183363589 learned_addr learned my addr 172.21.15.135:0/183363589 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 -- 172.21.15.135:0/183363589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500aecf0 msgr2=0x7f03500c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 --2- 172.21.15.135:0/183363589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500aecf0 0x7f03500c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 -- 172.21.15.135:0/183363589 >> v2:172.21.15.139:3300/0 conn(0x7f03500c4820 msgr2=0x7f0350073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 --2- 172.21.15.135:0/183363589 >> v2:172.21.15.139:3300/0 conn(0x7f03500c4820 0x7f0350073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.897+0000 7f034ffff640 1 -- 172.21.15.135:0/183363589 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0350157800 con 0x7f03500c3d40 2024-04-26T22:01:22.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f034ffff640 1 --2- 172.21.15.135:0/183363589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350073fb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f03500cb370 tx=0x7f034802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d9c04c9673bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:22.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f034e7fc640 1 -- 172.21.15.135:0/183363589 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348036070 con 0x7f03500c3d40 2024-04-26T22:01:22.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f034e7fc640 1 -- 172.21.15.135:0/183363589 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0348031600 con 0x7f03500c3d40 2024-04-26T22:01:22.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f034e7fc640 1 -- 172.21.15.135:0/183363589 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03480319d0 con 0x7f03500c3d40 2024-04-26T22:01:22.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 -- 172.21.15.135:0/183363589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 msgr2=0x7f0350073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 --2- 172.21.15.135:0/183363589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350073fb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f03500cb370 tx=0x7f034802a610 comp rx=0 tx=0).stop 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 -- 172.21.15.135:0/183363589 shutdown_connections 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 --2- 172.21.15.135:0/183363589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350073fb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 --2- 172.21.15.135:0/183363589 >> v2:172.21.15.139:3300/0 conn(0x7f03500c4820 0x7f0350073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 --2- 172.21.15.135:0/183363589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500aecf0 0x7f03500c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 -- 172.21.15.135:0/183363589 >> 172.21.15.135:0/183363589 conn(0x7f03500c6ed0 msgr2=0x7f03500c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.898+0000 7f03561c9640 1 -- 172.21.15.135:0/183363589 shutdown_connections 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.899+0000 7f03561c9640 1 -- 172.21.15.135:0/183363589 wait complete. 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 Processor -- start 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 -- start start 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03500aecf0 0x7f03501560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500c4820 0x7f0350158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03500af0d0 con 0x7f03500c3d40 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0350156c10 con 0x7f03500aecf0 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.900+0000 7f03561c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f03500af250 con 0x7f03500c4820 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51474/0 (socket says 172.21.15.135:51474) 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 -- 172.21.15.135:0/2601785360 learned_addr learned my addr 172.21.15.135:0/2601785360 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034ffff640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500c4820 0x7f0350158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034f7fe640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03500aecf0 0x7f03501560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 -- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500c4820 msgr2=0x7f0350158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500c4820 0x7f0350158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 -- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03500aecf0 msgr2=0x7f03501560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03500aecf0 0x7f03501560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 -- 172.21.15.135:0/2601785360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501efa40 con 0x7f03500c3d40 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034ffff640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500c4820 0x7f0350158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:22.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f034effd640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350158000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f034000b410 tx=0x7f034000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:22.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340015020 con 0x7f03500c3d40 2024-04-26T22:01:22.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0340011850 con 0x7f03500c3d40 2024-04-26T22:01:22.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501efd20 con 0x7f03500c3d40 2024-04-26T22:01:22.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340011c40 con 0x7f03500c3d40 2024-04-26T22:01:22.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.901+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501f02e0 con 0x7f03500c3d40 2024-04-26T22:01:22.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.902+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03500507a0 con 0x7f03500c3d40 2024-04-26T22:01:22.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.903+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0340018020 con 0x7f03500c3d40 2024-04-26T22:01:22.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.903+0000 7f0337fff640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f033c06c920 0x7f033c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:22.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.903+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f034008bd60 con 0x7f03500c3d40 2024-04-26T22:01:22.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.903+0000 7f034f7fe640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f033c06c920 0x7f033c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:22.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.904+0000 7f034f7fe640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f033c06c920 0x7f033c06ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0350153850 tx=0x7f0344008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:22.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:22.908+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f034001f020 con 0x7f03500c3d40 2024-04-26T22:01:23.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.118+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f035015aa50 con 0x7f03500c3d40 2024-04-26T22:01:23.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.119+0000 7f0337fff640 1 -- 172.21.15.135:0/2601785360 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f03400172c0 con 0x7f03500c3d40 2024-04-26T22:01:23.119 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:23.119 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:23.119 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f033c06c920 msgr2=0x7f033c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f033c06c920 0x7f033c06ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0350153850 tx=0x7f0344008040 comp rx=0 tx=0).stop 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 msgr2=0x7f0350158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350158000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f034000b410 tx=0x7f034000b8d0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 shutdown_connections 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f033c06c920 0x7f033c06ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03500c4820 0x7f0350158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03500c3d40 0x7f0350158000 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 --2- 172.21.15.135:0/2601785360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03500aecf0 0x7f03501560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 >> 172.21.15.135:0/2601785360 conn(0x7f03500c6ed0 msgr2=0x7f0350151d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 shutdown_connections 2024-04-26T22:01:23.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.121+0000 7f03561c9640 1 -- 172.21.15.135:0/2601785360 wait complete. 2024-04-26T22:01:23.132 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:01:23.251 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:23.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.348+0000 7f5b8fe20640 1 Processor -- start 2024-04-26T22:01:23.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 -- start start 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b881510c0 0x7f5b88072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5b880734d0 0x7f5b8806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5b880c4160 0x7f5b88157070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5b880c50d0 con 0x7f5b880c4160 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5b880c4b50 con 0x7f5b880734d0 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8fe20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5b881514a0 con 0x7f5b881510c0 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8e3bd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5b880c4160 0x7f5b88157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8e3bd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5b880c4160 0x7f5b88157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51484/0 (socket says 172.21.15.135:51484) 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.349+0000 7f5b8e3bd640 1 -- 172.21.15.135:0/3581723354 learned_addr learned my addr 172.21.15.135:0/3581723354 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:23.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8d3bb640 1 --2- 172.21.15.135:0/3581723354 >> v2:172.21.15.139:3300/0 conn(0x7f5b880734d0 0x7f5b8806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8dbbc640 1 --2- 172.21.15.135:0/3581723354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b881510c0 0x7f5b88072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8e3bd640 1 -- 172.21.15.135:0/3581723354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b881510c0 msgr2=0x7f5b88072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8e3bd640 1 --2- 172.21.15.135:0/3581723354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b881510c0 0x7f5b88072f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8e3bd640 1 -- 172.21.15.135:0/3581723354 >> v2:172.21.15.139:3300/0 conn(0x7f5b880734d0 msgr2=0x7f5b8806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8e3bd640 1 --2- 172.21.15.135:0/3581723354 >> v2:172.21.15.139:3300/0 conn(0x7f5b880734d0 0x7f5b8806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8e3bd640 1 -- 172.21.15.135:0/3581723354 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b88154920 con 0x7f5b880c4160 2024-04-26T22:01:23.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.350+0000 7f5b8e3bd640 1 --2- 172.21.15.135:0/3581723354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b880c4160 0x7f5b88157070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5b8806de70 tx=0x7f5b8002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9710184ae74eefa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8cbba640 1 -- 172.21.15.135:0/3581723354 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80036070 con 0x7f5b880c4160 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8cbba640 1 -- 172.21.15.135:0/3581723354 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b80031600 con 0x7f5b880c4160 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8cbba640 1 -- 172.21.15.135:0/3581723354 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b800319d0 con 0x7f5b880c4160 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 -- 172.21.15.135:0/3581723354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b880c4160 msgr2=0x7f5b88157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/3581723354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b880c4160 0x7f5b88157070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5b8806de70 tx=0x7f5b8002a570 comp rx=0 tx=0).stop 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 -- 172.21.15.135:0/3581723354 shutdown_connections 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/3581723354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b880c4160 0x7f5b88157070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/3581723354 >> v2:172.21.15.139:3300/0 conn(0x7f5b880734d0 0x7f5b8806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/3581723354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b881510c0 0x7f5b88072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 -- 172.21.15.135:0/3581723354 >> 172.21.15.135:0/3581723354 conn(0x7f5b880c5ce0 msgr2=0x7f5b880c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 -- 172.21.15.135:0/3581723354 shutdown_connections 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.351+0000 7f5b8fe20640 1 -- 172.21.15.135:0/3581723354 wait complete. 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.352+0000 7f5b8fe20640 1 Processor -- start 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.352+0000 7f5b8fe20640 1 -- start start 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8fe20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b880c4160 0x7f5b881ed800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8fe20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 0x7f5b880ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8fe20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b88156ca0 0x7f5b8815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8fe20640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880c5920 con 0x7f5b881510c0 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8fe20640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880c5400 con 0x7f5b88156ca0 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8fe20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8806f500 con 0x7f5b880c4160 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 0x7f5b880ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 0x7f5b880ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51500/0 (socket says 172.21.15.135:51500) 2024-04-26T22:01:23.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 -- 172.21.15.135:0/1338758063 learned_addr learned my addr 172.21.15.135:0/1338758063 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8dbbc640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b880c4160 0x7f5b881ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8e3bd640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b88156ca0 0x7f5b8815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 -- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b880c4160 msgr2=0x7f5b881ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b880c4160 0x7f5b881ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 -- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b88156ca0 msgr2=0x7f5b8815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b88156ca0 0x7f5b8815d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 -- 172.21.15.135:0/1338758063 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b881edd40 con 0x7f5b881510c0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8e3bd640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b88156ca0 0x7f5b8815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.353+0000 7f5b8d3bb640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 0x7f5b880ada40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800d830 tx=0x7f5b7800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.354+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78018020 con 0x7f5b881510c0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.354+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78009660 con 0x7f5b881510c0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.354+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b880c7e00 con 0x7f5b881510c0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.354+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78009a30 con 0x7f5b881510c0 2024-04-26T22:01:23.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.354+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b880c83c0 con 0x7f5b881510c0 2024-04-26T22:01:23.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.355+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b880507a0 con 0x7f5b881510c0 2024-04-26T22:01:23.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.355+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7801b020 con 0x7f5b881510c0 2024-04-26T22:01:23.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.355+0000 7f5b7e7fc640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5b6c06c8f0 0x7f5b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:23.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.355+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f5b7808c2a0 con 0x7f5b881510c0 2024-04-26T22:01:23.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.356+0000 7f5b8dbbc640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5b6c06c8f0 0x7f5b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:23.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.356+0000 7f5b8dbbc640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5b6c06c8f0 0x7f5b6c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b840079a0 tx=0x7f5b84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:23.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.360+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b78022030 con 0x7f5b881510c0 2024-04-26T22:01:23.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.575+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b880c3ed0 con 0x7f5b881510c0 2024-04-26T22:01:23.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.575+0000 7f5b7e7fc640 1 -- 172.21.15.135:0/1338758063 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f5b7805a3a0 con 0x7f5b881510c0 2024-04-26T22:01:23.575 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:23.576 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:23.576 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5b6c06c8f0 msgr2=0x7f5b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5b6c06c8f0 0x7f5b6c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b840079a0 tx=0x7f5b84008040 comp rx=0 tx=0).stop 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 msgr2=0x7f5b880ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 0x7f5b880ada40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800d830 tx=0x7f5b7800dcf0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 shutdown_connections 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5b6c06c8f0 0x7f5b6c06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b88156ca0 0x7f5b8815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b881510c0 0x7f5b880ada40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 --2- 172.21.15.135:0/1338758063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b880c4160 0x7f5b881ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:23.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.577+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 >> 172.21.15.135:0/1338758063 conn(0x7f5b880c5ce0 msgr2=0x7f5b881516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:23.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.578+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 shutdown_connections 2024-04-26T22:01:23.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:23.578+0000 7f5b8fe20640 1 -- 172.21.15.135:0/1338758063 wait complete. 2024-04-26T22:01:23.589 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:25.133 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:25.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.242+0000 7f0891c86640 1 Processor -- start 2024-04-26T22:01:25.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.242+0000 7f0891c86640 1 -- start start 2024-04-26T22:01:25.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f0891c86640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f088c0c5050 0x7f088c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f0891c86640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f088c0c2b80 0x7f088c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f0891c86640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088c156c10 con 0x7f088c13feb0 2024-04-26T22:01:25.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f0891c86640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f088c15a8b0 con 0x7f088c0c2b80 2024-04-26T22:01:25.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f0891c86640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f088c157240 con 0x7f088c0c5050 2024-04-26T22:01:25.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f088c0c2b80 0x7f088c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f088c0c2b80 0x7f088c06d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51516/0 (socket says 172.21.15.135:51516) 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 -- 172.21.15.135:0/2843991947 learned_addr learned my addr 172.21.15.135:0/2843991947 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 -- 172.21.15.135:0/2843991947 >> v2:172.21.15.139:3300/0 conn(0x7f088c0c5050 msgr2=0x7f088c1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 --2- 172.21.15.135:0/2843991947 >> v2:172.21.15.139:3300/0 conn(0x7f088c0c5050 0x7f088c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 -- 172.21.15.135:0/2843991947 >> v1:172.21.15.139:6789/0 conn(0x7f088c13feb0 legacy=0x7f088c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.243+0000 7f088bfff640 1 -- 172.21.15.135:0/2843991947 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c0c5570 con 0x7f088c0c2b80 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.244+0000 7f088bfff640 1 --2- 172.21.15.135:0/2843991947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c2b80 0x7f088c06d320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f088c070760 tx=0x7f088402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e55640396eabc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.244+0000 7f088a7fc640 1 -- 172.21.15.135:0/2843991947 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884036070 con 0x7f088c0c2b80 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.244+0000 7f088a7fc640 1 -- 172.21.15.135:0/2843991947 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0884031600 con 0x7f088c0c2b80 2024-04-26T22:01:25.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.244+0000 7f088a7fc640 1 -- 172.21.15.135:0/2843991947 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840319d0 con 0x7f088c0c2b80 2024-04-26T22:01:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 -- 172.21.15.135:0/2843991947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c2b80 msgr2=0x7f088c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 --2- 172.21.15.135:0/2843991947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c2b80 0x7f088c06d320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f088c070760 tx=0x7f088402a5d0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 -- 172.21.15.135:0/2843991947 shutdown_connections 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 --2- 172.21.15.135:0/2843991947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c2b80 0x7f088c06d320 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 --2- 172.21.15.135:0/2843991947 >> v2:172.21.15.139:3300/0 conn(0x7f088c0c5050 0x7f088c1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 -- 172.21.15.135:0/2843991947 >> 172.21.15.135:0/2843991947 conn(0x7f088c0c5ce0 msgr2=0x7f088c0c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 -- 172.21.15.135:0/2843991947 shutdown_connections 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.245+0000 7f0891c86640 1 -- 172.21.15.135:0/2843991947 wait complete. 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.246+0000 7f0891c86640 1 Processor -- start 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.246+0000 7f0891c86640 1 -- start start 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0891c86640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088c0c2b80 0x7f088c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0891c86640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 0x7f088c1ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0891c86640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088c13feb0 0x7f088c155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0891c86640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f088c140290 con 0x7f088c0c5050 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0891c86640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f088c156ed0 con 0x7f088c0c2b80 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0891c86640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f088c140410 con 0x7f088c13feb0 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 0x7f088c1ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 0x7f088c1ecec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51528/0 (socket says 172.21.15.135:51528) 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 -- 172.21.15.135:0/1632327907 learned_addr learned my addr 172.21.15.135:0/1632327907 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088b7fe640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088c0c2b80 0x7f088c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088bfff640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088c13feb0 0x7f088c155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 -- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088c13feb0 msgr2=0x7f088c155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088c13feb0 0x7f088c155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 -- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088c0c2b80 msgr2=0x7f088c0c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088c0c2b80 0x7f088c0c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 -- 172.21.15.135:0/1632327907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c0c8840 con 0x7f088c0c5050 2024-04-26T22:01:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f088affd640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 0x7f088c1ecec0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f088000b4d0 tx=0x7f088000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.247+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880014070 con 0x7f088c0c5050 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.248+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0880010870 con 0x7f088c0c5050 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.248+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c0c8b20 con 0x7f088c0c5050 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.248+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880010c60 con 0x7f088c0c5050 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.248+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c073ce0 con 0x7f088c0c5050 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.248+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c0507a0 con 0x7f088c0c5050 2024-04-26T22:01:25.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.249+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0880017070 con 0x7f088c0c5050 2024-04-26T22:01:25.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.249+0000 7f0873fff640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f087806c8f0 0x7f087806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.249+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f088008bbb0 con 0x7f088c0c5050 2024-04-26T22:01:25.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.249+0000 7f088b7fe640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f087806c8f0 0x7f087806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.250+0000 7f088b7fe640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f087806c8f0 0x7f087806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f087c0017f0 tx=0x7f087c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:25.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.254+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0880059660 con 0x7f088c0c5050 2024-04-26T22:01:25.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.463+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f088c1576b0 con 0x7f088c0c5050 2024-04-26T22:01:25.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.464+0000 7f0873fff640 1 -- 172.21.15.135:0/1632327907 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7f088005d960 con 0x7f088c0c5050 2024-04-26T22:01:25.464 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:25.464 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:25.464 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f087806c8f0 msgr2=0x7f087806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f087806c8f0 0x7f087806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f087c0017f0 tx=0x7f087c009040 comp rx=0 tx=0).stop 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 msgr2=0x7f088c1ecec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 0x7f088c1ecec0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f088000b4d0 tx=0x7f088000b990 comp rx=0 tx=0).stop 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 shutdown_connections 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f087806c8f0 0x7f087806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088c13feb0 0x7f088c155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f088c0c5050 0x7f088c1ecec0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 --2- 172.21.15.135:0/1632327907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088c0c2b80 0x7f088c0c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 >> 172.21.15.135:0/1632327907 conn(0x7f088c0c5ce0 msgr2=0x7f088c13d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:25.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 shutdown_connections 2024-04-26T22:01:25.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.466+0000 7f0891c86640 1 -- 172.21.15.135:0/1632327907 wait complete. 2024-04-26T22:01:25.477 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:01:25.589 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:25.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.689+0000 7ff79f35f640 1 Processor -- start 2024-04-26T22:01:25.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.689+0000 7ff79f35f640 1 -- start start 2024-04-26T22:01:25.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79f35f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff798151df0 0x7ff7980c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79f35f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff79806fd10 0x7ff7980c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79f35f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7980c3120 con 0x7ff7980c8280 2024-04-26T22:01:25.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79f35f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff798154a80 con 0x7ff79806fd10 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79f35f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff798154d60 con 0x7ff798151df0 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff79806fd10 0x7ff7980c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff79806fd10 0x7ff7980c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51542/0 (socket says 172.21.15.135:51542) 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 -- 172.21.15.135:0/3391562805 learned_addr learned my addr 172.21.15.135:0/3391562805 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79d0fb640 1 --2- 172.21.15.135:0/3391562805 >> v2:172.21.15.139:3300/0 conn(0x7ff798151df0 0x7ff7980c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 -- 172.21.15.135:0/3391562805 >> v2:172.21.15.139:3300/0 conn(0x7ff798151df0 msgr2=0x7ff7980c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 --2- 172.21.15.135:0/3391562805 >> v2:172.21.15.139:3300/0 conn(0x7ff798151df0 0x7ff7980c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 -- 172.21.15.135:0/3391562805 >> v1:172.21.15.139:6789/0 conn(0x7ff7980c8280 legacy=0x7ff7980c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.690+0000 7ff79c8fa640 1 -- 172.21.15.135:0/3391562805 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff798155040 con 0x7ff79806fd10 2024-04-26T22:01:25.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.691+0000 7ff79c8fa640 1 --2- 172.21.15.135:0/3391562805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff7980c6a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff798075020 tx=0x7ff78802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b003f55267b0e316 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:25.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.691+0000 7ff78ffff640 1 -- 172.21.15.135:0/3391562805 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff79806fd10 2024-04-26T22:01:25.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.691+0000 7ff78ffff640 1 -- 172.21.15.135:0/3391562805 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff78802ae20 con 0x7ff79806fd10 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.691+0000 7ff78ffff640 1 -- 172.21.15.135:0/3391562805 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788032660 con 0x7ff79806fd10 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 -- 172.21.15.135:0/3391562805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 msgr2=0x7ff7980c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 --2- 172.21.15.135:0/3391562805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff7980c6a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff798075020 tx=0x7ff78802a110 comp rx=0 tx=0).stop 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 -- 172.21.15.135:0/3391562805 shutdown_connections 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 --2- 172.21.15.135:0/3391562805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff7980c6a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 --2- 172.21.15.135:0/3391562805 >> v2:172.21.15.139:3300/0 conn(0x7ff798151df0 0x7ff7980c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 -- 172.21.15.135:0/3391562805 >> 172.21.15.135:0/3391562805 conn(0x7ff7980c8c40 msgr2=0x7ff7980c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 -- 172.21.15.135:0/3391562805 shutdown_connections 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.692+0000 7ff79f35f640 1 -- 172.21.15.135:0/3391562805 wait complete. 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 Processor -- start 2024-04-26T22:01:25.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 -- start start 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff798152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7980c8280 0x7ff798153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798151df0 0x7ff798146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981521d0 con 0x7ff79806fd10 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff7980c76b0 con 0x7ff798151df0 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79f35f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff7980703c0 con 0x7ff7980c8280 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79d0fb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff798152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79d0fb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff798152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51546/0 (socket says 172.21.15.135:51546) 2024-04-26T22:01:25.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.693+0000 7ff79d0fb640 1 -- 172.21.15.135:0/2087081531 learned_addr learned my addr 172.21.15.135:0/2087081531 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79d0fb640 1 -- 172.21.15.135:0/2087081531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7980c8280 msgr2=0x7ff798153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79d0fb640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7980c8280 0x7ff798153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79d0fb640 1 -- 172.21.15.135:0/2087081531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798151df0 msgr2=0x7ff798146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79d0fb640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798151df0 0x7ff798146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79d0fb640 1 -- 172.21.15.135:0/2087081531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7980747f0 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79d0fb640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff798152cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff79400d5f0 tx=0x7ff79400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7940125f0 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff794019070 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7940129d0 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff798074ad0 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.694+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff798152480 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.695+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7980ab860 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.695+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff794012b70 con 0x7ff79806fd10 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.696+0000 7ff78dffb640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff77c06c920 0x7ff77c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:25.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.696+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7ff79408c240 con 0x7ff79806fd10 2024-04-26T22:01:25.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.696+0000 7ff79c8fa640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff77c06c920 0x7ff77c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:25.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.696+0000 7ff79c8fa640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff77c06c920 0x7ff77c06ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:25.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.700+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff79405a340 con 0x7ff79806fd10 2024-04-26T22:01:25.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.912+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff798076690 con 0x7ff79806fd10 2024-04-26T22:01:25.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.913+0000 7ff78dffb640 1 -- 172.21.15.135:0/2087081531 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6494 (secure 0 0 0) 0x7ff794059c40 con 0x7ff79806fd10 2024-04-26T22:01:25.913 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:25.913 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:00:00.110574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1410523687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1410523687},{"type":"v1","addr":"172.21.15.135:6837","nonce":1410523687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:25.913 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 13 2024-04-26T22:01:25.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff77c06c920 msgr2=0x7ff77c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff77c06c920 0x7ff77c06ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802c000 comp rx=0 tx=0).stop 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 msgr2=0x7ff798152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff798152cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff79400d5f0 tx=0x7ff79400dab0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 shutdown_connections 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff77c06c920 0x7ff77c06ede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798151df0 0x7ff798146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7980c8280 0x7ff798153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 --2- 172.21.15.135:0/2087081531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff79806fd10 0x7ff798152cf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 >> 172.21.15.135:0/2087081531 conn(0x7ff7980c8c40 msgr2=0x7ff7980a86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 shutdown_connections 2024-04-26T22:01:25.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:25.915+0000 7ff79f35f640 1 -- 172.21.15.135:0/2087081531 wait complete. 2024-04-26T22:01:25.926 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:27.478 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:27.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.586+0000 7f09b1225640 1 Processor -- start 2024-04-26T22:01:27.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.586+0000 7f09b1225640 1 -- start start 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.586+0000 7f09b1225640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f09ac0c3060 0x7f09ac0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.586+0000 7f09b1225640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f09ac0c40e0 0x7f09ac151e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09b1225640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ac154ba0 con 0x7f09ac152360 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09b1225640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f09ac154e80 con 0x7f09ac0c40e0 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09b1225640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f09ac155160 con 0x7f09ac0c3060 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09a3fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f09ac0c40e0 0x7f09ac151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09ab59e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f09ac152360 0x7f09ac0c46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44668/0 (socket says 172.21.15.135:44668) 2024-04-26T22:01:27.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09ab59e640 1 -- 172.21.15.135:0/2786938729 learned_addr learned my addr 172.21.15.135:0/2786938729 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:27.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09a3fff640 1 -- 172.21.15.135:0/2786938729 >> v2:172.21.15.139:3300/0 conn(0x7f09ac0c3060 msgr2=0x7f09ac0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:27.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09aad9d640 1 --2- 172.21.15.135:0/2786938729 >> v2:172.21.15.139:3300/0 conn(0x7f09ac0c3060 0x7f09ac0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09a3fff640 1 --2- 172.21.15.135:0/2786938729 >> v2:172.21.15.139:3300/0 conn(0x7f09ac0c3060 0x7f09ac0720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09aa59c640 1 -- 172.21.15.135:0/2786938729 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625698504 0 0) 0x7f09ac154ba0 con 0x7f09ac152360 2024-04-26T22:01:27.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09a3fff640 1 -- 172.21.15.135:0/2786938729 >> v1:172.21.15.135:6789/0 conn(0x7f09ac152360 legacy=0x7f09ac0c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.587+0000 7f09a3fff640 1 -- 172.21.15.135:0/2786938729 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac155440 con 0x7f09ac0c40e0 2024-04-26T22:01:27.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09a3fff640 1 --2- 172.21.15.135:0/2786938729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac151e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f09ac06d280 tx=0x7f099402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f06f610a802467 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09aa59c640 1 -- 172.21.15.135:0/2786938729 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994037070 con 0x7f09ac0c40e0 2024-04-26T22:01:27.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09aa59c640 1 -- 172.21.15.135:0/2786938729 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0994032440 con 0x7f09ac0c40e0 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09aa59c640 1 -- 172.21.15.135:0/2786938729 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099402ae60 con 0x7f09ac0c40e0 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 -- 172.21.15.135:0/2786938729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 msgr2=0x7f09ac151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 --2- 172.21.15.135:0/2786938729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac151e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f09ac06d280 tx=0x7f099402a110 comp rx=0 tx=0).stop 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 -- 172.21.15.135:0/2786938729 shutdown_connections 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 --2- 172.21.15.135:0/2786938729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac151e20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 --2- 172.21.15.135:0/2786938729 >> v2:172.21.15.139:3300/0 conn(0x7f09ac0c3060 0x7f09ac0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 -- 172.21.15.135:0/2786938729 >> 172.21.15.135:0/2786938729 conn(0x7f09ac06c310 msgr2=0x7f09ac06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 -- 172.21.15.135:0/2786938729 shutdown_connections 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.588+0000 7f09b1225640 1 -- 172.21.15.135:0/2786938729 wait complete. 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 Processor -- start 2024-04-26T22:01:27.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 -- start start 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09ac0c3060 0x7f09ac158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09ac152360 0x7f09ac144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac0c3440 con 0x7f09ac0c40e0 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac0c3aa0 con 0x7f09ac152360 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.590+0000 7f09b1225640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f09ac0c35c0 con 0x7f09ac0c3060 2024-04-26T22:01:27.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac0c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51552/0 (socket says 172.21.15.135:51552) 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 -- 172.21.15.135:0/374040635 learned_addr learned my addr 172.21.15.135:0/374040635 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09ab59e640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09ac152360 0x7f09ac144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09aad9d640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09ac0c3060 0x7f09ac158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 -- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09ac0c3060 msgr2=0x7f09ac158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09ac0c3060 0x7f09ac158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 -- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09ac152360 msgr2=0x7f09ac144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09ac152360 0x7f09ac144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 -- 172.21.15.135:0/374040635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac1eac90 con 0x7f09ac0c40e0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a3fff640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac0c5a70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0994009890 tx=0x7f09940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994037040 con 0x7f09ac0c40e0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0994042070 con 0x7f09ac0c40e0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994030a50 con 0x7f09ac0c40e0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac156e20 con 0x7f09ac0c40e0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.591+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1573e0 con 0x7f09ac0c40e0 2024-04-26T22:01:27.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.592+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac156c10 con 0x7f09ac0c40e0 2024-04-26T22:01:27.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.593+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0994030bf0 con 0x7f09ac0c40e0 2024-04-26T22:01:27.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.593+0000 7f09a37fe640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f098c06c8f0 0x7f098c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.593+0000 7f09aad9d640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f098c06c8f0 0x7f098c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.593+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6666+0+0 (secure 0 0 0) 0x7f09940ac8e0 con 0x7f09ac0c40e0 2024-04-26T22:01:27.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.594+0000 7f09aad9d640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f098c06c8f0 0x7f098c06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f099c006fd0 tx=0x7f099c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.598+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f099407a9e0 con 0x7f09ac0c40e0 2024-04-26T22:01:27.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.826+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09ac0507a0 con 0x7f09ac0c40e0 2024-04-26T22:01:27.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.826+0000 7f09a37fe640 1 -- 172.21.15.135:0/374040635 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5663 (secure 0 0 0) 0x7f099407a2e0 con 0x7f09ac0c40e0 2024-04-26T22:01:27.826 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:27.826 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:27.768622+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:27.827 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 14 2024-04-26T22:01:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f098c06c8f0 msgr2=0x7f098c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f098c06c8f0 0x7f098c06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f099c006fd0 tx=0x7f099c008040 comp rx=0 tx=0).stop 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 msgr2=0x7f09ac0c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac0c5a70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0994009890 tx=0x7f09940046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 shutdown_connections 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f098c06c8f0 0x7f098c06edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09ac152360 0x7f09ac144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09ac0c40e0 0x7f09ac0c5a70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 --2- 172.21.15.135:0/374040635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09ac0c3060 0x7f09ac158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 >> 172.21.15.135:0/374040635 conn(0x7f09ac06c310 msgr2=0x7f09ac074270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 shutdown_connections 2024-04-26T22:01:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.828+0000 7f09b1225640 1 -- 172.21.15.135:0/374040635 wait complete. 2024-04-26T22:01:27.840 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-04-26T22:01:27.840 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-04-26T22:01:27.840 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:27.927 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:27.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.939+0000 7fe5f3cbd640 1 Processor -- start 2024-04-26T22:01:27.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.939+0000 7fe5f3cbd640 1 -- start start 2024-04-26T22:01:27.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f3cbd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec0c3c70 0x7fe5ec074370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f3cbd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe5ec070c80 0x7fe5ec06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f3cbd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5ec072e60 con 0x7fe5ec150e90 2024-04-26T22:01:27.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f3cbd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe5ec06fc50 con 0x7fe5ec070c80 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f3cbd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ec154580 con 0x7fe5ec0c3c70 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f1a59640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe5ec150e90 0x7fe5ec06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44672/0 (socket says 172.21.15.135:44672) 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f1a59640 1 -- 172.21.15.135:0/2099336752 learned_addr learned my addr 172.21.15.135:0/2099336752 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f1258640 1 --2- 172.21.15.135:0/2099336752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec0c3c70 0x7fe5ec074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f225a640 1 --2- 172.21.15.135:0/2099336752 >> v2:172.21.15.139:3300/0 conn(0x7fe5ec070c80 0x7fe5ec06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f0a57640 1 -- 172.21.15.135:0/2099336752 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4095058149 0 0) 0x7fe5ec072e60 con 0x7fe5ec150e90 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.940+0000 7fe5f0a57640 1 -- 172.21.15.135:0/2099336752 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5d8003520 con 0x7fe5ec150e90 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.941+0000 7fe5f1258640 1 -- 172.21.15.135:0/2099336752 >> v2:172.21.15.139:3300/0 conn(0x7fe5ec070c80 msgr2=0x7fe5ec06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.941+0000 7fe5f1258640 1 --2- 172.21.15.135:0/2099336752 >> v2:172.21.15.139:3300/0 conn(0x7fe5ec070c80 0x7fe5ec06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.941+0000 7fe5f1258640 1 -- 172.21.15.135:0/2099336752 >> v1:172.21.15.135:6789/0 conn(0x7fe5ec150e90 legacy=0x7fe5ec06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.941 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.941+0000 7fe5f1258640 1 -- 172.21.15.135:0/2099336752 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec072810 con 0x7fe5ec0c3c70 2024-04-26T22:01:27.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.942+0000 7fe5f1258640 1 --2- 172.21.15.135:0/2099336752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec0c3c70 0x7fe5ec074370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0097a0 tx=0x7fe5dc02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead403735f10dce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.942+0000 7fe5f0a57640 1 -- 172.21.15.135:0/2099336752 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc037070 con 0x7fe5ec0c3c70 2024-04-26T22:01:27.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.942+0000 7fe5f0a57640 1 -- 172.21.15.135:0/2099336752 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5dc02adb0 con 0x7fe5ec0c3c70 2024-04-26T22:01:27.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.942+0000 7fe5f0a57640 1 -- 172.21.15.135:0/2099336752 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc032640 con 0x7fe5ec0c3c70 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/2099336752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec0c3c70 msgr2=0x7fe5ec074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/2099336752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec0c3c70 0x7fe5ec074370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0097a0 tx=0x7fe5dc02a060 comp rx=0 tx=0).stop 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/2099336752 shutdown_connections 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/2099336752 >> v2:172.21.15.139:3300/0 conn(0x7fe5ec070c80 0x7fe5ec06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/2099336752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec0c3c70 0x7fe5ec074370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/2099336752 >> 172.21.15.135:0/2099336752 conn(0x7fe5ec0c5d70 msgr2=0x7fe5ec0c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/2099336752 shutdown_connections 2024-04-26T22:01:27.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.943+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/2099336752 wait complete. 2024-04-26T22:01:27.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.944+0000 7fe5f3cbd640 1 Processor -- start 2024-04-26T22:01:27.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 -- start start 2024-04-26T22:01:27.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 0x7fe5ec075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5ec0c3c70 0x7fe5ec1ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec150e90 0x7fe5ec1de160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec151270 con 0x7fe5ec070c80 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec154230 con 0x7fe5ec0c3c70 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.945+0000 7fe5f3cbd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ec1513f0 con 0x7fe5ec150e90 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 0x7fe5ec075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 0x7fe5ec075960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51554/0 (socket says 172.21.15.135:51554) 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 -- 172.21.15.135:0/1340069747 learned_addr learned my addr 172.21.15.135:0/1340069747 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f225a640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec150e90 0x7fe5ec1de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1258640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5ec0c3c70 0x7fe5ec1ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 -- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec150e90 msgr2=0x7fe5ec1de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec150e90 0x7fe5ec1de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 -- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5ec0c3c70 msgr2=0x7fe5ec1ddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5ec0c3c70 0x7fe5ec1ddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 -- 172.21.15.135:0/1340069747 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec1ea840 con 0x7fe5ec070c80 2024-04-26T22:01:27.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f225a640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec150e90 0x7fe5ec1de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:27.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f1a59640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 0x7fe5ec075960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c430 tx=0x7fe5e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8017070 con 0x7fe5ec070c80 2024-04-26T22:01:27.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e8005ce0 con 0x7fe5ec070c80 2024-04-26T22:01:27.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ec1eab20 con 0x7fe5ec070c80 2024-04-26T22:01:27.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80072f0 con 0x7fe5ec070c80 2024-04-26T22:01:27.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.946+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ec1eb0e0 con 0x7fe5ec070c80 2024-04-26T22:01:27.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.947+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ec157c70 con 0x7fe5ec070c80 2024-04-26T22:01:27.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.948+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5e801a070 con 0x7fe5ec070c80 2024-04-26T22:01:27.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.948+0000 7fe5e27fc640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5d006c8f0 0x7fe5d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:27.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.949+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7fe5e808b4c0 con 0x7fe5ec070c80 2024-04-26T22:01:27.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.949+0000 7fe5f1258640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5d006c8f0 0x7fe5d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:27.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.949+0000 7fe5f1258640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5d006c8f0 0x7fe5d006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc034000 tx=0x7fe5dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:27.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:27.953+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5e8059460 con 0x7fe5ec070c80 2024-04-26T22:01:27.997 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:01:27.999 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:01:28.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.030+0000 7fb8ced24640 1 Processor -- start 2024-04-26T22:01:28.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.030+0000 7fb8ced24640 1 -- start start 2024-04-26T22:01:28.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ced24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8c806d590 0x7fb8c8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:28.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ced24640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb8c80af140 0x7fb8c80c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:28.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ced24640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c8154e10 con 0x7fb8c8070120 2024-04-26T22:01:28.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ced24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c81559e0 con 0x7fb8c806d590 2024-04-26T22:01:28.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ced24640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c8159240 con 0x7fb8c80af140 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8c806d590 0x7fb8c8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8c806d590 0x7fb8c8158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51566/0 (socket says 172.21.15.135:51566) 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 -- 172.21.15.135:0/2841494166 learned_addr learned my addr 172.21.15.135:0/2841494166 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8cd2c1640 1 --2- 172.21.15.135:0/2841494166 >> v2:172.21.15.139:3300/0 conn(0x7fb8c80af140 0x7fb8c80c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 -- 172.21.15.135:0/2841494166 >> v2:172.21.15.139:3300/0 conn(0x7fb8c80af140 msgr2=0x7fb8c80c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 --2- 172.21.15.135:0/2841494166 >> v2:172.21.15.139:3300/0 conn(0x7fb8c80af140 0x7fb8c80c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 -- 172.21.15.135:0/2841494166 >> v1:172.21.15.139:6789/0 conn(0x7fb8c8070120 legacy=0x7fb8c80aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.031+0000 7fb8ccac0640 1 -- 172.21.15.135:0/2841494166 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c81550b0 con 0x7fb8c806d590 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.032+0000 7fb8ccac0640 1 --2- 172.21.15.135:0/2841494166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c8158d00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80c6b30 tx=0x7fb8c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3710278faeeadb68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:28.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.032+0000 7fb8bf7fe640 1 -- 172.21.15.135:0/2841494166 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4037070 con 0x7fb8c806d590 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.032+0000 7fb8bf7fe640 1 -- 172.21.15.135:0/2841494166 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4032440 con 0x7fb8c806d590 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.032+0000 7fb8bf7fe640 1 -- 172.21.15.135:0/2841494166 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4032810 con 0x7fb8c806d590 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.032+0000 7fb8ced24640 1 -- 172.21.15.135:0/2841494166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 msgr2=0x7fb8c8158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.032+0000 7fb8ced24640 1 --2- 172.21.15.135:0/2841494166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c8158d00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80c6b30 tx=0x7fb8c402a1e0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.033+0000 7fb8ced24640 1 -- 172.21.15.135:0/2841494166 shutdown_connections 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.033+0000 7fb8ced24640 1 --2- 172.21.15.135:0/2841494166 >> v2:172.21.15.139:3300/0 conn(0x7fb8c80af140 0x7fb8c80c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.033+0000 7fb8ced24640 1 --2- 172.21.15.135:0/2841494166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c8158d00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.033+0000 7fb8ced24640 1 -- 172.21.15.135:0/2841494166 >> 172.21.15.135:0/2841494166 conn(0x7fb8c80c5ce0 msgr2=0x7fb8c80c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.033+0000 7fb8ced24640 1 -- 172.21.15.135:0/2841494166 shutdown_connections 2024-04-26T22:01:28.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.033+0000 7fb8ced24640 1 -- 172.21.15.135:0/2841494166 wait complete. 2024-04-26T22:01:28.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.034+0000 7fb8ced24640 1 Processor -- start 2024-04-26T22:01:28.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.034+0000 7fb8ced24640 1 -- start start 2024-04-26T22:01:28.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ced24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c81e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:28.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ced24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8c8070120 0x7fb8c80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:28.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ced24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c80af140 0x7fb8c81ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ced24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c806d970 con 0x7fb8c806d590 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ced24640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8155670 con 0x7fb8c80af140 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ced24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c806daf0 con 0x7fb8c8070120 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c81e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c81e1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51574/0 (socket says 172.21.15.135:51574) 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 -- 172.21.15.135:0/1082500740 learned_addr learned my addr 172.21.15.135:0/1082500740 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8cd2c1640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c80af140 0x7fb8c81ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8bffff640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8c8070120 0x7fb8c80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 -- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8c8070120 msgr2=0x7fb8c80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8c8070120 0x7fb8c80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 -- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c80af140 msgr2=0x7fb8c81ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c80af140 0x7fb8c81ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 -- 172.21.15.135:0/1082500740 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c81e1550 con 0x7fb8c806d590 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8cd2c1640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c80af140 0x7fb8c81ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:28.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.036+0000 7fb8ccac0640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c81e1ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402a220 tx=0x7fb8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.037+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4037040 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.037+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4042070 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.037+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c814d740 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.037+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4030880 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.037+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c80ae320 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.038+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8c4030a20 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.038+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c80c2ca0 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.038+0000 7fb8bd7fa640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8b406c8f0 0x7fb8b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.038+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7fb8c40ac5c0 con 0x7fb8c806d590 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.039+0000 7fb8bffff640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8b406c8f0 0x7fb8b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:28.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.039+0000 7fb8bffff640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8b406c8f0 0x7fb8b406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8158300 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:28.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.043+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8c407a020 con 0x7fb8c806d590 2024-04-26T22:01:28.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.201+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5ec0aed50 con 0x7fe5ec070c80 2024-04-26T22:01:28.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.201+0000 7fe5e27fc640 1 -- 172.21.15.135:0/1340069747 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5663 (secure 0 0 0) 0x7fe5e8058d60 con 0x7fe5ec070c80 2024-04-26T22:01:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:28.201 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:27.768622+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:28.202 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 14 2024-04-26T22:01:28.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5d006c8f0 msgr2=0x7fe5d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5d006c8f0 0x7fe5d006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc034000 tx=0x7fe5dc02e000 comp rx=0 tx=0).stop 2024-04-26T22:01:28.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 msgr2=0x7fe5ec075960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 0x7fe5ec075960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c430 tx=0x7fe5e800c8f0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 shutdown_connections 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5d006c8f0 0x7fe5d006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5ec150e90 0x7fe5ec1de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5ec0c3c70 0x7fe5ec1ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 --2- 172.21.15.135:0/1340069747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5ec070c80 0x7fe5ec075960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.203+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 >> 172.21.15.135:0/1340069747 conn(0x7fe5ec0c5d70 msgr2=0x7fe5ec06d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.204+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 shutdown_connections 2024-04-26T22:01:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.204+0000 7fe5f3cbd640 1 -- 172.21.15.135:0/1340069747 wait complete. 2024-04-26T22:01:28.215 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 14, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 14, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4244": { "addr": "172.21.15.139:6835/2608264407", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 2608264407, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 2608264407, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 14, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:01:27.768622+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4244 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 13, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4275": { "addr": "172.21.15.135:6836/3981978490", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 3981978490, "type": "v2" }, { "addr": "172.21.15.135:6836", "nonce": 3981978490, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:00:10.177793+0000", "qdb_cluster": [ 4275 ], "qdb_leader": 4275, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4275 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3183513686", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3183513686, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3183513686, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:01:28.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.280+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c80507a0 con 0x7fb8c806d590 2024-04-26T22:01:28.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.281+0000 7fb8bd7fa640 1 -- 172.21.15.135:0/1082500740 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5663 (secure 0 0 0) 0x7fb8c407e320 con 0x7fb8c806d590 2024-04-26T22:01:28.281 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:28.281 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:27.768622+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:28.281 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 14 2024-04-26T22:01:28.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8b406c8f0 msgr2=0x7fb8b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8b406c8f0 0x7fb8b406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8158300 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2024-04-26T22:01:28.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 msgr2=0x7fb8c81e1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:28.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c81e1ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402a220 tx=0x7fb8c40046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 shutdown_connections 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8b406c8f0 0x7fb8b406edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8c80af140 0x7fb8c81ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8c8070120 0x7fb8c80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 --2- 172.21.15.135:0/1082500740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c806d590 0x7fb8c81e1ce0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 >> 172.21.15.135:0/1082500740 conn(0x7fb8c80c5ce0 msgr2=0x7fb8c8154250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 shutdown_connections 2024-04-26T22:01:28.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:28.283+0000 7fb8ced24640 1 -- 172.21.15.135:0/1082500740 wait complete. 2024-04-26T22:01:28.294 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:30.215 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:30.295 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:30.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.317+0000 7fcd319af640 1 Processor -- start 2024-04-26T22:01:30.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.317+0000 7fcd319af640 1 -- start start 2024-04-26T22:01:30.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd319af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd2c0747d0 0x7fcd2c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd319af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd2c0708c0 con 0x7fcd2c13c370 2024-04-26T22:01:30.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd319af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd2c070ba0 con 0x7fcd2c06d860 2024-04-26T22:01:30.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd319af640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcd2c073da0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd2affd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fcd2c13c370 0x7fcd2c0c54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51348/0 (socket says 172.21.15.135:51348) 2024-04-26T22:01:30.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd2affd640 1 -- 172.21.15.135:0/113196248 learned_addr learned my addr 172.21.15.135:0/113196248 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:30.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd23fff640 1 --2- 172.21.15.135:0/113196248 >> v2:172.21.15.139:3300/0 conn(0x7fcd2c0747d0 0x7fcd2c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd2a7fc640 1 -- 172.21.15.135:0/113196248 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633392287 0 0) 0x7fcd2c0708c0 con 0x7fcd2c13c370 2024-04-26T22:01:30.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd2a7fc640 1 -- 172.21.15.135:0/113196248 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd18003520 con 0x7fcd2c13c370 2024-04-26T22:01:30.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd23fff640 1 -- 172.21.15.135:0/113196248 >> v1:172.21.15.139:6789/0 conn(0x7fcd2c06d860 legacy=0x7fcd2c153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd23fff640 1 -- 172.21.15.135:0/113196248 >> v1:172.21.15.135:6789/0 conn(0x7fcd2c13c370 legacy=0x7fcd2c0c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.318+0000 7fcd23fff640 1 -- 172.21.15.135:0/113196248 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c070e80 con 0x7fcd2c0747d0 2024-04-26T22:01:30.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.319+0000 7fcd23fff640 1 --2- 172.21.15.135:0/113196248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c06d320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd140097a0 tx=0x7fcd1402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307e19da0baebc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:30.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.319+0000 7fcd2a7fc640 1 -- 172.21.15.135:0/113196248 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14037070 con 0x7fcd2c0747d0 2024-04-26T22:01:30.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.319+0000 7fcd2a7fc640 1 -- 172.21.15.135:0/113196248 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd14032440 con 0x7fcd2c0747d0 2024-04-26T22:01:30.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.319+0000 7fcd2a7fc640 1 -- 172.21.15.135:0/113196248 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140327f0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 -- 172.21.15.135:0/113196248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 msgr2=0x7fcd2c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 --2- 172.21.15.135:0/113196248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c06d320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd140097a0 tx=0x7fcd1402a230 comp rx=0 tx=0).stop 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 -- 172.21.15.135:0/113196248 shutdown_connections 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 --2- 172.21.15.135:0/113196248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c06d320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 -- 172.21.15.135:0/113196248 >> 172.21.15.135:0/113196248 conn(0x7fcd2c0c91e0 msgr2=0x7fcd2c0c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 -- 172.21.15.135:0/113196248 shutdown_connections 2024-04-26T22:01:30.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.320+0000 7fcd319af640 1 -- 172.21.15.135:0/113196248 wait complete. 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.321+0000 7fcd319af640 1 Processor -- start 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.321+0000 7fcd319af640 1 -- start start 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd319af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 0x7fcd2c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd319af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd319af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd2c13c370 0x7fcd2c1edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd319af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c13c750 con 0x7fcd2c06d860 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd319af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c0742b0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd319af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2c072740 con 0x7fcd2c13c370 2024-04-26T22:01:30.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd2affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 0x7fcd2c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd2affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 0x7fcd2c1509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33336/0 (socket says 172.21.15.135:33336) 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:58332/0 (socket says 172.21.15.135:58332) 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd2affd640 1 -- 172.21.15.135:0/1997370769 learned_addr learned my addr 172.21.15.135:0/1997370769 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd2b7fe640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd2c13c370 0x7fcd2c1edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 -- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd2c13c370 msgr2=0x7fcd2c1edaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd2c13c370 0x7fcd2c1edaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 -- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 msgr2=0x7fcd2c1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 0x7fcd2c1509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.322+0000 7fcd23fff640 1 -- 172.21.15.135:0/1997370769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c13b820 con 0x7fcd2c0747d0 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd2affd640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 0x7fcd2c1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:30.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd23fff640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c150ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0ca250 tx=0x7fcd140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:30.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14037050 con 0x7fcd2c0747d0 2024-04-26T22:01:30.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd14038040 con 0x7fcd2c0747d0 2024-04-26T22:01:30.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14004430 con 0x7fcd2c0747d0 2024-04-26T22:01:30.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c155410 con 0x7fcd2c0747d0 2024-04-26T22:01:30.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.323+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1559d0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.324+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c0696c0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.324+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd1403c6d0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.325+0000 7fcd28ff9640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0c06c920 0x7fcd0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.325+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7fcd140ac770 con 0x7fcd2c0747d0 2024-04-26T22:01:30.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.325+0000 7fcd2affd640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0c06c920 0x7fcd0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.325+0000 7fcd2affd640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0c06c920 0x7fcd0c06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0011e0 tx=0x7fcd1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:30.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.329+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd1407a150 con 0x7fcd2c0747d0 2024-04-26T22:01:30.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.397+0000 7f4712797640 1 Processor -- start 2024-04-26T22:01:30.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.397+0000 7f4712797640 1 -- start start 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4712797640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f470c156bb0 0x7f470c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4712797640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f470c0c8df0 con 0x7f470c154160 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4712797640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f470c154540 con 0x7f470c0ca5a0 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4712797640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f470c15a6e0 con 0x7f470c156bb0 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f470bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f470c154160 0x7f470c0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51352/0 (socket says 172.21.15.135:51352) 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f470bfff640 1 -- 172.21.15.135:0/807968987 learned_addr learned my addr 172.21.15.135:0/807968987 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4710d34640 1 --2- 172.21.15.135:0/807968987 >> v2:172.21.15.139:3300/0 conn(0x7f470c156bb0 0x7f470c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712881328 0 0) 0x7f46fc001110 con 0x7f470c154160 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46f4003520 con 0x7f470c154160 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769510142 0 0) 0x7f470c154540 con 0x7f470c0ca5a0 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46fc001110 con 0x7f470c0ca5a0 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4710d34640 1 -- 172.21.15.135:0/807968987 >> v1:172.21.15.139:6789/0 conn(0x7f470c0ca5a0 legacy=0x7f470c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4710d34640 1 -- 172.21.15.135:0/807968987 >> v1:172.21.15.135:6789/0 conn(0x7f470c154160 legacy=0x7f470c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.398+0000 7f4710d34640 1 -- 172.21.15.135:0/807968987 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c15a910 con 0x7f470c156bb0 2024-04-26T22:01:30.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4710d34640 1 --2- 172.21.15.135:0/807968987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c156bb0 0x7f470c15a1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4704009d90 tx=0x7f470402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b91ac98636e3b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704007740 con 0x7f470c156bb0 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47040078e0 con 0x7f470c156bb0 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f470affd640 1 -- 172.21.15.135:0/807968987 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704007cb0 con 0x7f470c156bb0 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 -- 172.21.15.135:0/807968987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c156bb0 msgr2=0x7f470c15a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 --2- 172.21.15.135:0/807968987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c156bb0 0x7f470c15a1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4704009d90 tx=0x7f470402a6f0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 -- 172.21.15.135:0/807968987 shutdown_connections 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 --2- 172.21.15.135:0/807968987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c156bb0 0x7f470c15a1a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 -- 172.21.15.135:0/807968987 >> 172.21.15.135:0/807968987 conn(0x7f470c0ae010 msgr2=0x7f470c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 -- 172.21.15.135:0/807968987 shutdown_connections 2024-04-26T22:01:30.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.400+0000 7f4712797640 1 -- 172.21.15.135:0/807968987 wait complete. 2024-04-26T22:01:30.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.401+0000 7f4712797640 1 Processor -- start 2024-04-26T22:01:30.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 -- start start 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 0x7f470c0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 0x7f470c1e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 0x7f470c1e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f470c15aed0 con 0x7f470c156bb0 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f470c0c9100 con 0x7f470c154160 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4712797640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f470c0c7510 con 0x7f470c0ca5a0 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f470bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 0x7f470c0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4710d34640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 0x7f470c1e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f470b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 0x7f470c1e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f470bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 0x7f470c0c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:57124/0 (socket says 172.21.15.135:57124) 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f4710d34640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 0x7f470c1e97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33350/0 (socket says 172.21.15.135:33350) 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f470bfff640 1 -- 172.21.15.135:0/2078126383 learned_addr learned my addr 172.21.15.135:0/2078126383 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.402+0000 7f470b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 0x7f470c1e92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:58348/0 (socket says 172.21.15.135:58348) 2024-04-26T22:01:30.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4710d34640 1 -- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 msgr2=0x7f470c0c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4710d34640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 0x7f470c0c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4710d34640 1 -- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 msgr2=0x7f470c1e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4710d34640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 0x7f470c1e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4710d34640 1 -- 172.21.15.135:0/2078126383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c1f01c0 con 0x7f470c156bb0 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f470bfff640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 0x7f470c0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f470b7fe640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 0x7f470c1e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4710d34640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 0x7f470c1e97e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4704031000 tx=0x7f4704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470403f070 con 0x7f470c156bb0 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c1f0460 con 0x7f470c156bb0 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4704031c00 con 0x7f470c156bb0 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704009450 con 0x7f470c156bb0 2024-04-26T22:01:30.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.403+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c0c6100 con 0x7f470c156bb0 2024-04-26T22:01:30.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.404+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c076650 con 0x7f470c156bb0 2024-04-26T22:01:30.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.405+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4704003990 con 0x7f470c156bb0 2024-04-26T22:01:30.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.406+0000 7f4708ff9640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46f806c920 0x7f46f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:30.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.406+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7f47040ad4b0 con 0x7f470c156bb0 2024-04-26T22:01:30.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.406+0000 7f470bfff640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46f806c920 0x7f46f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:30.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.406+0000 7f470bfff640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46f806c920 0x7f46f806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000f60 tx=0x7f46fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:30.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.415+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f470407ae90 con 0x7f470c156bb0 2024-04-26T22:01:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.564+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd2c0507a0 con 0x7fcd2c0747d0 2024-04-26T22:01:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.565+0000 7fcd28ff9640 1 -- 172.21.15.135:0/1997370769 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5664 (secure 0 0 0) 0x7fcd1407e450 con 0x7fcd2c0747d0 2024-04-26T22:01:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:30.565 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:28.775771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 16 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.568+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0c06c920 msgr2=0x7fcd0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.568+0000 7fcd319af640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0c06c920 0x7fcd0c06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0011e0 tx=0x7fcd1c009040 comp rx=0 tx=0).stop 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.568+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 msgr2=0x7fcd2c150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.568+0000 7fcd319af640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c150ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0ca250 tx=0x7fcd140046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 shutdown_connections 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0c06c920 0x7fcd0c06ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd2c13c370 0x7fcd2c1edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd2c0747d0 0x7fcd2c150ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 --2- 172.21.15.135:0/1997370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2c06d860 0x7fcd2c1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 >> 172.21.15.135:0/1997370769 conn(0x7fcd2c0c91e0 msgr2=0x7fcd2c138450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 shutdown_connections 2024-04-26T22:01:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.569+0000 7fcd319af640 1 -- 172.21.15.135:0/1997370769 wait complete. 2024-04-26T22:01:30.583 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:01:30.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.658+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f470c0507a0 con 0x7f470c156bb0 2024-04-26T22:01:30.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.659+0000 7f4708ff9640 1 -- 172.21.15.135:0/2078126383 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5664 (secure 0 0 0) 0x7f470407f190 con 0x7f470c156bb0 2024-04-26T22:01:30.659 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:30.659 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:28.775771+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:30.659 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 16 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46f806c920 msgr2=0x7f46f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46f806c920 0x7f46f806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000f60 tx=0x7f46fc009040 comp rx=0 tx=0).stop 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 msgr2=0x7f470c1e97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 0x7f470c1e97e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4704031000 tx=0x7f4704004a40 comp rx=0 tx=0).stop 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 shutdown_connections 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46f806c920 0x7f46f806ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c156bb0 0x7f470c1e97e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f470c154160 0x7f470c1e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 --2- 172.21.15.135:0/2078126383 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f470c0ca5a0 0x7f470c0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:30.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 >> 172.21.15.135:0/2078126383 conn(0x7f470c0ae010 msgr2=0x7f470c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:30.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 shutdown_connections 2024-04-26T22:01:30.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:30.661+0000 7f4712797640 1 -- 172.21.15.135:0/2078126383 wait complete. 2024-04-26T22:01:30.672 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:31.623 INFO:tasks.ceph.mds.d.smithi139.stderr:2024-04-26T22:01:31.623+0000 7f4b0912a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:01:32.584 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:32.673 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.681+0000 7f942e2ea640 1 Processor -- start 2024-04-26T22:01:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.681+0000 7f942e2ea640 1 -- start start 2024-04-26T22:01:32.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942e2ea640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f942806f8a0 0x7f94280c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942e2ea640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9428158980 0x7f942806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942e2ea640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9428154a90 con 0x7f942814eb50 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942e2ea640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f94280c5080 con 0x7f9428158980 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942e2ea640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9428155930 con 0x7f942806f8a0 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942c887640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9428158980 0x7f942806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942c887640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9428158980 0x7f942806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33356/0 (socket says 172.21.15.135:33356) 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f942c887640 1 -- 172.21.15.135:0/2597238173 learned_addr learned my addr 172.21.15.135:0/2597238173 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.682+0000 7f94277fe640 1 --2- 172.21.15.135:0/2597238173 >> v2:172.21.15.139:3300/0 conn(0x7f942806f8a0 0x7f94280c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.683+0000 7f942c887640 1 -- 172.21.15.135:0/2597238173 >> v2:172.21.15.139:3300/0 conn(0x7f942806f8a0 msgr2=0x7f94280c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.683+0000 7f942c887640 1 --2- 172.21.15.135:0/2597238173 >> v2:172.21.15.139:3300/0 conn(0x7f942806f8a0 0x7f94280c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.683+0000 7f942c887640 1 -- 172.21.15.135:0/2597238173 >> v1:172.21.15.139:6789/0 conn(0x7f942814eb50 legacy=0x7f9428155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.683+0000 7f942c887640 1 -- 172.21.15.135:0/2597238173 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942806ff90 con 0x7f9428158980 2024-04-26T22:01:32.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.683+0000 7f942c887640 1 --2- 172.21.15.135:0/2597238173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f942806d320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f94280c6e30 tx=0x7f942002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fba520aa416e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f9426ffd640 1 -- 172.21.15.135:0/2597238173 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420036070 con 0x7f9428158980 2024-04-26T22:01:32.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f9426ffd640 1 -- 172.21.15.135:0/2597238173 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420031600 con 0x7f9428158980 2024-04-26T22:01:32.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f9426ffd640 1 -- 172.21.15.135:0/2597238173 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200319d0 con 0x7f9428158980 2024-04-26T22:01:32.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 -- 172.21.15.135:0/2597238173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 msgr2=0x7f942806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2597238173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f942806d320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f94280c6e30 tx=0x7f942002a730 comp rx=0 tx=0).stop 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 -- 172.21.15.135:0/2597238173 shutdown_connections 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2597238173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f942806d320 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2597238173 >> v2:172.21.15.139:3300/0 conn(0x7f942806f8a0 0x7f94280c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 -- 172.21.15.135:0/2597238173 >> 172.21.15.135:0/2597238173 conn(0x7f94280c5d70 msgr2=0x7f94280c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 -- 172.21.15.135:0/2597238173 shutdown_connections 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.684+0000 7f942e2ea640 1 -- 172.21.15.135:0/2597238173 wait complete. 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.685+0000 7f942e2ea640 1 Processor -- start 2024-04-26T22:01:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 -- start start 2024-04-26T22:01:32.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f942806f8a0 0x7f942814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f942814eb50 0x7f94280ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f94281517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9428158db0 con 0x7f9428158980 2024-04-26T22:01:32.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9428154d00 con 0x7f942814eb50 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942e2ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9428070550 con 0x7f942806f8a0 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942c887640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f94281517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942c887640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f94281517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33368/0 (socket says 172.21.15.135:33368) 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f942c887640 1 -- 172.21.15.135:0/2133036495 learned_addr learned my addr 172.21.15.135:0/2133036495 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f94277fe640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f942814eb50 0x7f94280ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.686+0000 7f9427fff640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f942806f8a0 0x7f942814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942c887640 1 -- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f942806f8a0 msgr2=0x7f942814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942c887640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f942806f8a0 0x7f942814f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942c887640 1 -- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f942814eb50 msgr2=0x7f94280ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942c887640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f942814eb50 0x7f94280ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942c887640 1 -- 172.21.15.135:0/2133036495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94280c76d0 con 0x7f9428158980 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942c887640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f94281517b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9420034040 tx=0x7f9420004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420036050 con 0x7f9428158980 2024-04-26T22:01:32.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420042070 con 0x7f9428158980 2024-04-26T22:01:32.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94280c7940 con 0x7f9428158980 2024-04-26T22:01:32.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.687+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94280addb0 con 0x7f9428158980 2024-04-26T22:01:32.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.688+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420009510 con 0x7f9428158980 2024-04-26T22:01:32.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.689+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94280ac7f0 con 0x7f9428158980 2024-04-26T22:01:32.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.689+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9420007c70 con 0x7f9428158980 2024-04-26T22:01:32.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.689+0000 7f9424ff9640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f941406c920 0x7f941406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.690+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7f94200acb00 con 0x7f9428158980 2024-04-26T22:01:32.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.690+0000 7f9427fff640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f941406c920 0x7f941406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.690+0000 7f9427fff640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f941406c920 0x7f941406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94180016e0 tx=0x7f9418009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.694+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94280ac7f0 con 0x7f9428158980 2024-04-26T22:01:32.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.774+0000 7fe2cb4d9640 1 Processor -- start 2024-04-26T22:01:32.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.774+0000 7fe2cb4d9640 1 -- start start 2024-04-26T22:01:32.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2cb4d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe2c4154160 0x7fe2c4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2cb4d9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe2c40ca5a0 0x7fe2c415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2cb4d9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c40c6e00 0x7fe2c40c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2cb4d9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c415a6e0 con 0x7fe2c4154160 2024-04-26T22:01:32.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2cb4d9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c40c9100 con 0x7fe2c40ca5a0 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2cb4d9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c40c8df0 con 0x7fe2c40c6e00 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe2c4154160 0x7fe2c4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe2c4154160 0x7fe2c4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33378/0 (socket says 172.21.15.135:33378) 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 -- 172.21.15.135:0/101372666 learned_addr learned my addr 172.21.15.135:0/101372666 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9a76640 1 --2- 172.21.15.135:0/101372666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c40c6e00 0x7fe2c40c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c8a74640 1 --2- 172.21.15.135:0/101372666 >> v2:172.21.15.139:3300/0 conn(0x7fe2c40ca5a0 0x7fe2c415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 -- 172.21.15.135:0/101372666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c40c6e00 msgr2=0x7fe2c40c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 --2- 172.21.15.135:0/101372666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c40c6e00 0x7fe2c40c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 -- 172.21.15.135:0/101372666 >> v2:172.21.15.139:3300/0 conn(0x7fe2c40ca5a0 msgr2=0x7fe2c415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 --2- 172.21.15.135:0/101372666 >> v2:172.21.15.139:3300/0 conn(0x7fe2c40ca5a0 0x7fe2c415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.775+0000 7fe2c9275640 1 -- 172.21.15.135:0/101372666 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c4154540 con 0x7fe2c4154160 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.776+0000 7fe2c9275640 1 --2- 172.21.15.135:0/101372666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c4154160 0x7fe2c4156bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4069220 tx=0x7fe2c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dfc5e23cf3e03b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.776+0000 7fe2bbfff640 1 -- 172.21.15.135:0/101372666 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0037070 con 0x7fe2c4154160 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.776+0000 7fe2bbfff640 1 -- 172.21.15.135:0/101372666 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0032440 con 0x7fe2c4154160 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.776+0000 7fe2bbfff640 1 -- 172.21.15.135:0/101372666 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0032810 con 0x7fe2c4154160 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/101372666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c4154160 msgr2=0x7fe2c4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/101372666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c4154160 0x7fe2c4156bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4069220 tx=0x7fe2c002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/101372666 shutdown_connections 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/101372666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c40c6e00 0x7fe2c40c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/101372666 >> v2:172.21.15.139:3300/0 conn(0x7fe2c40ca5a0 0x7fe2c415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/101372666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c4154160 0x7fe2c4156bb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/101372666 >> 172.21.15.135:0/101372666 conn(0x7fe2c40ae010 msgr2=0x7fe2c40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:32.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/101372666 shutdown_connections 2024-04-26T22:01:32.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.777+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/101372666 wait complete. 2024-04-26T22:01:32.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.779+0000 7fe2cb4d9640 1 Processor -- start 2024-04-26T22:01:32.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.779+0000 7fe2cb4d9640 1 -- start start 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2cb4d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 0x7fe2c4152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2cb4d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2c40ca5a0 0x7fe2c41532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2cb4d9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c4154160 0x7fe2c4147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2cb4d9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c415aed0 con 0x7fe2c40c6e00 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2cb4d9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c415abf0 con 0x7fe2c40ca5a0 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2cb4d9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c40c9590 con 0x7fe2c4154160 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9275640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 0x7fe2c4152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c8a74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2c40ca5a0 0x7fe2c41532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9275640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 0x7fe2c4152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33390/0 (socket says 172.21.15.135:33390) 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9275640 1 -- 172.21.15.135:0/331252781 learned_addr learned my addr 172.21.15.135:0/331252781 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9a76640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c4154160 0x7fe2c4147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9a76640 1 -- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2c40ca5a0 msgr2=0x7fe2c41532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9a76640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2c40ca5a0 0x7fe2c41532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9a76640 1 -- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 msgr2=0x7fe2c4152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9a76640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 0x7fe2c4152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9a76640 1 -- 172.21.15.135:0/331252781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c4151e70 con 0x7fe2c4154160 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c8a74640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2c40ca5a0 0x7fe2c41532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.780+0000 7fe2c9275640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 0x7fe2c4152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.781+0000 7fe2c9a76640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c4154160 0x7fe2c4147820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009d90 tx=0x7fe2bc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.781+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc018020 con 0x7fe2c4154160 2024-04-26T22:01:32.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.781+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc014480 con 0x7fe2c4154160 2024-04-26T22:01:32.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.781+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c4159d10 con 0x7fe2c4154160 2024-04-26T22:01:32.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.781+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc014870 con 0x7fe2c4154160 2024-04-26T22:01:32.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.781+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c4152ad0 con 0x7fe2c4154160 2024-04-26T22:01:32.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.782+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c4159ee0 con 0x7fe2c4154160 2024-04-26T22:01:32.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.783+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2bc005d10 con 0x7fe2c4154160 2024-04-26T22:01:32.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.783+0000 7fe2b9ffb640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe2a806c8f0 0x7fe2a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:32.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.784+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7fe2bc08cbb0 con 0x7fe2c4154160 2024-04-26T22:01:32.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.784+0000 7fe2c9275640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe2a806c8f0 0x7fe2a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:32.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.784+0000 7fe2c9275640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe2a806c8f0 0x7fe2a806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0035040 tx=0x7fe2c002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:32.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.788+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2c4159ee0 con 0x7fe2c4154160 2024-04-26T22:01:32.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.936+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9428074b80 con 0x7f9428158980 2024-04-26T22:01:32.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.936+0000 7f9424ff9640 1 -- 172.21.15.135:0/2133036495 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5671 (secure 0 0 0) 0x7f942007a5d0 con 0x7f9428158980 2024-04-26T22:01:32.936 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:32.936 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:32.936 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 17 2024-04-26T22:01:32.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f941406c920 msgr2=0x7f941406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f941406c920 0x7f941406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94180016e0 tx=0x7f9418009040 comp rx=0 tx=0).stop 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 msgr2=0x7f94281517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f94281517b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9420034040 tx=0x7f9420004a40 comp rx=0 tx=0).stop 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 shutdown_connections 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f941406c920 0x7f941406ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9428158980 0x7f94281517b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.938+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f942814eb50 0x7f94280ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.939+0000 7f942e2ea640 1 --2- 172.21.15.135:0/2133036495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f942806f8a0 0x7f942814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.939+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 >> 172.21.15.135:0/2133036495 conn(0x7f94280c5d70 msgr2=0x7f9428153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.939+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 shutdown_connections 2024-04-26T22:01:32.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:32.939+0000 7f942e2ea640 1 -- 172.21.15.135:0/2133036495 wait complete. 2024-04-26T22:01:32.951 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=4278 2024-04-26T22:01:32.951 INFO:tasks.mds_thrash.fs.[a]:waiting for 9 secs before reviving mds.a 2024-04-26T22:01:33.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.027+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2c4158e10 con 0x7fe2c4154160 2024-04-26T22:01:33.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.027+0000 7fe2b9ffb640 1 -- 172.21.15.135:0/331252781 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5671 (secure 0 0 0) 0x7fe2bc022030 con 0x7fe2c4154160 2024-04-26T22:01:33.027 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:33.027 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:33.028 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 17 2024-04-26T22:01:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe2a806c8f0 msgr2=0x7fe2a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe2a806c8f0 0x7fe2a806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0035040 tx=0x7fe2c002e000 comp rx=0 tx=0).stop 2024-04-26T22:01:33.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c4154160 msgr2=0x7fe2c4147820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c4154160 0x7fe2c4147820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009d90 tx=0x7fe2bc00c490 comp rx=0 tx=0).stop 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 shutdown_connections 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe2a806c8f0 0x7fe2a806edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2c4154160 0x7fe2c4147820 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2c40ca5a0 0x7fe2c41532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 --2- 172.21.15.135:0/331252781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c6e00 0x7fe2c4152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 >> 172.21.15.135:0/331252781 conn(0x7fe2c40ae010 msgr2=0x7fe2c40c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 shutdown_connections 2024-04-26T22:01:33.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:33.029+0000 7fe2cb4d9640 1 -- 172.21.15.135:0/331252781 wait complete. 2024-04-26T22:01:33.041 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:35.042 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:35.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.141+0000 7fc0bfb19640 1 Processor -- start 2024-04-26T22:01:35.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.141+0000 7fc0bfb19640 1 -- start start 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bfb19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bfb19640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc0b8070120 0x7fc0b80aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bfb19640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b8154e10 con 0x7fc0b80aef70 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bfb19640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b80af400 con 0x7fc0b8070120 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bfb19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b81559e0 con 0x7fc0b806d590 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bd0b4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc0b8070120 0x7fc0b80aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0be0b6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc0b80aef70 0x7fc0b80c3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:36804/0 (socket says 172.21.15.135:36804) 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0be0b6640 1 -- 172.21.15.135:0/3100760436 learned_addr learned my addr 172.21.15.135:0/3100760436 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bd8b5640 1 --2- 172.21.15.135:0/3100760436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/3100760436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 msgr2=0x7fc0b8158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bc8b3640 1 -- 172.21.15.135:0/3100760436 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037775748 0 0) 0x7fc0b8154e10 con 0x7fc0b80aef70 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bd0b4640 1 --2- 172.21.15.135:0/3100760436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.142+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/3100760436 >> v1:172.21.15.139:6789/0 conn(0x7fc0b80aef70 legacy=0x7fc0b80c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.143+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/3100760436 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b8159240 con 0x7fc0b8070120 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.143+0000 7fc0bd0b4640 1 --2- 172.21.15.135:0/3100760436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80aea30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80097a0 tx=0x7fc0a802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c8a36e19a3f07f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.143+0000 7fc0bc8b3640 1 -- 172.21.15.135:0/3100760436 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8037070 con 0x7fc0b8070120 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.143+0000 7fc0bc8b3640 1 -- 172.21.15.135:0/3100760436 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a802ae50 con 0x7fc0b8070120 2024-04-26T22:01:35.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.143+0000 7fc0bc8b3640 1 -- 172.21.15.135:0/3100760436 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8032660 con 0x7fc0b8070120 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 -- 172.21.15.135:0/3100760436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 msgr2=0x7fc0b80aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/3100760436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80aea30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80097a0 tx=0x7fc0a802a100 comp rx=0 tx=0).stop 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 -- 172.21.15.135:0/3100760436 shutdown_connections 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/3100760436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80aea30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/3100760436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 -- 172.21.15.135:0/3100760436 >> 172.21.15.135:0/3100760436 conn(0x7fc0b80c5ce0 msgr2=0x7fc0b80c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 -- 172.21.15.135:0/3100760436 shutdown_connections 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.144+0000 7fc0bfb19640 1 -- 172.21.15.135:0/3100760436 wait complete. 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.145+0000 7fc0bfb19640 1 Processor -- start 2024-04-26T22:01:35.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.145+0000 7fc0bfb19640 1 -- start start 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bfb19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8072600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bfb19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bfb19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0b80aef70 0x7fc0b8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bfb19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8155670 con 0x7fc0b8070120 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bfb19640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8155390 con 0x7fc0b80aef70 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bfb19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b806d970 con 0x7fc0b806d590 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33418/0 (socket says 172.21.15.135:33418) 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/1631571694 learned_addr learned my addr 172.21.15.135:0/1631571694 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd8b5640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0be0b6640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0b80aef70 0x7fc0b8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 msgr2=0x7fc0b8072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0b80aef70 msgr2=0x7fc0b8157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0b80aef70 0x7fc0b8157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.146+0000 7fc0bd0b4640 1 -- 172.21.15.135:0/1631571694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b81540a0 con 0x7fc0b8070120 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.147+0000 7fc0bd0b4640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80c2910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80097a0 tx=0x7fc0a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.147+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8037040 con 0x7fc0b8070120 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.147+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a8004030 con 0x7fc0b8070120 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.147+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8004420 con 0x7fc0b8070120 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.147+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b81ed490 con 0x7fc0b8070120 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.147+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b81572c0 con 0x7fc0b8070120 2024-04-26T22:01:35.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.148+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b80507a0 con 0x7fc0b8070120 2024-04-26T22:01:35.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.148+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0a8003610 con 0x7fc0b8070120 2024-04-26T22:01:35.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.149+0000 7fc0ae7fc640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc09c06c8f0 0x7fc09c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:35.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.149+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7fc0a80aca50 con 0x7fc0b8070120 2024-04-26T22:01:35.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.149+0000 7fc0bd8b5640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc09c06c8f0 0x7fc09c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:35.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.149+0000 7fc0bd8b5640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc09c06c8f0 0x7fc09c06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0b81ea850 tx=0x7fc0b4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:35.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.153+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0a807a5d0 con 0x7fc0b8070120 2024-04-26T22:01:35.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.380+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b806ff00 con 0x7fc0b8070120 2024-04-26T22:01:35.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.381+0000 7fc0ae7fc640 1 -- 172.21.15.135:0/1631571694 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5671 (secure 0 0 0) 0x7fc0a807e8d0 con 0x7fc0b8070120 2024-04-26T22:01:35.382 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:35.382 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:35.382 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 17 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc09c06c8f0 msgr2=0x7fc09c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc09c06c8f0 0x7fc09c06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0b81ea850 tx=0x7fc0b4009190 comp rx=0 tx=0).stop 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 msgr2=0x7fc0b80c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80c2910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80097a0 tx=0x7fc0a80046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 shutdown_connections 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc09c06c8f0 0x7fc09c06edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0b80aef70 0x7fc0b8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0b8070120 0x7fc0b80c2910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 --2- 172.21.15.135:0/1631571694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0b806d590 0x7fc0b8072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:35.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 >> 172.21.15.135:0/1631571694 conn(0x7fc0b80c5ce0 msgr2=0x7fc0b8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:35.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 shutdown_connections 2024-04-26T22:01:35.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:35.384+0000 7fc0bfb19640 1 -- 172.21.15.135:0/1631571694 wait complete. 2024-04-26T22:01:35.399 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:37.400 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:37.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.498+0000 7f1f1ff11640 1 Processor -- start 2024-04-26T22:01:37.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.498+0000 7f1f1ff11640 1 -- start start 2024-04-26T22:01:37.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ff11640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f1815a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:37.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ff11640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f180c3d10 con 0x7f1f18157680 2024-04-26T22:01:37.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ff11640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f180c4290 con 0x7f1f18150d20 2024-04-26T22:01:37.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ff11640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f180c3fb0 con 0x7f1f1806f790 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1e4ae640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f1f18157680 0x7f1f1806d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51358/0 (socket says 172.21.15.135:51358) 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1e4ae640 1 -- 172.21.15.135:0/3288391679 learned_addr learned my addr 172.21.15.135:0/3288391679 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1d4ac640 1 --2- 172.21.15.135:0/3288391679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f1815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962807467 0 0) 0x7f1f180c3d10 con 0x7f1f18157680 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f04003520 con 0x7f1f18157680 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548885691 0 0) 0x7f1f180c4290 con 0x7f1f18150d20 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f180c3d10 con 0x7f1f18150d20 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3288391679 >> v1:172.21.15.139:6789/0 conn(0x7f1f18150d20 legacy=0x7f1f180c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3288391679 >> v1:172.21.15.135:6789/0 conn(0x7f1f18157680 legacy=0x7f1f1806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.499+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3288391679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f18156e80 con 0x7f1f1806f790 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.500+0000 7f1f1d4ac640 1 --2- 172.21.15.135:0/3288391679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f1815a360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097d0 tx=0x7f1f0802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a8369476907b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.500+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08037070 con 0x7f1f1806f790 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.500+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08032440 con 0x7f1f1806f790 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.500+0000 7f1f1ccab640 1 -- 172.21.15.135:0/3288391679 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f080327f0 con 0x7f1f1806f790 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3288391679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 msgr2=0x7f1f1815a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3288391679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f1815a360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097d0 tx=0x7f1f0802a180 comp rx=0 tx=0).stop 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3288391679 shutdown_connections 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3288391679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f1815a360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3288391679 >> 172.21.15.135:0/3288391679 conn(0x7f1f180c5ce0 msgr2=0x7f1f180c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3288391679 shutdown_connections 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.501+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3288391679 wait complete. 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.502+0000 7f1f1ff11640 1 Processor -- start 2024-04-26T22:01:37.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.502+0000 7f1f1ff11640 1 -- start start 2024-04-26T22:01:37.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1ff11640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f180ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:37.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1ff11640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 0x7f1f181ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1ff11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f18157680 0x7f1f181ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1ff11640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806dbb0 con 0x7f1f18150d20 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1ff11640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806da30 con 0x7f1f18157680 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1ff11640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f18151100 con 0x7f1f1806f790 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 0x7f1f181ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 0x7f1f181ec9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:33424/0 (socket says 172.21.15.135:33424) 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1dcad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f180ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3298076981 learned_addr learned my addr 172.21.15.135:0/3298076981 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1e4ae640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f18157680 0x7f1f181ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:37.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 msgr2=0x7f1f180ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f180ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f18157680 msgr2=0x7f1f181ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f18157680 0x7f1f181ecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 -- 172.21.15.135:0/3298076981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181e1210 con 0x7f1f18150d20 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1dcad640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f180ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.503+0000 7f1f1d4ac640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 0x7f1f181ec9d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802a1c0 tx=0x7f1f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.504+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08041020 con 0x7f1f18150d20 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.504+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0803d6e0 con 0x7f1f18150d20 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.504+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08037040 con 0x7f1f18150d20 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.504+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f181f00e0 con 0x7f1f18150d20 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.504+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f180ac910 con 0x7f1f18150d20 2024-04-26T22:01:37.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.505+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f18068730 con 0x7f1f18150d20 2024-04-26T22:01:37.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.506+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f08044020 con 0x7f1f18150d20 2024-04-26T22:01:37.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.506+0000 7f1f0e7fc640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1efc06c8f0 0x7f1efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:37.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.506+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6752+0+0 (secure 0 0 0) 0x7f1f080ac570 con 0x7f1f18150d20 2024-04-26T22:01:37.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.506+0000 7f1f1dcad640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1efc06c8f0 0x7f1efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:37.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.507+0000 7f1f1dcad640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1efc06c8f0 0x7f1efc06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f14007fd0 tx=0x7f1f14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:37.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.511+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f0804b030 con 0x7f1f18150d20 2024-04-26T22:01:37.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.747+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f180507a0 con 0x7f1f18150d20 2024-04-26T22:01:37.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.747+0000 7f1f0e7fc640 1 -- 172.21.15.135:0/3298076981 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5671 (secure 0 0 0) 0x7f1f08079f50 con 0x7f1f18150d20 2024-04-26T22:01:37.748 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:37.748 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:00:10.177793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.135:6836/3981978490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3981978490},{"type":"v1","addr":"172.21.15.135:6836","nonce":3981978490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":2}]} 2024-04-26T22:01:37.749 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 17 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1efc06c8f0 msgr2=0x7f1efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1efc06c8f0 0x7f1efc06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f14007fd0 tx=0x7f1f14009040 comp rx=0 tx=0).stop 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 msgr2=0x7f1f181ec9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 0x7f1f181ec9d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802a1c0 tx=0x7f1f080046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 shutdown_connections 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1efc06c8f0 0x7f1efc06edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f18157680 0x7f1f181ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18150d20 0x7f1f181ec9d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 --2- 172.21.15.135:0/3298076981 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f1806f790 0x7f1f180ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:37.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 >> 172.21.15.135:0/3298076981 conn(0x7f1f180c5ce0 msgr2=0x7f1f180c31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:37.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 shutdown_connections 2024-04-26T22:01:37.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:37.750+0000 7f1f1ff11640 1 -- 172.21.15.135:0/3298076981 wait complete. 2024-04-26T22:01:37.761 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:01:39.763 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:39.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.876+0000 7f730fe3f640 1 Processor -- start 2024-04-26T22:01:39.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.876+0000 7f730fe3f640 1 -- start start 2024-04-26T22:01:39.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730fe3f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7308154160 0x7f7308156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730fe3f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73080c8df0 con 0x7f73080ca5a0 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730fe3f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7308154540 con 0x7f730815a1a0 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730fe3f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f73080c9100 con 0x7f7308154160 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730d3da640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f73080ca5a0 0x7f73080c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44276/0 (socket says 172.21.15.135:44276) 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730d3da640 1 -- 172.21.15.135:0/1767738326 learned_addr learned my addr 172.21.15.135:0/1767738326 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730dbdb640 1 --2- 172.21.15.135:0/1767738326 >> v2:172.21.15.139:3300/0 conn(0x7f7308154160 0x7f7308156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730cbd9640 1 -- 172.21.15.135:0/1767738326 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825851500 0 0) 0x7f72f8001110 con 0x7f73080ca5a0 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730cbd9640 1 -- 172.21.15.135:0/1767738326 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72f4003520 con 0x7f73080ca5a0 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730dbdb640 1 -- 172.21.15.135:0/1767738326 >> v1:172.21.15.139:6789/0 conn(0x7f730815a1a0 legacy=0x7f73080c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730dbdb640 1 -- 172.21.15.135:0/1767738326 >> v1:172.21.15.135:6789/0 conn(0x7f73080ca5a0 legacy=0x7f73080c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730cbd9640 1 -- 172.21.15.135:0/1767738326 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752913936 0 0) 0x7f7308154540 con 0x7f730815a1a0 2024-04-26T22:01:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.877+0000 7f730dbdb640 1 -- 172.21.15.135:0/1767738326 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730815a910 con 0x7f7308154160 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730dbdb640 1 --2- 172.21.15.135:0/1767738326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7308154160 0x7f7308156bb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f73040097a0 tx=0x7f730402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3ee48e632327002 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730cbd9640 1 -- 172.21.15.135:0/1767738326 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304037070 con 0x7f7308154160 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730cbd9640 1 -- 172.21.15.135:0/1767738326 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7304032440 con 0x7f7308154160 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730cbd9640 1 -- 172.21.15.135:0/1767738326 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304032810 con 0x7f7308154160 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730fe3f640 1 -- 172.21.15.135:0/1767738326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7308154160 msgr2=0x7f7308156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1767738326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7308154160 0x7f7308156bb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f73040097a0 tx=0x7f730402a230 comp rx=0 tx=0).stop 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730fe3f640 1 -- 172.21.15.135:0/1767738326 shutdown_connections 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1767738326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7308154160 0x7f7308156bb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.879+0000 7f730fe3f640 1 -- 172.21.15.135:0/1767738326 >> 172.21.15.135:0/1767738326 conn(0x7f73080ae010 msgr2=0x7f73080ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.880+0000 7f730fe3f640 1 -- 172.21.15.135:0/1767738326 shutdown_connections 2024-04-26T22:01:39.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.880+0000 7f730fe3f640 1 -- 172.21.15.135:0/1767738326 wait complete. 2024-04-26T22:01:39.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.881+0000 7f730fe3f640 1 Processor -- start 2024-04-26T22:01:39.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.881+0000 7f730fe3f640 1 -- start start 2024-04-26T22:01:39.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730fe3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080ca5a0 0x7f7308159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730fe3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 0x7f730814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730fe3f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f730815a1a0 0x7f730814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730fe3f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f730815a5b0 con 0x7f7308154160 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730fe3f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f730815aed0 con 0x7f73080ca5a0 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730fe3f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73080c7510 con 0x7f730815a1a0 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730d3da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 0x7f730814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730d3da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 0x7f730814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36320/0 (socket says 172.21.15.135:36320) 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730d3da640 1 -- 172.21.15.135:0/1707816807 learned_addr learned my addr 172.21.15.135:0/1707816807 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730dbdb640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080ca5a0 0x7f7308159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.882+0000 7f730e3dc640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f730815a1a0 0x7f730814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730d3da640 1 -- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f730815a1a0 msgr2=0x7f730814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730d3da640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f730815a1a0 0x7f730814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730d3da640 1 -- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080ca5a0 msgr2=0x7f7308159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:39.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730d3da640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080ca5a0 0x7f7308159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730d3da640 1 -- 172.21.15.135:0/1707816807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73080cbda0 con 0x7f7308154160 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730d3da640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 0x7f730814aa30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c730 tx=0x7f72f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8017070 con 0x7f7308154160 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f8005ad0 con 0x7f7308154160 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7308151960 con 0x7f7308154160 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8007e20 con 0x7f7308154160 2024-04-26T22:01:39.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.883+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7308151f20 con 0x7f7308154160 2024-04-26T22:01:39.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.884+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7308076650 con 0x7f7308154160 2024-04-26T22:01:39.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.885+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72f801a070 con 0x7f7308154160 2024-04-26T22:01:39.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.885+0000 7f72fe7fc640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f72ec06c8f0 0x7f72ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:39.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.885+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f72f808b240 con 0x7f7308154160 2024-04-26T22:01:39.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.885+0000 7f730dbdb640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f72ec06c8f0 0x7f72ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:39.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.886+0000 7f730dbdb640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f72ec06c8f0 0x7f72ec06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7304034000 tx=0x7f730402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:39.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:39.892+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73080c8df0 con 0x7f7308154160 2024-04-26T22:01:40.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.134+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73080cb290 con 0x7f7308154160 2024-04-26T22:01:40.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.135+0000 7f72fe7fc640 1 -- 172.21.15.135:0/1707816807 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+4841 (secure 0 0 0) 0x7f72f8058c40 con 0x7f7308154160 2024-04-26T22:01:40.136 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:40.136 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:38.795652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:01:40.136 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 20 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f72ec06c8f0 msgr2=0x7f72ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f72ec06c8f0 0x7f72ec06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7304034000 tx=0x7f730402e000 comp rx=0 tx=0).stop 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 msgr2=0x7f730814aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 0x7f730814aa30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c730 tx=0x7f72f800cbf0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 shutdown_connections 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f72ec06c8f0 0x7f72ec06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f730815a1a0 0x7f730814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7308154160 0x7f730814aa30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 --2- 172.21.15.135:0/1707816807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080ca5a0 0x7f7308159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 >> 172.21.15.135:0/1707816807 conn(0x7f73080ae010 msgr2=0x7f73081570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:40.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 shutdown_connections 2024-04-26T22:01:40.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.137+0000 7f730fe3f640 1 -- 172.21.15.135:0/1707816807 wait complete. 2024-04-26T22:01:40.150 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-04-26T22:01:40.150 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:01:40.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:40.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.249+0000 7f38611bf640 1 Processor -- start 2024-04-26T22:01:40.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.249+0000 7f38611bf640 1 -- start start 2024-04-26T22:01:40.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.250+0000 7f38611bf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c150d20 0x7f385c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:40.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.250+0000 7f38611bf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f385c157680 0x7f385c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:40.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.250+0000 7f38611bf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c0c3d10 con 0x7f385c06f790 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.250+0000 7f38611bf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f385c0c4290 con 0x7f385c157680 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f38611bf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f385c0c3fb0 con 0x7f385c150d20 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385b59e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f385c157680 0x7f385c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c150d20 0x7f385c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385a59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f385c06f790 0x7f385c0c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:39122/0 (socket says 172.21.15.135:39122) 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385a59c640 1 -- 172.21.15.135:0/779431431 learned_addr learned my addr 172.21.15.135:0/779431431 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385b59e640 1 -- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c150d20 msgr2=0x7f385c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385b59e640 1 --2- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c150d20 0x7f385c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385b59e640 1 -- 172.21.15.135:0/779431431 >> v1:172.21.15.139:6789/0 conn(0x7f385c06f790 legacy=0x7f385c0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385b59e640 1 -- 172.21.15.135:0/779431431 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c156e80 con 0x7f385c157680 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f3859d9b640 1 -- 172.21.15.135:0/779431431 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287983120 0 0) 0x7f385c0c3d10 con 0x7f385c06f790 2024-04-26T22:01:40.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.251+0000 7f385ad9d640 1 --2- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c150d20 0x7f385c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:40.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.252+0000 7f385b59e640 1 --2- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c157680 0x7f385c06d320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3854009d90 tx=0x7f385402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0281036ebe396c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.252+0000 7f3859d9b640 1 -- 172.21.15.135:0/779431431 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854031600 con 0x7f385c157680 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.252+0000 7f3859d9b640 1 -- 172.21.15.135:0/779431431 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38540317a0 con 0x7f385c157680 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.252+0000 7f3859d9b640 1 -- 172.21.15.135:0/779431431 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854031b70 con 0x7f385c157680 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 -- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c157680 msgr2=0x7f385c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 --2- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c157680 0x7f385c06d320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3854009d90 tx=0x7f385402a5d0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 -- 172.21.15.135:0/779431431 shutdown_connections 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 --2- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c157680 0x7f385c06d320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 --2- 172.21.15.135:0/779431431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c150d20 0x7f385c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 -- 172.21.15.135:0/779431431 >> 172.21.15.135:0/779431431 conn(0x7f385c0c5ce0 msgr2=0x7f385c0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 -- 172.21.15.135:0/779431431 shutdown_connections 2024-04-26T22:01:40.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.253+0000 7f38611bf640 1 -- 172.21.15.135:0/779431431 wait complete. 2024-04-26T22:01:40.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.255+0000 7f38611bf640 1 Processor -- start 2024-04-26T22:01:40.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.255+0000 7f38611bf640 1 -- start start 2024-04-26T22:01:40.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f38611bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c06f790 0x7f385c073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:40.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f38611bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c150d20 0x7f385c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:40.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f38611bf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c157680 0x7f385c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:40.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f38611bf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f385c151280 con 0x7f385c150d20 2024-04-26T22:01:40.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f38611bf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f385c151100 con 0x7f385c06f790 2024-04-26T22:01:40.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f385a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c150d20 0x7f385c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f385a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c150d20 0x7f385c14dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36336/0 (socket says 172.21.15.135:36336) 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f385a59c640 1 -- 172.21.15.135:0/2133820485 learned_addr learned my addr 172.21.15.135:0/2133820485 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f385ad9d640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c06f790 0x7f385c073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f385c157b10 con 0x7f385c157680 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.256+0000 7f385b59e640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c157680 0x7f385c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385ad9d640 1 -- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c157680 msgr2=0x7f385c14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385ad9d640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c157680 0x7f385c14e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385ad9d640 1 -- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c150d20 msgr2=0x7f385c14dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385ad9d640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c150d20 0x7f385c14dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385ad9d640 1 -- 172.21.15.135:0/2133820485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1ec900 con 0x7f385c06f790 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385b59e640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c157680 0x7f385c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f385ad9d640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c06f790 0x7f385c073a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f384c00e930 tx=0x7f384c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c017070 con 0x7f385c06f790 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c00ca00 con 0x7f385c06f790 2024-04-26T22:01:40.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.257+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c00cdd0 con 0x7f385c06f790 2024-04-26T22:01:40.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.258+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1ecbe0 con 0x7f385c06f790 2024-04-26T22:01:40.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.258+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c1ed1a0 con 0x7f385c06f790 2024-04-26T22:01:40.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.259+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c0507a0 con 0x7f385c06f790 2024-04-26T22:01:40.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.260+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f384c01a070 con 0x7f385c06f790 2024-04-26T22:01:40.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.260+0000 7f38437fe640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f384806c8f0 0x7f384806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:40.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.260+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f384c08bd60 con 0x7f385c06f790 2024-04-26T22:01:40.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.260+0000 7f385a59c640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f384806c8f0 0x7f384806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:40.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.261+0000 7f385a59c640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f384806c8f0 0x7f384806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f385c15a820 tx=0x7f3850007e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:40.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.264+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f384c059760 con 0x7f385c06f790 2024-04-26T22:01:40.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.511+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385c06f500 con 0x7f385c06f790 2024-04-26T22:01:40.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.512+0000 7f38437fe640 1 -- 172.21.15.135:0/2133820485 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+4841 (secure 0 0 0) 0x7f384c05da60 con 0x7f385c06f790 2024-04-26T22:01:40.513 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:40.513 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:38.795652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:01:40.513 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 20 2024-04-26T22:01:40.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.515+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f384806c8f0 msgr2=0x7f384806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.515+0000 7f38611bf640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f384806c8f0 0x7f384806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f385c15a820 tx=0x7f3850007e10 comp rx=0 tx=0).stop 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.515+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c06f790 msgr2=0x7f385c073a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.515+0000 7f38611bf640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c06f790 0x7f385c073a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f384c00e930 tx=0x7f384c00edf0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.515+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 shutdown_connections 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.515+0000 7f38611bf640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f384806c8f0 0x7f384806edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.516+0000 7f38611bf640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f385c157680 0x7f385c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.516+0000 7f38611bf640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c150d20 0x7f385c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.516+0000 7f38611bf640 1 --2- 172.21.15.135:0/2133820485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f385c06f790 0x7f385c073a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.516+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 >> 172.21.15.135:0/2133820485 conn(0x7f385c0c5ce0 msgr2=0x7f385c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.516+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 shutdown_connections 2024-04-26T22:01:40.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:40.516+0000 7f38611bf640 1 -- 172.21.15.135:0/2133820485 wait complete. 2024-04-26T22:01:40.532 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 20, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 17, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4244": { "addr": "172.21.15.139:6835/2608264407", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 2608264407, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 2608264407, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 14, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:01:30.775227+0000", "qdb_cluster": [ 4244 ], "qdb_leader": 4244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4244 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 20, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4241": { "addr": "172.21.15.139:6833/3183513686", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3183513686, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3183513686, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 18, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:rejoin", "state_seq": 29 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:01:38.795652+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4241 } } } ], "standbys": [] } 2024-04-26T22:01:41.623 INFO:tasks.ceph.mds.b.smithi139.stderr:2024-04-26T22:01:41.623+0000 7f6751348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:01:41.951 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-04-26T22:01:41.952 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:01:41.952 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:01:41.955 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:01:41.955 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:42.034 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-26T22:01:42.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.089+0000 7fc37b19e640 1 Processor -- start 2024-04-26T22:01:42.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.089+0000 7fc37b19e640 1 -- start start 2024-04-26T22:01:42.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37b19e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 0x7fc3740756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37b19e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 0x7fc37415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37b19e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3741544f0 con 0x7fc3740c4530 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37b19e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc374072b10 con 0x7fc374074710 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37b19e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc3740c56b0 con 0x7fc374154110 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37999b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 0x7fc37415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37a19c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 0x7fc3740756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37999b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 0x7fc37415a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:34232/0 (socket says 172.21.15.135:34232) 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37a99d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc3740c4530 0x7fc3740c2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:39128/0 (socket says 172.21.15.135:39128) 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37a19c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 0x7fc3740756f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:53188/0 (socket says 172.21.15.135:53188) 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.090+0000 7fc37999b640 1 -- 172.21.15.135:0/1486091195 learned_addr learned my addr 172.21.15.135:0/1486091195 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.091+0000 7fc37a19c640 1 -- 172.21.15.135:0/1486091195 >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 msgr2=0x7fc37415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.091+0000 7fc37a19c640 1 --2- 172.21.15.135:0/1486091195 >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 0x7fc37415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.091+0000 7fc37a19c640 1 -- 172.21.15.135:0/1486091195 >> v1:172.21.15.139:6789/0 conn(0x7fc3740c4530 legacy=0x7fc3740c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.091+0000 7fc37a19c640 1 -- 172.21.15.135:0/1486091195 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37415a260 con 0x7fc374154110 2024-04-26T22:01:42.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.091+0000 7fc37999b640 1 --2- 172.21.15.135:0/1486091195 >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 0x7fc37415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37a19c640 1 --2- 172.21.15.135:0/1486091195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 0x7fc3740756f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3700097a0 tx=0x7fc37002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c6738e667fa58d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37919a640 1 -- 172.21.15.135:0/1486091195 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037070 con 0x7fc374154110 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37919a640 1 -- 172.21.15.135:0/1486091195 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37002ae50 con 0x7fc374154110 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37919a640 1 -- 172.21.15.135:0/1486091195 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037070 con 0x7fc374154110 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37b19e640 1 -- 172.21.15.135:0/1486091195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 msgr2=0x7fc3740756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37b19e640 1 --2- 172.21.15.135:0/1486091195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 0x7fc3740756f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3700097a0 tx=0x7fc37002a100 comp rx=0 tx=0).stop 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37b19e640 1 -- 172.21.15.135:0/1486091195 shutdown_connections 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37b19e640 1 --2- 172.21.15.135:0/1486091195 >> v2:172.21.15.139:3300/0 conn(0x7fc374074710 0x7fc37415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37b19e640 1 --2- 172.21.15.135:0/1486091195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374154110 0x7fc3740756f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.092+0000 7fc37b19e640 1 -- 172.21.15.135:0/1486091195 >> 172.21.15.135:0/1486091195 conn(0x7fc37406c310 msgr2=0x7fc37406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.093+0000 7fc37b19e640 1 -- 172.21.15.135:0/1486091195 shutdown_connections 2024-04-26T22:01:42.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.093+0000 7fc37b19e640 1 -- 172.21.15.135:0/1486091195 wait complete. 2024-04-26T22:01:42.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.094+0000 7fc37b19e640 1 Processor -- start 2024-04-26T22:01:42.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.094+0000 7fc37b19e640 1 -- start start 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37b19e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 0x7fc3741ed720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37b19e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc374154110 0x7fc374159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37b19e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374150aa0 0x7fc3740c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37b19e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc374156ad0 con 0x7fc374154110 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37a19c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 0x7fc3741ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37b19e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc374073bf0 con 0x7fc3740c4530 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37b19e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc374156c50 con 0x7fc374150aa0 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37a19c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 0x7fc3741ed720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:34246/0 (socket says 172.21.15.135:34246) 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37a19c640 1 -- 172.21.15.135:0/3086287896 learned_addr learned my addr 172.21.15.135:0/3086287896 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:42.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37999b640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc374154110 0x7fc374159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.095+0000 7fc37a99d640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374150aa0 0x7fc3740c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37a19c640 1 -- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374150aa0 msgr2=0x7fc3740c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37a19c640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374150aa0 0x7fc3740c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37a19c640 1 -- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc374154110 msgr2=0x7fc374159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37a19c640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc374154110 0x7fc374159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37a19c640 1 -- 172.21.15.135:0/3086287896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3741edc60 con 0x7fc3740c4530 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37a19c640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 0x7fc3741ed720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc37002a140 tx=0x7fc3700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370042070 con 0x7fc3740c4530 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37003b610 con 0x7fc3740c4530 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037070 con 0x7fc3740c4530 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3741f0360 con 0x7fc3740c4530 2024-04-26T22:01:42.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.096+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc374159170 con 0x7fc3740c4530 2024-04-26T22:01:42.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.097+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3740506c0 con 0x7fc3740c4530 2024-04-26T22:01:42.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.098+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc370045070 con 0x7fc3740c4530 2024-04-26T22:01:42.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.098+0000 7fc36affd640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc35806c8f0 0x7fc35806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.098+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7fc3700ac0d0 con 0x7fc3740c4530 2024-04-26T22:01:42.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.098+0000 7fc37999b640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc35806c8f0 0x7fc35806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.099+0000 7fc37999b640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc35806c8f0 0x7fc35806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc364009630 tx=0x7fc364009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:42.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.106+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc370079ad0 con 0x7fc3740c4530 2024-04-26T22:01:42.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.386+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3740c4310 con 0x7fc3740c4530 2024-04-26T22:01:42.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.387+0000 7fc36affd640 1 -- 172.21.15.135:0/3086287896 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+4848 (secure 0 0 0) 0x7fc37007ddd0 con 0x7fc3740c4530 2024-04-26T22:01:42.388 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:42.388 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:01:42.388 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 21 2024-04-26T22:01:42.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc35806c8f0 msgr2=0x7fc35806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc35806c8f0 0x7fc35806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc364009630 tx=0x7fc364009240 comp rx=0 tx=0).stop 2024-04-26T22:01:42.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 msgr2=0x7fc3741ed720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 0x7fc3741ed720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc37002a140 tx=0x7fc3700046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 shutdown_connections 2024-04-26T22:01:42.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc35806c8f0 0x7fc35806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc374150aa0 0x7fc3740c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc374154110 0x7fc374159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 --2- 172.21.15.135:0/3086287896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3740c4530 0x7fc3741ed720 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 >> 172.21.15.135:0/3086287896 conn(0x7fc37406c310 msgr2=0x7fc37415a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:42.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 shutdown_connections 2024-04-26T22:01:42.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.389+0000 7fc37b19e640 1 -- 172.21.15.135:0/3086287896 wait complete. 2024-04-26T22:01:42.401 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-26T22:01:42.533 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.629+0000 7f838d138640 1 Processor -- start 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.629+0000 7f838d138640 1 -- start start 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838d138640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838d138640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83880bfb40 0x7f8388157240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838d138640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83880cc770 con 0x7f83880ca3b0 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838d138640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8388157780 con 0x7f83880bfb40 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838d138640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f838815abd0 con 0x7f83881540b0 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83880bfb40 0x7f8388157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83880bfb40 0x7f8388157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36360/0 (socket says 172.21.15.135:36360) 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 -- 172.21.15.135:0/2312970071 learned_addr learned my addr 172.21.15.135:0/2312970071 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f8386d9d640 1 --2- 172.21.15.135:0/2312970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 -- 172.21.15.135:0/2312970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 msgr2=0x7f83880c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 --2- 172.21.15.135:0/2312970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 -- 172.21.15.135:0/2312970071 >> v1:172.21.15.139:6789/0 conn(0x7f83880ca3b0 legacy=0x7f8388156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f8385d9b640 1 -- 172.21.15.135:0/2312970071 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484732689 0 0) 0x7f8378001110 con 0x7f83880ca3b0 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.630+0000 7f838759e640 1 -- 172.21.15.135:0/2312970071 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838815aeb0 con 0x7f83880bfb40 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.631+0000 7f838759e640 1 --2- 172.21.15.135:0/2312970071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880bfb40 0x7f8388157240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8380009e50 tx=0x7f838002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d0cd528570003c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.631+0000 7f8385d9b640 1 -- 172.21.15.135:0/2312970071 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380036070 con 0x7f83880bfb40 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.631+0000 7f8385d9b640 1 -- 172.21.15.135:0/2312970071 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8380031600 con 0x7f83880bfb40 2024-04-26T22:01:42.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.631+0000 7f8385d9b640 1 -- 172.21.15.135:0/2312970071 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800319d0 con 0x7f83880bfb40 2024-04-26T22:01:42.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 -- 172.21.15.135:0/2312970071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880bfb40 msgr2=0x7f8388157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 --2- 172.21.15.135:0/2312970071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880bfb40 0x7f8388157240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8380009e50 tx=0x7f838002a5f0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 -- 172.21.15.135:0/2312970071 shutdown_connections 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 --2- 172.21.15.135:0/2312970071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880bfb40 0x7f8388157240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 --2- 172.21.15.135:0/2312970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 -- 172.21.15.135:0/2312970071 >> 172.21.15.135:0/2312970071 conn(0x7f838806c4f0 msgr2=0x7f838806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 -- 172.21.15.135:0/2312970071 shutdown_connections 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.632+0000 7f838d138640 1 -- 172.21.15.135:0/2312970071 wait complete. 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.633+0000 7f838d138640 1 Processor -- start 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.633+0000 7f838d138640 1 -- start start 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838d138640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83880bfb40 0x7f838814cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838d138640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 0x7f83880cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838d138640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838d138640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8388154490 con 0x7f83880ca3b0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838d138640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83880cccd0 con 0x7f83880bfb40 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838d138640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8388072930 con 0x7f83881540b0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 0x7f83880cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 0x7f83880cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36374/0 (socket says 172.21.15.135:36374) 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f8386d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83880bfb40 0x7f838814cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838759e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 -- 172.21.15.135:0/315571375 learned_addr learned my addr 172.21.15.135:0/315571375 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 -- 172.21.15.135:0/315571375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 msgr2=0x7f83880bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880bffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 -- 172.21.15.135:0/315571375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83880bfb40 msgr2=0x7f838814cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83880bfb40 0x7f838814cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 -- 172.21.15.135:0/315571375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83880c2bd0 con 0x7f83880ca3b0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.634+0000 7f838659c640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 0x7f83880cb5e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f837800c7b0 tx=0x7f837800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.635+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378017070 con 0x7f83880ca3b0 2024-04-26T22:01:42.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.635+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8378005070 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.635+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838814d1a0 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.635+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378005460 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.635+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838814f0a0 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.636+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83880507a0 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.636+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f837801a070 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.637+0000 7f836f7fe640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f837406c8f0 0x7f837406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.637+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f837808b200 con 0x7f83880ca3b0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.637+0000 7f8386d9d640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f837406c8f0 0x7f837406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:42.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.637+0000 7f8386d9d640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f837406c8f0 0x7f837406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f838815aa90 tx=0x7f837c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:42.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.644+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83880c2bd0 con 0x7f83880ca3b0 2024-04-26T22:01:42.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.935+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8388073590 con 0x7f83880ca3b0 2024-04-26T22:01:42.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.936+0000 7f836f7fe640 1 -- 172.21.15.135:0/315571375 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5675 (secure 0 0 0) 0x7f83880cc770 con 0x7f83880ca3b0 2024-04-26T22:01:42.937 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:42.937 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:01:42.937 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 22 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f837406c8f0 msgr2=0x7f837406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f837406c8f0 0x7f837406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f838815aa90 tx=0x7f837c009190 comp rx=0 tx=0).stop 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 msgr2=0x7f83880cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 0x7f83880cb5e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f837800c7b0 tx=0x7f837800cc70 comp rx=0 tx=0).stop 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 shutdown_connections 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f837406c8f0 0x7f837406edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83881540b0 0x7f83880bffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83880ca3b0 0x7f83880cb5e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 --2- 172.21.15.135:0/315571375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83880bfb40 0x7f838814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:42.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 >> 172.21.15.135:0/315571375 conn(0x7f838806c4f0 msgr2=0x7f83881594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:42.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 shutdown_connections 2024-04-26T22:01:42.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:42.938+0000 7f838d138640 1 -- 172.21.15.135:0/315571375 wait complete. 2024-04-26T22:01:42.951 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=4275 2024-04-26T22:01:42.951 INFO:tasks.mds_thrash.fs.[b]:waiting for 4 secs before reviving mds.c 2024-04-26T22:01:44.402 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:44.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.503+0000 7f68d676a640 1 Processor -- start 2024-04-26T22:01:44.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d676a640 1 -- start start 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d676a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d676a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68d015a1a0 0x7f68d00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d676a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d00c8df0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d676a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f68d00c9100 con 0x7f68d015a1a0 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d676a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68d0154540 con 0x7f68d0154160 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.504+0000 7f68d4d07640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68d015a1a0 0x7f68d00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68d4d07640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68d015a1a0 0x7f68d00c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36382/0 (socket says 172.21.15.135:36382) 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68d4d07640 1 -- 172.21.15.135:0/3110030889 learned_addr learned my addr 172.21.15.135:0/3110030889 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:44.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68cffff640 1 --2- 172.21.15.135:0/3110030889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:44.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68d4d07640 1 -- 172.21.15.135:0/3110030889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 msgr2=0x7f68d0156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:44.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68d4d07640 1 --2- 172.21.15.135:0/3110030889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d0156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68d4d07640 1 -- 172.21.15.135:0/3110030889 >> v1:172.21.15.139:6789/0 conn(0x7f68d00ca5a0 legacy=0x7f68d00c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.505+0000 7f68d4d07640 1 -- 172.21.15.135:0/3110030889 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d015a910 con 0x7f68d015a1a0 2024-04-26T22:01:44.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.506+0000 7f68d4d07640 1 --2- 172.21.15.135:0/3110030889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d015a1a0 0x7f68d00c9590 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009e50 tx=0x7f68c802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb696a529de50904 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:44.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.506+0000 7f68ceffd640 1 -- 172.21.15.135:0/3110030889 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8036070 con 0x7f68d015a1a0 2024-04-26T22:01:44.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.506+0000 7f68ceffd640 1 -- 172.21.15.135:0/3110030889 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c8031600 con 0x7f68d015a1a0 2024-04-26T22:01:44.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.506+0000 7f68ceffd640 1 -- 172.21.15.135:0/3110030889 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c80319d0 con 0x7f68d015a1a0 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 -- 172.21.15.135:0/3110030889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d015a1a0 msgr2=0x7f68d00c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 --2- 172.21.15.135:0/3110030889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d015a1a0 0x7f68d00c9590 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009e50 tx=0x7f68c802a710 comp rx=0 tx=0).stop 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 -- 172.21.15.135:0/3110030889 shutdown_connections 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 --2- 172.21.15.135:0/3110030889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d015a1a0 0x7f68d00c9590 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 --2- 172.21.15.135:0/3110030889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d0156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 -- 172.21.15.135:0/3110030889 >> 172.21.15.135:0/3110030889 conn(0x7f68d00ae010 msgr2=0x7f68d00ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 -- 172.21.15.135:0/3110030889 shutdown_connections 2024-04-26T22:01:44.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.507+0000 7f68d676a640 1 -- 172.21.15.135:0/3110030889 wait complete. 2024-04-26T22:01:44.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.509+0000 7f68d676a640 1 Processor -- start 2024-04-26T22:01:44.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.509+0000 7f68d676a640 1 -- start start 2024-04-26T22:01:44.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d676a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 0x7f68d0149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:44.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d676a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d01431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:44.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d676a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68d015a1a0 0x7f68d01436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:44.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d676a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68d015a630 con 0x7f68d00ca5a0 2024-04-26T22:01:44.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d676a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68d015aed0 con 0x7f68d015a1a0 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d676a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68d00c7510 con 0x7f68d0154160 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 0x7f68d0149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 0x7f68d0149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36390/0 (socket says 172.21.15.135:36390) 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 -- 172.21.15.135:0/1256859444 learned_addr learned my addr 172.21.15.135:0/1256859444 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68d4d07640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68d015a1a0 0x7f68d01436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cf7fe640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d01431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 -- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 msgr2=0x7f68d01431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d01431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 -- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68d015a1a0 msgr2=0x7f68d01436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68d015a1a0 0x7f68d01436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.510+0000 7f68cffff640 1 -- 172.21.15.135:0/1256859444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d00cb800 con 0x7f68d00ca5a0 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.511+0000 7f68cffff640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 0x7f68d0149360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e9b0 tx=0x7f68c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.511+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0017070 con 0x7f68d00ca5a0 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.511+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c000c9a0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.511+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d01581f0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.511+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c000cd90 con 0x7f68d00ca5a0 2024-04-26T22:01:44.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.511+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d01587b0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.512+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d0158eb0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.513+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68c001a070 con 0x7f68d00ca5a0 2024-04-26T22:01:44.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.513+0000 7f68ccff9640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68bc06c8f0 0x7f68bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:44.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.513+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f68c008c2b0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.513+0000 7f68cf7fe640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68bc06c8f0 0x7f68bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:44.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.514+0000 7f68cf7fe640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68bc06c8f0 0x7f68bc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f68d00cb670 tx=0x7f68c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:44.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.518+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68d0158eb0 con 0x7f68d00ca5a0 2024-04-26T22:01:44.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.739+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68d0076650 con 0x7f68d00ca5a0 2024-04-26T22:01:44.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.739+0000 7f68ccff9640 1 -- 172.21.15.135:0/1256859444 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5675 (secure 0 0 0) 0x7f68c0059c20 con 0x7f68d00ca5a0 2024-04-26T22:01:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:44.740 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:01:44.740 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 22 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68bc06c8f0 msgr2=0x7f68bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68bc06c8f0 0x7f68bc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f68d00cb670 tx=0x7f68c4009040 comp rx=0 tx=0).stop 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 msgr2=0x7f68d0149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 0x7f68d0149360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e9b0 tx=0x7f68c000ee70 comp rx=0 tx=0).stop 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 shutdown_connections 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f68bc06c8f0 0x7f68bc06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.741+0000 7f68d676a640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68d015a1a0 0x7f68d01436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.742+0000 7f68d676a640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68d0154160 0x7f68d01431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.742+0000 7f68d676a640 1 --2- 172.21.15.135:0/1256859444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68d00ca5a0 0x7f68d0149360 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:44.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.742+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 >> 172.21.15.135:0/1256859444 conn(0x7f68d00ae010 msgr2=0x7f68d00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:44.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.742+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 shutdown_connections 2024-04-26T22:01:44.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:44.742+0000 7f68d676a640 1 -- 172.21.15.135:0/1256859444 wait complete. 2024-04-26T22:01:44.755 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-04-26T22:01:44.755 INFO:tasks.mds_thrash.fs.[a]:waiting for 67 secs before thrashing 2024-04-26T22:01:46.952 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-04-26T22:01:46.953 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:01:46.953 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:01:46.955 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:01:46.956 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:47.035 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-26T22:01:47.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.061+0000 7fc4bc322640 1 Processor -- start 2024-04-26T22:01:47.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.062+0000 7fc4bc322640 1 -- start start 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.062+0000 7fc4bc322640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b41555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.062+0000 7fc4bc322640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4b4073690 0x7fc4b4073b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4bc322640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b414f0b0 con 0x7fc4b414ecd0 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4bc322640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b4074c90 con 0x7fc4b4073690 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4bc322640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b40c6400 con 0x7fc4b40ac710 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba8bf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4b4073690 0x7fc4b4073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4b98bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b41555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba0be640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4b414ecd0 0x7fc4b40ab470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:39152/0 (socket says 172.21.15.135:39152) 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba0be640 1 -- 172.21.15.135:0/208704905 learned_addr learned my addr 172.21.15.135:0/208704905 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba8bf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4b4073690 0x7fc4b4073b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:34288/0 (socket says 172.21.15.135:34288) 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba8bf640 1 -- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 msgr2=0x7fc4b41555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba8bf640 1 --2- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b41555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba8bf640 1 -- 172.21.15.135:0/208704905 >> v1:172.21.15.139:6789/0 conn(0x7fc4b414ecd0 legacy=0x7fc4b40ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.063+0000 7fc4ba8bf640 1 -- 172.21.15.135:0/208704905 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40c66e0 con 0x7fc4b4073690 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.064+0000 7fc4ba8bf640 1 --2- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b4073b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009d90 tx=0x7fc4ac02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee1112d88d98026a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.064+0000 7fc4b90bc640 1 -- 172.21.15.135:0/208704905 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac007740 con 0x7fc4b4073690 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.064+0000 7fc4b90bc640 1 -- 172.21.15.135:0/208704905 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ac0078e0 con 0x7fc4b4073690 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.064+0000 7fc4b90bc640 1 -- 172.21.15.135:0/208704905 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac007cb0 con 0x7fc4b4073690 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.064+0000 7fc4bc322640 1 -- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 msgr2=0x7fc4b4073b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.064+0000 7fc4bc322640 1 --2- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b4073b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009d90 tx=0x7fc4ac02a530 comp rx=0 tx=0).stop 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.065+0000 7fc4bc322640 1 -- 172.21.15.135:0/208704905 shutdown_connections 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.065+0000 7fc4bc322640 1 --2- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b4073b10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.065+0000 7fc4bc322640 1 --2- 172.21.15.135:0/208704905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b41555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.065+0000 7fc4bc322640 1 -- 172.21.15.135:0/208704905 >> 172.21.15.135:0/208704905 conn(0x7fc4b40c6d80 msgr2=0x7fc4b40c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.065+0000 7fc4bc322640 1 -- 172.21.15.135:0/208704905 shutdown_connections 2024-04-26T22:01:47.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.065+0000 7fc4bc322640 1 -- 172.21.15.135:0/208704905 wait complete. 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.066+0000 7fc4bc322640 1 Processor -- start 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.066+0000 7fc4bc322640 1 -- start start 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4bc322640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4bc322640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4bc322640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b414ecd0 0x7fc4b41ee450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4bc322640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4155080 con 0x7fc4b414ecd0 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4ba0be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b414b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:53278/0 (socket says 172.21.15.135:53278) 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 -- 172.21.15.135:0/971901433 learned_addr learned my addr 172.21.15.135:0/971901433 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4154da0 con 0x7fc4b4073690 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b40c69c0 con 0x7fc4b40ac710 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4ba8bf640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b414ecd0 0x7fc4b41ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 -- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 msgr2=0x7fc4b40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b40ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 -- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b414ecd0 msgr2=0x7fc4b41ee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b414ecd0 0x7fc4b41ee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40c89e0 con 0x7fc4b40ac710 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4ba8bf640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b414ecd0 0x7fc4b41ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.067+0000 7fc4b98bd640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b414b9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400e870 tx=0x7fc4a400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.068+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4017070 con 0x7fc4b40ac710 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.068+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a400c940 con 0x7fc4b40ac710 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.068+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400cd10 con 0x7fc4b40ac710 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.068+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b40c8cc0 con 0x7fc4b40ac710 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.068+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41edc50 con 0x7fc4b40ac710 2024-04-26T22:01:47.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.069+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b40507a0 con 0x7fc4b40ac710 2024-04-26T22:01:47.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.070+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4a401a070 con 0x7fc4b40ac710 2024-04-26T22:01:47.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.070+0000 7fc4aaffd640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc49806c8f0 0x7fc49806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:47.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.070+0000 7fc4ba0be640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc49806c8f0 0x7fc49806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:47.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.070+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7fc4a408c0a0 con 0x7fc4b40ac710 2024-04-26T22:01:47.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.071+0000 7fc4ba0be640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc49806c8f0 0x7fc49806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0005880 tx=0x7fc4b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:47.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.075+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4a4059aa0 con 0x7fc4b40ac710 2024-04-26T22:01:47.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.509+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4b4076650 con 0x7fc4b40ac710 2024-04-26T22:01:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.510+0000 7fc4aaffd640 1 -- 172.21.15.135:0/971901433 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5675 (secure 0 0 0) 0x7fc4a405dda0 con 0x7fc4b40ac710 2024-04-26T22:01:47.510 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:47.510 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:01:47.510 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 22 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc49806c8f0 msgr2=0x7fc49806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc49806c8f0 0x7fc49806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0005880 tx=0x7fc4b0009040 comp rx=0 tx=0).stop 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 msgr2=0x7fc4b414b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b414b9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400e870 tx=0x7fc4a400ed30 comp rx=0 tx=0).stop 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 shutdown_connections 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc49806c8f0 0x7fc49806edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b414ecd0 0x7fc4b41ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40ac710 0x7fc4b414b9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 --2- 172.21.15.135:0/971901433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b4073690 0x7fc4b40ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:47.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 >> 172.21.15.135:0/971901433 conn(0x7fc4b40c6d80 msgr2=0x7fc4b4153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:47.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 shutdown_connections 2024-04-26T22:01:47.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:47.513+0000 7fc4bc322640 1 -- 172.21.15.135:0/971901433 wait complete. 2024-04-26T22:01:47.528 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-26T22:01:49.529 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:49.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.636+0000 7fc069d58640 1 Processor -- start 2024-04-26T22:01:49.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.636+0000 7fc069d58640 1 -- start start 2024-04-26T22:01:49.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc069d58640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc064156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc069d58640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc06415a1a0 0x7fc0640c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc069d58640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0640c8df0 con 0x7fc0640ca5a0 2024-04-26T22:01:49.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc069d58640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc0640c9100 con 0x7fc06415a1a0 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc069d58640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc064154540 con 0x7fc064154160 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc063fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc06415a1a0 0x7fc0640c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc062ffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc0640ca5a0 0x7fc0640c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55892/0 (socket says 172.21.15.135:55892) 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc062ffd640 1 -- 172.21.15.135:0/3768074546 learned_addr learned my addr 172.21.15.135:0/3768074546 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc0637fe640 1 --2- 172.21.15.135:0/3768074546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc064156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc0627fc640 1 -- 172.21.15.135:0/3768074546 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975963490 0 0) 0x7fc058001110 con 0x7fc0640ca5a0 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.637+0000 7fc0627fc640 1 -- 172.21.15.135:0/3768074546 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc04c003520 con 0x7fc0640ca5a0 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.638+0000 7fc063fff640 1 -- 172.21.15.135:0/3768074546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 msgr2=0x7fc064156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.638+0000 7fc063fff640 1 --2- 172.21.15.135:0/3768074546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc064156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.638+0000 7fc063fff640 1 -- 172.21.15.135:0/3768074546 >> v1:172.21.15.135:6789/0 conn(0x7fc0640ca5a0 legacy=0x7fc0640c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.638+0000 7fc063fff640 1 -- 172.21.15.135:0/3768074546 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06415a910 con 0x7fc06415a1a0 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc063fff640 1 --2- 172.21.15.135:0/3768074546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06415a1a0 0x7fc0640c9590 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009d90 tx=0x7fc05c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe6ccd6cb8cbd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc0627fc640 1 -- 172.21.15.135:0/3768074546 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c031600 con 0x7fc06415a1a0 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc0627fc640 1 -- 172.21.15.135:0/3768074546 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05c0317a0 con 0x7fc06415a1a0 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc0627fc640 1 -- 172.21.15.135:0/3768074546 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c031b70 con 0x7fc06415a1a0 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc069d58640 1 -- 172.21.15.135:0/3768074546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06415a1a0 msgr2=0x7fc0640c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc069d58640 1 --2- 172.21.15.135:0/3768074546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06415a1a0 0x7fc0640c9590 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009d90 tx=0x7fc05c02a610 comp rx=0 tx=0).stop 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc069d58640 1 -- 172.21.15.135:0/3768074546 shutdown_connections 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc069d58640 1 --2- 172.21.15.135:0/3768074546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06415a1a0 0x7fc0640c9590 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc069d58640 1 --2- 172.21.15.135:0/3768074546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc064156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.639+0000 7fc069d58640 1 -- 172.21.15.135:0/3768074546 >> 172.21.15.135:0/3768074546 conn(0x7fc0640ae010 msgr2=0x7fc0640ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.640+0000 7fc069d58640 1 -- 172.21.15.135:0/3768074546 shutdown_connections 2024-04-26T22:01:49.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.640+0000 7fc069d58640 1 -- 172.21.15.135:0/3768074546 wait complete. 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.641+0000 7fc069d58640 1 Processor -- start 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.641+0000 7fc069d58640 1 -- start start 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc069d58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 0x7fc0640b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc069d58640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc0641ed510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc069d58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc06415a1a0 0x7fc064159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc069d58640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc06415a630 con 0x7fc0640ca5a0 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc069d58640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc06415aed0 con 0x7fc06415a1a0 2024-04-26T22:01:49.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc069d58640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc0640c7510 con 0x7fc064154160 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 0x7fc0640b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 0x7fc0640b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34680/0 (socket says 172.21.15.135:34680) 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 -- 172.21.15.135:0/2512554326 learned_addr learned my addr 172.21.15.135:0/2512554326 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc062ffd640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc0641ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc063fff640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc06415a1a0 0x7fc064159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 -- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 msgr2=0x7fc0641ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc0641ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 -- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc06415a1a0 msgr2=0x7fc064159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc06415a1a0 0x7fc064159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.642+0000 7fc0637fe640 1 -- 172.21.15.135:0/2512554326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc064151af0 con 0x7fc0640ca5a0 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.643+0000 7fc0637fe640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 0x7fc0640b1bc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc05400e9b0 tx=0x7fc05400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.643+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054017070 con 0x7fc0640ca5a0 2024-04-26T22:01:49.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.643+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc064151dd0 con 0x7fc0640ca5a0 2024-04-26T22:01:49.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.643+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05400c9a0 con 0x7fc0640ca5a0 2024-04-26T22:01:49.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.643+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05400cd90 con 0x7fc0640ca5a0 2024-04-26T22:01:49.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.643+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc064152390 con 0x7fc0640ca5a0 2024-04-26T22:01:49.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.644+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0640507a0 con 0x7fc0640ca5a0 2024-04-26T22:01:49.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.645+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc05401a070 con 0x7fc0640ca5a0 2024-04-26T22:01:49.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.646+0000 7fc04bfff640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc05006c920 0x7fc05006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.646+0000 7fc062ffd640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc05006c920 0x7fc05006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.646+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7fc05408c2c0 con 0x7fc0640ca5a0 2024-04-26T22:01:49.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.647+0000 7fc062ffd640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc05006c920 0x7fc05006ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc064147720 tx=0x7fc058007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.652+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc054059c40 con 0x7fc0640ca5a0 2024-04-26T22:01:49.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.877+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0640ca380 con 0x7fc0640ca5a0 2024-04-26T22:01:49.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.878+0000 7fc04bfff640 1 -- 172.21.15.135:0/2512554326 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7fc05405df40 con 0x7fc0640ca5a0 2024-04-26T22:01:49.878 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:01:49.878 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:01:49.878 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc05006c920 msgr2=0x7fc05006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc05006c920 0x7fc05006ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc064147720 tx=0x7fc058007d80 comp rx=0 tx=0).stop 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 msgr2=0x7fc0640b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 0x7fc0640b1bc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc05400e9b0 tx=0x7fc05400ee70 comp rx=0 tx=0).stop 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 shutdown_connections 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc05006c920 0x7fc05006ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc06415a1a0 0x7fc064159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc064154160 0x7fc0641ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 --2- 172.21.15.135:0/2512554326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0640ca5a0 0x7fc0640b1bc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 >> 172.21.15.135:0/2512554326 conn(0x7fc0640ae010 msgr2=0x7fc0640cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 shutdown_connections 2024-04-26T22:01:49.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:01:49.880+0000 7fc069d58640 1 -- 172.21.15.135:0/2512554326 wait complete. 2024-04-26T22:01:49.891 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-04-26T22:01:49.891 INFO:tasks.mds_thrash.fs.[b]:waiting for 83 secs before thrashing 2024-04-26T22:01:58.193 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:01:58.196 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:02:10.569 INFO:tasks.workunit.client.0.smithi135.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T22:02:10.569 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.569 INFO:tasks.workunit.client.0.smithi135.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: git switch -c 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr:Or undo this operation with: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: git switch - 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.570 INFO:tasks.workunit.client.0.smithi135.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T22:02:10.571 INFO:tasks.workunit.client.0.smithi135.stderr: 2024-04-26T22:02:10.571 INFO:tasks.workunit.client.0.smithi135.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T22:02:10.575 DEBUG:teuthology.orchestra.run.smithi135:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T22:02:10.605 INFO:tasks.workunit.client.0.smithi135.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T22:02:10.607 INFO:tasks.workunit.client.0.smithi135.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T22:02:10.607 INFO:tasks.workunit.client.0.smithi135.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T22:02:11.151 INFO:tasks.workunit.client.0.smithi135.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T22:02:11.207 INFO:tasks.workunit.client.0.smithi135.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T22:02:11.246 INFO:tasks.workunit.client.0.smithi135.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T22:02:11.248 INFO:tasks.workunit.client.0.smithi135.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T22:02:11.248 INFO:tasks.workunit.client.0.smithi135.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T22:02:11.286 INFO:tasks.workunit.client.0.smithi135.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T22:02:11.289 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T22:02:11.289 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T22:02:11.345 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-04-26T22:02:11.346 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-26T22:02:11.346 DEBUG:teuthology.orchestra.run.smithi135:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-26T22:02:11.405 INFO:tasks.workunit.client.0.smithi135.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-26T22:02:11.406 INFO:tasks.workunit.client.0.smithi135.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-04-26T22:02:11.406 INFO:tasks.workunit.client.0.smithi135.stderr:+ trap -- retry EXIT 2024-04-26T22:02:11.406 INFO:tasks.workunit.client.0.smithi135.stderr:+ rm -rf ffsb 2024-04-26T22:02:11.406 INFO:tasks.workunit.client.0.smithi135.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-26T22:02:11.409 INFO:tasks.workunit.client.0.smithi135.stderr:Cloning into 'ffsb'... 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.594 INFO:tasks.workunit.client.1.smithi139.stderr: git switch -c 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr:Or undo this operation with: 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr: git switch - 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr: 2024-04-26T22:02:11.595 INFO:tasks.workunit.client.1.smithi139.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T22:02:11.600 DEBUG:teuthology.orchestra.run.smithi139:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-26T22:02:11.676 INFO:tasks.workunit.client.1.smithi139.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T22:02:11.678 INFO:tasks.workunit.client.1.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-26T22:02:11.678 INFO:tasks.workunit.client.1.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T22:02:12.202 INFO:tasks.workunit.client.1.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T22:02:12.217 INFO:tasks.workunit.client.0.smithi135.stderr:+ trap - EXIT 2024-04-26T22:02:12.217 INFO:tasks.workunit.client.0.smithi135.stderr:+ cd ffsb 2024-04-26T22:02:12.217 INFO:tasks.workunit.client.0.smithi135.stderr:+ ./configure 2024-04-26T22:02:12.256 INFO:tasks.workunit.client.1.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T22:02:12.296 INFO:tasks.workunit.client.1.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-26T22:02:12.298 INFO:tasks.workunit.client.1.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-26T22:02:12.298 INFO:tasks.workunit.client.1.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T22:02:12.337 INFO:tasks.workunit.client.1.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-26T22:02:12.339 INFO:tasks.workunit.client.0.smithi135.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-26T22:02:12.340 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T22:02:12.340 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-26T22:02:12.355 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.1... 2024-04-26T22:02:12.356 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-26T22:02:12.356 DEBUG:teuthology.orchestra.run.smithi139:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh 2024-04-26T22:02:12.415 INFO:tasks.workunit.client.1.smithi139.stderr:++ dirname /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh 2024-04-26T22:02:12.416 INFO:tasks.workunit.client.1.smithi139.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.1/qa/workunits/suites 2024-04-26T22:02:12.416 INFO:tasks.workunit.client.1.smithi139.stderr:+ trap -- retry EXIT 2024-04-26T22:02:12.416 INFO:tasks.workunit.client.1.smithi139.stderr:+ rm -rf ffsb 2024-04-26T22:02:12.416 INFO:tasks.workunit.client.1.smithi139.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-26T22:02:12.418 INFO:tasks.workunit.client.1.smithi139.stderr:Cloning into 'ffsb'... 2024-04-26T22:02:13.188 INFO:tasks.workunit.client.1.smithi139.stderr:+ trap - EXIT 2024-04-26T22:02:13.188 INFO:tasks.workunit.client.1.smithi139.stderr:+ cd ffsb 2024-04-26T22:02:13.188 INFO:tasks.workunit.client.1.smithi139.stderr:+ ./configure 2024-04-26T22:02:13.305 INFO:tasks.workunit.client.1.smithi139.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-26T22:02:13.345 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether build environment is sane... yes 2024-04-26T22:02:13.353 INFO:tasks.workunit.client.0.smithi135.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-26T22:02:13.353 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gawk... gawk 2024-04-26T22:02:13.366 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether make sets $(MAKE)... yes 2024-04-26T22:02:13.392 INFO:tasks.workunit.client.0.smithi135.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:13.393 INFO:tasks.workunit.client.0.smithi135.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:13.402 INFO:tasks.workunit.client.0.smithi135.stdout:checking for style of include used by make... GNU 2024-04-26T22:02:13.406 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc... gcc 2024-04-26T22:02:13.453 INFO:tasks.workunit.client.0.smithi135.stdout:checking for C compiler default output file name... a.out 2024-04-26T22:02:13.454 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether the C compiler works... yes 2024-04-26T22:02:13.456 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether we are cross compiling... no 2024-04-26T22:02:13.490 INFO:tasks.workunit.client.0.smithi135.stdout:checking for suffix of executables... 2024-04-26T22:02:13.521 INFO:tasks.workunit.client.0.smithi135.stdout:checking for suffix of object files... o 2024-04-26T22:02:13.549 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T22:02:13.577 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether gcc accepts -g... yes 2024-04-26T22:02:13.644 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T22:02:13.730 INFO:tasks.workunit.client.0.smithi135.stdout:checking dependency style of gcc... gcc3 2024-04-26T22:02:13.792 INFO:tasks.workunit.client.0.smithi135.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T22:02:13.831 INFO:tasks.workunit.client.0.smithi135.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T22:02:13.832 INFO:tasks.workunit.client.0.smithi135.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T22:02:13.842 INFO:tasks.workunit.client.0.smithi135.stdout:checking for AIX... no 2024-04-26T22:02:13.857 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc... (cached) gcc 2024-04-26T22:02:13.864 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T22:02:13.865 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T22:02:13.865 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T22:02:13.865 INFO:tasks.workunit.client.0.smithi135.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-26T22:02:13.911 INFO:tasks.workunit.client.0.smithi135.stdout:checking for main in -lm... yes 2024-04-26T22:02:13.957 INFO:tasks.workunit.client.0.smithi135.stdout:checking for main in -lpthread... yes 2024-04-26T22:02:14.101 INFO:tasks.workunit.client.0.smithi135.stdout:checking for ANSI C header files... yes 2024-04-26T22:02:14.234 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-26T22:02:14.276 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/types.h... yes 2024-04-26T22:02:14.310 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether build environment is sane... yes 2024-04-26T22:02:14.318 INFO:tasks.workunit.client.1.smithi139.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-26T22:02:14.318 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gawk... gawk 2024-04-26T22:02:14.318 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/stat.h... yes 2024-04-26T22:02:14.330 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether make sets $(MAKE)... yes 2024-04-26T22:02:14.357 INFO:tasks.workunit.client.1.smithi139.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:14.357 INFO:tasks.workunit.client.1.smithi139.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:14.363 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stdlib.h... yes 2024-04-26T22:02:14.366 INFO:tasks.workunit.client.1.smithi139.stdout:checking for style of include used by make... GNU 2024-04-26T22:02:14.372 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc... gcc 2024-04-26T22:02:14.411 INFO:tasks.workunit.client.0.smithi135.stdout:checking for string.h... yes 2024-04-26T22:02:14.419 INFO:tasks.workunit.client.1.smithi139.stdout:checking for C compiler default output file name... a.out 2024-04-26T22:02:14.421 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether the C compiler works... yes 2024-04-26T22:02:14.424 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether we are cross compiling... no 2024-04-26T22:02:14.456 INFO:tasks.workunit.client.0.smithi135.stdout:checking for memory.h... yes 2024-04-26T22:02:14.458 INFO:tasks.workunit.client.1.smithi139.stdout:checking for suffix of executables... 2024-04-26T22:02:14.489 INFO:tasks.workunit.client.1.smithi139.stdout:checking for suffix of object files... o 2024-04-26T22:02:14.499 INFO:tasks.workunit.client.0.smithi135.stdout:checking for strings.h... yes 2024-04-26T22:02:14.516 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T22:02:14.543 INFO:tasks.workunit.client.0.smithi135.stdout:checking for inttypes.h... yes 2024-04-26T22:02:14.544 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether gcc accepts -g... yes 2024-04-26T22:02:14.588 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stdint.h... yes 2024-04-26T22:02:14.634 INFO:tasks.workunit.client.0.smithi135.stdout:checking for unistd.h... yes 2024-04-26T22:02:14.666 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T22:02:14.683 INFO:tasks.workunit.client.0.smithi135.stdout:checking pthread.h usability... yes 2024-04-26T22:02:14.704 INFO:tasks.workunit.client.0.smithi135.stdout:checking pthread.h presence... yes 2024-04-26T22:02:14.704 INFO:tasks.workunit.client.0.smithi135.stdout:checking for pthread.h... yes 2024-04-26T22:02:14.755 INFO:tasks.workunit.client.0.smithi135.stdout:checking fcntl.h usability... yes 2024-04-26T22:02:14.763 INFO:tasks.workunit.client.1.smithi139.stdout:checking dependency style of gcc... gcc3 2024-04-26T22:02:14.774 INFO:tasks.workunit.client.0.smithi135.stdout:checking fcntl.h presence... yes 2024-04-26T22:02:14.774 INFO:tasks.workunit.client.0.smithi135.stdout:checking for fcntl.h... yes 2024-04-26T22:02:14.825 INFO:tasks.workunit.client.0.smithi135.stdout:checking limits.h usability... yes 2024-04-26T22:02:14.844 INFO:tasks.workunit.client.0.smithi135.stdout:checking limits.h presence... yes 2024-04-26T22:02:14.845 INFO:tasks.workunit.client.0.smithi135.stdout:checking for limits.h... yes 2024-04-26T22:02:14.848 INFO:tasks.workunit.client.1.smithi139.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T22:02:14.862 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stdint.h... (cached) yes 2024-04-26T22:02:14.884 INFO:tasks.workunit.client.1.smithi139.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T22:02:14.885 INFO:tasks.workunit.client.1.smithi139.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T22:02:14.893 INFO:tasks.workunit.client.1.smithi139.stdout:checking for AIX... no 2024-04-26T22:02:14.907 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/time.h usability... yes 2024-04-26T22:02:14.910 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc... (cached) gcc 2024-04-26T22:02:14.918 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T22:02:14.918 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T22:02:14.918 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T22:02:14.918 INFO:tasks.workunit.client.1.smithi139.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-26T22:02:14.925 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/time.h presence... yes 2024-04-26T22:02:14.926 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/time.h... yes 2024-04-26T22:02:14.938 INFO:tasks.workunit.client.0.smithi135.stdout:checking for unistd.h... (cached) yes 2024-04-26T22:02:14.963 INFO:tasks.workunit.client.1.smithi139.stdout:checking for main in -lm... yes 2024-04-26T22:02:14.982 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/vfs.h usability... yes 2024-04-26T22:02:15.003 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/vfs.h presence... yes 2024-04-26T22:02:15.003 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/vfs.h... yes 2024-04-26T22:02:15.011 INFO:tasks.workunit.client.1.smithi139.stdout:checking for main in -lpthread... yes 2024-04-26T22:02:15.051 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/limits.h usability... no 2024-04-26T22:02:15.070 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/limits.h presence... no 2024-04-26T22:02:15.070 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/limits.h... no 2024-04-26T22:02:15.102 INFO:tasks.workunit.client.0.smithi135.stdout:checking for an ANSI C-conforming const... yes 2024-04-26T22:02:15.133 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-26T22:02:15.134 INFO:tasks.workunit.client.0.smithi135.stdout:checking for function prototypes... yes 2024-04-26T22:02:15.154 INFO:tasks.workunit.client.1.smithi139.stdout:checking for ANSI C header files... yes 2024-04-26T22:02:15.159 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether setvbuf arguments are reversed... no 2024-04-26T22:02:15.202 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-26T22:02:15.227 INFO:tasks.workunit.client.0.smithi135.stdout:checking for system... yes 2024-04-26T22:02:15.242 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/types.h... yes 2024-04-26T22:02:15.282 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/stat.h... yes 2024-04-26T22:02:15.287 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gettimeofday... yes 2024-04-26T22:02:15.325 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stdlib.h... yes 2024-04-26T22:02:15.345 INFO:tasks.workunit.client.0.smithi135.stdout:checking for mkdir... yes 2024-04-26T22:02:15.368 INFO:tasks.workunit.client.1.smithi139.stdout:checking for string.h... yes 2024-04-26T22:02:15.405 INFO:tasks.workunit.client.0.smithi135.stdout:checking for strerror... yes 2024-04-26T22:02:15.411 INFO:tasks.workunit.client.1.smithi139.stdout:checking for memory.h... yes 2024-04-26T22:02:15.452 INFO:tasks.workunit.client.1.smithi139.stdout:checking for strings.h... yes 2024-04-26T22:02:15.463 INFO:tasks.workunit.client.0.smithi135.stdout:checking for open64... yes 2024-04-26T22:02:15.494 INFO:tasks.workunit.client.1.smithi139.stdout:checking for inttypes.h... yes 2024-04-26T22:02:15.521 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stat64... yes 2024-04-26T22:02:15.537 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stdint.h... yes 2024-04-26T22:02:15.580 INFO:tasks.workunit.client.1.smithi139.stdout:checking for unistd.h... yes 2024-04-26T22:02:15.583 INFO:tasks.workunit.client.0.smithi135.stdout:checking for fseeko64... yes 2024-04-26T22:02:15.629 INFO:tasks.workunit.client.1.smithi139.stdout:checking pthread.h usability... yes 2024-04-26T22:02:15.643 INFO:tasks.workunit.client.0.smithi135.stdout:checking for lrand48_r... yes 2024-04-26T22:02:15.648 INFO:tasks.workunit.client.1.smithi139.stdout:checking pthread.h presence... yes 2024-04-26T22:02:15.649 INFO:tasks.workunit.client.1.smithi139.stdout:checking for pthread.h... yes 2024-04-26T22:02:15.695 INFO:tasks.workunit.client.1.smithi139.stdout:checking fcntl.h usability... yes 2024-04-26T22:02:15.703 INFO:tasks.workunit.client.0.smithi135.stdout:checking for srand48_r... yes 2024-04-26T22:02:15.713 INFO:tasks.workunit.client.1.smithi139.stdout:checking fcntl.h presence... yes 2024-04-26T22:02:15.713 INFO:tasks.workunit.client.1.smithi139.stdout:checking for fcntl.h... yes 2024-04-26T22:02:15.728 INFO:tasks.workunit.client.0.smithi135.stdout:configure: creating ./config.status 2024-04-26T22:02:15.760 INFO:tasks.workunit.client.1.smithi139.stdout:checking limits.h usability... yes 2024-04-26T22:02:15.777 INFO:tasks.workunit.client.1.smithi139.stdout:checking limits.h presence... yes 2024-04-26T22:02:15.778 INFO:tasks.workunit.client.1.smithi139.stdout:checking for limits.h... yes 2024-04-26T22:02:15.791 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stdint.h... (cached) yes 2024-04-26T22:02:15.819 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: creating Makefile 2024-04-26T22:02:15.831 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: creating config.h 2024-04-26T22:02:15.839 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/time.h usability... yes 2024-04-26T22:02:15.848 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: executing depfiles commands 2024-04-26T22:02:15.857 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/time.h presence... yes 2024-04-26T22:02:15.857 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/time.h... yes 2024-04-26T22:02:15.871 INFO:tasks.workunit.client.1.smithi139.stdout:checking for unistd.h... (cached) yes 2024-04-26T22:02:15.890 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: executing default commands 2024-04-26T22:02:15.912 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/vfs.h usability... yes 2024-04-26T22:02:15.930 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/vfs.h presence... yes 2024-04-26T22:02:15.931 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/vfs.h... yes 2024-04-26T22:02:15.932 INFO:tasks.workunit.client.0.smithi135.stderr:+ make 2024-04-26T22:02:15.939 INFO:tasks.workunit.client.0.smithi135.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-04-26T22:02:15.941 INFO:tasks.workunit.client.0.smithi135.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-26T22:02:15.942 INFO:tasks.workunit.client.0.smithi135.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-26T22:02:15.942 INFO:tasks.workunit.client.0.smithi135.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-26T22:02:15.942 INFO:tasks.workunit.client.0.smithi135.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-26T22:02:15.942 INFO:tasks.workunit.client.0.smithi135.stderr: any GNU archive site. 2024-04-26T22:02:15.944 INFO:tasks.workunit.client.0.smithi135.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-26T22:02:15.945 INFO:tasks.workunit.client.0.smithi135.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-26T22:02:15.945 INFO:tasks.workunit.client.0.smithi135.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-26T22:02:15.945 INFO:tasks.workunit.client.0.smithi135.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-26T22:02:15.946 INFO:tasks.workunit.client.0.smithi135.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-26T22:02:15.946 INFO:tasks.workunit.client.0.smithi135.stderr: Grab them from any GNU archive site. 2024-04-26T22:02:15.978 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/limits.h usability... no 2024-04-26T22:02:15.980 INFO:tasks.workunit.client.0.smithi135.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-04-26T22:02:15.997 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/limits.h presence... no 2024-04-26T22:02:15.997 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/limits.h... no 2024-04-26T22:02:16.027 INFO:tasks.workunit.client.1.smithi139.stdout:checking for an ANSI C-conforming const... yes 2024-04-26T22:02:16.057 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-26T22:02:16.057 INFO:tasks.workunit.client.1.smithi139.stdout:checking for function prototypes... yes 2024-04-26T22:02:16.081 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether setvbuf arguments are reversed... no 2024-04-26T22:02:16.146 INFO:tasks.workunit.client.1.smithi139.stdout:checking for system... yes 2024-04-26T22:02:16.202 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gettimeofday... yes 2024-04-26T22:02:16.244 INFO:tasks.workunit.client.0.smithi135.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-26T22:02:16.244 INFO:tasks.workunit.client.0.smithi135.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-26T22:02:16.244 INFO:tasks.workunit.client.0.smithi135.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-26T22:02:16.244 INFO:tasks.workunit.client.0.smithi135.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-26T22:02:16.260 INFO:tasks.workunit.client.1.smithi139.stdout:checking for mkdir... yes 2024-04-26T22:02:16.315 INFO:tasks.workunit.client.1.smithi139.stdout:checking for strerror... yes 2024-04-26T22:02:16.357 INFO:tasks.workunit.client.0.smithi135.stdout:/bin/sh ./config.status --recheck 2024-04-26T22:02:16.376 INFO:tasks.workunit.client.1.smithi139.stdout:checking for open64... yes 2024-04-26T22:02:16.405 INFO:tasks.workunit.client.0.smithi135.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-26T22:02:16.438 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stat64... yes 2024-04-26T22:02:16.497 INFO:tasks.workunit.client.1.smithi139.stdout:checking for fseeko64... yes 2024-04-26T22:02:16.502 INFO:tasks.workunit.client.0.smithi135.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-26T22:02:16.552 INFO:tasks.workunit.client.1.smithi139.stdout:checking for lrand48_r... yes 2024-04-26T22:02:16.633 INFO:tasks.workunit.client.1.smithi139.stdout:checking for srand48_r... yes 2024-04-26T22:02:16.662 INFO:tasks.workunit.client.1.smithi139.stdout:configure: creating ./config.status 2024-04-26T22:02:16.746 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: creating Makefile 2024-04-26T22:02:16.758 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: creating config.h 2024-04-26T22:02:16.773 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: executing depfiles commands 2024-04-26T22:02:16.826 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: executing default commands 2024-04-26T22:02:16.880 INFO:tasks.workunit.client.1.smithi139.stderr:+ make 2024-04-26T22:02:16.885 INFO:tasks.workunit.client.1.smithi139.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run aclocal-1.10 2024-04-26T22:02:16.887 INFO:tasks.workunit.client.1.smithi139.stderr:/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-26T22:02:16.887 INFO:tasks.workunit.client.1.smithi139.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-26T22:02:16.888 INFO:tasks.workunit.client.1.smithi139.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-26T22:02:16.888 INFO:tasks.workunit.client.1.smithi139.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-26T22:02:16.888 INFO:tasks.workunit.client.1.smithi139.stderr: any GNU archive site. 2024-04-26T22:02:16.889 INFO:tasks.workunit.client.1.smithi139.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-26T22:02:16.890 INFO:tasks.workunit.client.1.smithi139.stderr:/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-26T22:02:16.891 INFO:tasks.workunit.client.1.smithi139.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-26T22:02:16.891 INFO:tasks.workunit.client.1.smithi139.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-26T22:02:16.891 INFO:tasks.workunit.client.1.smithi139.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-26T22:02:16.891 INFO:tasks.workunit.client.1.smithi139.stderr: Grab them from any GNU archive site. 2024-04-26T22:02:16.924 INFO:tasks.workunit.client.1.smithi139.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run autoconf 2024-04-26T22:02:17.222 INFO:tasks.workunit.client.1.smithi139.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-26T22:02:17.223 INFO:tasks.workunit.client.1.smithi139.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-26T22:02:17.223 INFO:tasks.workunit.client.1.smithi139.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-26T22:02:17.223 INFO:tasks.workunit.client.1.smithi139.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-26T22:02:17.335 INFO:tasks.workunit.client.1.smithi139.stdout:/bin/sh ./config.status --recheck 2024-04-26T22:02:17.383 INFO:tasks.workunit.client.1.smithi139.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-26T22:02:17.474 INFO:tasks.workunit.client.1.smithi139.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-26T22:02:17.509 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether build environment is sane... yes 2024-04-26T22:02:17.517 INFO:tasks.workunit.client.0.smithi135.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-26T22:02:17.518 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gawk... gawk 2024-04-26T22:02:17.556 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether make sets $(MAKE)... yes 2024-04-26T22:02:17.583 INFO:tasks.workunit.client.0.smithi135.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:17.583 INFO:tasks.workunit.client.0.smithi135.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:17.593 INFO:tasks.workunit.client.0.smithi135.stdout:checking for style of include used by make... GNU 2024-04-26T22:02:17.599 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc... gcc 2024-04-26T22:02:17.676 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether the C compiler works... yes 2024-04-26T22:02:17.676 INFO:tasks.workunit.client.0.smithi135.stdout:checking for C compiler default output file name... a.out 2024-04-26T22:02:17.711 INFO:tasks.workunit.client.0.smithi135.stdout:checking for suffix of executables... 2024-04-26T22:02:17.746 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether we are cross compiling... no 2024-04-26T22:02:17.781 INFO:tasks.workunit.client.0.smithi135.stdout:checking for suffix of object files... o 2024-04-26T22:02:17.807 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T22:02:17.838 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether gcc accepts -g... yes 2024-04-26T22:02:17.872 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T22:02:17.958 INFO:tasks.workunit.client.0.smithi135.stdout:checking dependency style of gcc... gcc3 2024-04-26T22:02:17.999 INFO:tasks.workunit.client.0.smithi135.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T22:02:18.044 INFO:tasks.workunit.client.0.smithi135.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T22:02:18.045 INFO:tasks.workunit.client.0.smithi135.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T22:02:18.160 INFO:tasks.workunit.client.0.smithi135.stdout:checking for ANSI C header files... yes 2024-04-26T22:02:18.191 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/types.h... yes 2024-04-26T22:02:18.229 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/stat.h... yes 2024-04-26T22:02:18.265 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stdlib.h... yes 2024-04-26T22:02:18.302 INFO:tasks.workunit.client.0.smithi135.stdout:checking for string.h... yes 2024-04-26T22:02:18.339 INFO:tasks.workunit.client.0.smithi135.stdout:checking for memory.h... yes 2024-04-26T22:02:18.376 INFO:tasks.workunit.client.0.smithi135.stdout:checking for strings.h... yes 2024-04-26T22:02:18.414 INFO:tasks.workunit.client.0.smithi135.stdout:checking for inttypes.h... yes 2024-04-26T22:02:18.451 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stdint.h... yes 2024-04-26T22:02:18.479 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether build environment is sane... yes 2024-04-26T22:02:18.483 INFO:tasks.workunit.client.1.smithi139.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-26T22:02:18.483 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gawk... gawk 2024-04-26T22:02:18.490 INFO:tasks.workunit.client.0.smithi135.stdout:checking for unistd.h... yes 2024-04-26T22:02:18.502 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether make sets $(MAKE)... yes 2024-04-26T22:02:18.527 INFO:tasks.workunit.client.1.smithi139.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:18.527 INFO:tasks.workunit.client.1.smithi139.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-26T22:02:18.533 INFO:tasks.workunit.client.0.smithi135.stdout:checking minix/config.h usability... no 2024-04-26T22:02:18.539 INFO:tasks.workunit.client.1.smithi139.stdout:checking for style of include used by make... GNU 2024-04-26T22:02:18.542 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc... gcc 2024-04-26T22:02:18.560 INFO:tasks.workunit.client.0.smithi135.stdout:checking minix/config.h presence... no 2024-04-26T22:02:18.560 INFO:tasks.workunit.client.0.smithi135.stdout:checking for minix/config.h... no 2024-04-26T22:02:18.594 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-26T22:02:18.595 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc... (cached) gcc 2024-04-26T22:02:18.619 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether the C compiler works... yes 2024-04-26T22:02:18.619 INFO:tasks.workunit.client.1.smithi139.stdout:checking for C compiler default output file name... a.out 2024-04-26T22:02:18.626 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T22:02:18.626 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T22:02:18.627 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T22:02:18.627 INFO:tasks.workunit.client.0.smithi135.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-26T22:02:18.655 INFO:tasks.workunit.client.1.smithi139.stdout:checking for suffix of executables... 2024-04-26T22:02:18.667 INFO:tasks.workunit.client.0.smithi135.stdout:checking for main in -lm... yes 2024-04-26T22:02:18.691 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether we are cross compiling... no 2024-04-26T22:02:18.709 INFO:tasks.workunit.client.0.smithi135.stdout:checking for main in -lpthread... yes 2024-04-26T22:02:18.710 INFO:tasks.workunit.client.0.smithi135.stdout:checking for ANSI C header files... (cached) yes 2024-04-26T22:02:18.726 INFO:tasks.workunit.client.1.smithi139.stdout:checking for suffix of object files... o 2024-04-26T22:02:18.740 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-26T22:02:18.751 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T22:02:18.772 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether gcc accepts -g... yes 2024-04-26T22:02:18.785 INFO:tasks.workunit.client.0.smithi135.stdout:checking pthread.h usability... yes 2024-04-26T22:02:18.802 INFO:tasks.workunit.client.0.smithi135.stdout:checking pthread.h presence... yes 2024-04-26T22:02:18.802 INFO:tasks.workunit.client.0.smithi135.stdout:checking for pthread.h... yes 2024-04-26T22:02:18.817 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T22:02:18.851 INFO:tasks.workunit.client.0.smithi135.stdout:checking fcntl.h usability... yes 2024-04-26T22:02:18.870 INFO:tasks.workunit.client.0.smithi135.stdout:checking fcntl.h presence... yes 2024-04-26T22:02:18.870 INFO:tasks.workunit.client.0.smithi135.stdout:checking for fcntl.h... yes 2024-04-26T22:02:18.903 INFO:tasks.workunit.client.1.smithi139.stdout:checking dependency style of gcc... gcc3 2024-04-26T22:02:18.917 INFO:tasks.workunit.client.0.smithi135.stdout:checking limits.h usability... yes 2024-04-26T22:02:18.932 INFO:tasks.workunit.client.0.smithi135.stdout:checking limits.h presence... yes 2024-04-26T22:02:18.932 INFO:tasks.workunit.client.0.smithi135.stdout:checking for limits.h... yes 2024-04-26T22:02:18.942 INFO:tasks.workunit.client.1.smithi139.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T22:02:18.951 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stdint.h... (cached) yes 2024-04-26T22:02:18.985 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/time.h usability... yes 2024-04-26T22:02:18.986 INFO:tasks.workunit.client.1.smithi139.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T22:02:18.987 INFO:tasks.workunit.client.1.smithi139.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T22:02:19.002 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/time.h presence... yes 2024-04-26T22:02:19.002 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/time.h... yes 2024-04-26T22:02:19.015 INFO:tasks.workunit.client.0.smithi135.stdout:checking for unistd.h... (cached) yes 2024-04-26T22:02:19.049 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/vfs.h usability... yes 2024-04-26T22:02:19.064 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/vfs.h presence... yes 2024-04-26T22:02:19.064 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/vfs.h... yes 2024-04-26T22:02:19.112 INFO:tasks.workunit.client.1.smithi139.stdout:checking for ANSI C header files... yes 2024-04-26T22:02:19.121 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/limits.h usability... no 2024-04-26T22:02:19.144 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/types.h... yes 2024-04-26T22:02:19.148 INFO:tasks.workunit.client.0.smithi135.stdout:checking sys/limits.h presence... no 2024-04-26T22:02:19.148 INFO:tasks.workunit.client.0.smithi135.stdout:checking for sys/limits.h... no 2024-04-26T22:02:19.172 INFO:tasks.workunit.client.0.smithi135.stdout:checking for an ANSI C-conforming const... yes 2024-04-26T22:02:19.178 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/stat.h... yes 2024-04-26T22:02:19.199 INFO:tasks.workunit.client.0.smithi135.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-26T22:02:19.212 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stdlib.h... yes 2024-04-26T22:02:19.247 INFO:tasks.workunit.client.1.smithi139.stdout:checking for string.h... yes 2024-04-26T22:02:19.256 INFO:tasks.workunit.client.0.smithi135.stdout:checking for system... yes 2024-04-26T22:02:19.284 INFO:tasks.workunit.client.1.smithi139.stdout:checking for memory.h... yes 2024-04-26T22:02:19.308 INFO:tasks.workunit.client.0.smithi135.stdout:checking for gettimeofday... yes 2024-04-26T22:02:19.319 INFO:tasks.workunit.client.1.smithi139.stdout:checking for strings.h... yes 2024-04-26T22:02:19.356 INFO:tasks.workunit.client.1.smithi139.stdout:checking for inttypes.h... yes 2024-04-26T22:02:19.361 INFO:tasks.workunit.client.0.smithi135.stdout:checking for mkdir... yes 2024-04-26T22:02:19.392 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stdint.h... yes 2024-04-26T22:02:19.416 INFO:tasks.workunit.client.0.smithi135.stdout:checking for strerror... yes 2024-04-26T22:02:19.431 INFO:tasks.workunit.client.1.smithi139.stdout:checking for unistd.h... yes 2024-04-26T22:02:19.470 INFO:tasks.workunit.client.0.smithi135.stdout:checking for open64... yes 2024-04-26T22:02:19.471 INFO:tasks.workunit.client.1.smithi139.stdout:checking minix/config.h usability... no 2024-04-26T22:02:19.495 INFO:tasks.workunit.client.1.smithi139.stdout:checking minix/config.h presence... no 2024-04-26T22:02:19.495 INFO:tasks.workunit.client.1.smithi139.stdout:checking for minix/config.h... no 2024-04-26T22:02:19.524 INFO:tasks.workunit.client.0.smithi135.stdout:checking for stat64... yes 2024-04-26T22:02:19.527 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-26T22:02:19.528 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc... (cached) gcc 2024-04-26T22:02:19.556 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T22:02:19.556 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T22:02:19.556 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T22:02:19.556 INFO:tasks.workunit.client.1.smithi139.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-26T22:02:19.577 INFO:tasks.workunit.client.0.smithi135.stdout:checking for fseeko64... yes 2024-04-26T22:02:19.598 INFO:tasks.workunit.client.1.smithi139.stdout:checking for main in -lm... yes 2024-04-26T22:02:19.635 INFO:tasks.workunit.client.0.smithi135.stdout:checking for lrand48_r... yes 2024-04-26T22:02:19.639 INFO:tasks.workunit.client.1.smithi139.stdout:checking for main in -lpthread... yes 2024-04-26T22:02:19.640 INFO:tasks.workunit.client.1.smithi139.stdout:checking for ANSI C header files... (cached) yes 2024-04-26T22:02:19.670 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-26T22:02:19.689 INFO:tasks.workunit.client.0.smithi135.stdout:checking for srand48_r... yes 2024-04-26T22:02:19.714 INFO:tasks.workunit.client.1.smithi139.stdout:checking pthread.h usability... yes 2024-04-26T22:02:19.714 INFO:tasks.workunit.client.0.smithi135.stdout:configure: creating ./config.status 2024-04-26T22:02:19.731 INFO:tasks.workunit.client.1.smithi139.stdout:checking pthread.h presence... yes 2024-04-26T22:02:19.731 INFO:tasks.workunit.client.1.smithi139.stdout:checking for pthread.h... yes 2024-04-26T22:02:19.773 INFO:tasks.workunit.client.1.smithi139.stdout:checking fcntl.h usability... yes 2024-04-26T22:02:19.775 INFO:tasks.workunit.client.0.smithi135.stdout: /bin/sh ./config.status 2024-04-26T22:02:19.791 INFO:tasks.workunit.client.1.smithi139.stdout:checking fcntl.h presence... yes 2024-04-26T22:02:19.791 INFO:tasks.workunit.client.1.smithi139.stdout:checking for fcntl.h... yes 2024-04-26T22:02:19.822 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: creating Makefile 2024-04-26T22:02:19.833 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: creating config.h 2024-04-26T22:02:19.838 INFO:tasks.workunit.client.1.smithi139.stdout:checking limits.h usability... yes 2024-04-26T22:02:19.853 INFO:tasks.workunit.client.1.smithi139.stdout:checking limits.h presence... yes 2024-04-26T22:02:19.853 INFO:tasks.workunit.client.1.smithi139.stdout:checking for limits.h... yes 2024-04-26T22:02:19.857 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: executing depfiles commands 2024-04-26T22:02:19.866 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stdint.h... (cached) yes 2024-04-26T22:02:19.866 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: executing default commands 2024-04-26T22:02:19.891 INFO:tasks.workunit.client.0.smithi135.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-04-26T22:02:19.900 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/time.h usability... yes 2024-04-26T22:02:19.915 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/time.h presence... yes 2024-04-26T22:02:19.915 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/time.h... yes 2024-04-26T22:02:19.927 INFO:tasks.workunit.client.1.smithi139.stdout:checking for unistd.h... (cached) yes 2024-04-26T22:02:19.960 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/vfs.h usability... yes 2024-04-26T22:02:19.975 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/vfs.h presence... yes 2024-04-26T22:02:19.975 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/vfs.h... yes 2024-04-26T22:02:20.024 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/limits.h usability... no 2024-04-26T22:02:20.032 INFO:tasks.workunit.client.0.smithi135.stdout:rm -f stamp-h1 2024-04-26T22:02:20.034 INFO:tasks.workunit.client.0.smithi135.stdout:touch config.h.in 2024-04-26T22:02:20.035 INFO:tasks.workunit.client.0.smithi135.stdout:cd . && /bin/sh ./config.status config.h 2024-04-26T22:02:20.051 INFO:tasks.workunit.client.1.smithi139.stdout:checking sys/limits.h presence... no 2024-04-26T22:02:20.051 INFO:tasks.workunit.client.1.smithi139.stdout:checking for sys/limits.h... no 2024-04-26T22:02:20.074 INFO:tasks.workunit.client.1.smithi139.stdout:checking for an ANSI C-conforming const... yes 2024-04-26T22:02:20.075 INFO:tasks.workunit.client.0.smithi135.stdout:config.status: creating config.h 2024-04-26T22:02:20.099 INFO:tasks.workunit.client.0.smithi135.stdout:make all-am 2024-04-26T22:02:20.100 INFO:tasks.workunit.client.1.smithi139.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-26T22:02:20.105 INFO:tasks.workunit.client.0.smithi135.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-26T22:02:20.105 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-26T22:02:20.155 INFO:tasks.workunit.client.1.smithi139.stdout:checking for system... yes 2024-04-26T22:02:20.208 INFO:tasks.workunit.client.1.smithi139.stdout:checking for gettimeofday... yes 2024-04-26T22:02:20.260 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-26T22:02:20.263 INFO:tasks.workunit.client.1.smithi139.stdout:checking for mkdir... yes 2024-04-26T22:02:20.263 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-26T22:02:20.314 INFO:tasks.workunit.client.1.smithi139.stdout:checking for strerror... yes 2024-04-26T22:02:20.346 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-26T22:02:20.348 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-26T22:02:20.364 INFO:tasks.workunit.client.1.smithi139.stdout:checking for open64... yes 2024-04-26T22:02:20.417 INFO:tasks.workunit.client.1.smithi139.stdout:checking for stat64... yes 2024-04-26T22:02:20.443 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-26T22:02:20.446 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-26T22:02:20.471 INFO:tasks.workunit.client.1.smithi139.stdout:checking for fseeko64... yes 2024-04-26T22:02:20.523 INFO:tasks.workunit.client.1.smithi139.stdout:checking for lrand48_r... yes 2024-04-26T22:02:20.533 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-26T22:02:20.536 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-26T22:02:20.573 INFO:tasks.workunit.client.1.smithi139.stdout:checking for srand48_r... yes 2024-04-26T22:02:20.594 INFO:tasks.workunit.client.1.smithi139.stdout:configure: creating ./config.status 2024-04-26T22:02:20.656 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-26T22:02:20.658 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-26T22:02:20.668 INFO:tasks.workunit.client.1.smithi139.stdout: /bin/sh ./config.status 2024-04-26T22:02:20.706 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-26T22:02:20.708 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: creating Makefile 2024-04-26T22:02:20.709 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-26T22:02:20.718 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: creating config.h 2024-04-26T22:02:20.739 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: executing depfiles commands 2024-04-26T22:02:20.749 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: executing default commands 2024-04-26T22:02:20.749 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-26T22:02:20.752 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-26T22:02:20.772 INFO:tasks.workunit.client.1.smithi139.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run autoheader 2024-04-26T22:02:20.798 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-26T22:02:20.802 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-26T22:02:20.913 INFO:tasks.workunit.client.1.smithi139.stdout:rm -f stamp-h1 2024-04-26T22:02:20.915 INFO:tasks.workunit.client.1.smithi139.stdout:touch config.h.in 2024-04-26T22:02:20.916 INFO:tasks.workunit.client.1.smithi139.stdout:cd . && /bin/sh ./config.status config.h 2024-04-26T22:02:20.950 INFO:tasks.workunit.client.1.smithi139.stdout:config.status: creating config.h 2024-04-26T22:02:20.972 INFO:tasks.workunit.client.1.smithi139.stdout:make all-am 2024-04-26T22:02:20.977 INFO:tasks.workunit.client.1.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb' 2024-04-26T22:02:20.977 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-26T22:02:21.002 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-26T22:02:21.005 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-26T22:02:21.129 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-26T22:02:21.131 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-26T22:02:21.133 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-26T22:02:21.135 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-26T22:02:21.220 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-26T22:02:21.222 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-26T22:02:21.276 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-26T22:02:21.279 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-26T22:02:21.300 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-26T22:02:21.302 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-26T22:02:21.344 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-26T22:02:21.346 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-26T22:02:21.385 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-26T22:02:21.387 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-26T22:02:21.413 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-26T22:02:21.416 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-26T22:02:21.496 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-26T22:02:21.498 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-26T22:02:21.516 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-26T22:02:21.518 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-26T22:02:21.543 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-26T22:02:21.545 INFO:tasks.workunit.client.0.smithi135.stderr:In file included from parser.h:22, 2024-04-26T22:02:21.545 INFO:tasks.workunit.client.0.smithi135.stderr: from parser.c:25: 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c: In function ‘init_filesys’: 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.546 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.547 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.548 INFO:tasks.workunit.client.0.smithi135.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.549 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.550 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:21.551 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.552 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.553 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:21.554 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.555 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: | ^ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:21.556 INFO:tasks.workunit.client.0.smithi135.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~ 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-26T22:02:21.557 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.558 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-26T22:02:21.585 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-26T22:02:21.587 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c: In function ‘get_optstr’: 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:200:14: note: source object declared here 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr: 200 | char temp[BUFSIZE]; 2024-04-26T22:02:21.589 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~ 2024-04-26T22:02:21.595 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c: In function ‘size64_convert’: 2024-04-26T22:02:21.596 INFO:tasks.workunit.client.0.smithi135.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-26T22:02:21.596 INFO:tasks.workunit.client.0.smithi135.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-26T22:02:21.596 INFO:tasks.workunit.client.0.smithi135.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:21.631 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-26T22:02:21.633 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-26T22:02:21.843 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-26T22:02:21.846 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-26T22:02:21.885 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-26T22:02:21.888 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-26T22:02:21.941 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-26T22:02:21.944 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-26T22:02:21.967 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-26T22:02:21.969 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-26T22:02:22.030 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-26T22:02:22.033 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-26T22:02:22.061 INFO:tasks.workunit.client.0.smithi135.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-26T22:02:22.065 INFO:tasks.workunit.client.0.smithi135.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-26T22:02:22.095 INFO:tasks.workunit.client.0.smithi135.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-26T22:02:22.096 INFO:tasks.workunit.client.0.smithi135.stderr:+ cd .. 2024-04-26T22:02:22.096 INFO:tasks.workunit.client.0.smithi135.stderr:+ mkdir tmp 2024-04-26T22:02:22.097 INFO:tasks.workunit.client.0.smithi135.stderr:+ cd tmp 2024-04-26T22:02:22.097 INFO:tasks.workunit.client.0.smithi135.stderr:+ for f in $mydir/*.ffsb 2024-04-26T22:02:22.097 INFO:tasks.workunit.client.0.smithi135.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-04-26T22:02:22.098 INFO:tasks.workunit.client.0.smithi135.stdout:FFSB version 6.0-RC2 started 2024-04-26T22:02:22.098 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.100 INFO:tasks.workunit.client.0.smithi135.stdout:benchmark time = 300 2024-04-26T22:02:22.100 INFO:tasks.workunit.client.0.smithi135.stdout:ThreadGroup 0 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout:================ 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: num_threads = 32 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: read_random = off 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: read_size = 0 (0B) 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: read_blocksize = 0 (0B) 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: read_skip = off 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: read_skipsize = 0 (0B) 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: write_random = on 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: write_size = 5242880 (5MB) 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: fsync_file = 0 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: write_blocksize = 4096 (4KB) 2024-04-26T22:02:22.101 INFO:tasks.workunit.client.0.smithi135.stdout: wait time = 0 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: op weights 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: read = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: readall = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: write = 1 (100.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: create = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: append = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: delete = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: metaop = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: createdir = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: stat = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: writeall = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: writeall_fsync = 0 (0.00%) 2024-04-26T22:02:22.102 INFO:tasks.workunit.client.0.smithi135.stdout: open_close = 0 (0.00%) 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: write_fsync = 0 (0.00%) 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: create_fsync = 0 (0.00%) 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: append_fsync = 0 (0.00%) 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout:FileSystem . 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout:========== 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: num_dirs = 0 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: starting files = 128 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: min file size = 104857600 (100MB) 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: max file size = 104857600 (100MB) 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: directio = off 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: alignedio = on 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: bufferedio = off 2024-04-26T22:02:22.103 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.104 INFO:tasks.workunit.client.0.smithi135.stdout: aging is off 2024-04-26T22:02:22.104 INFO:tasks.workunit.client.0.smithi135.stdout: current utilization = 0.00% 2024-04-26T22:02:22.104 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:22.104 INFO:tasks.workunit.client.0.smithi135.stdout:checking existing fs: . 2024-04-26T22:02:22.104 INFO:tasks.workunit.client.0.smithi135.stdout:recreating new fileset 2024-04-26T22:02:22.104 INFO:tasks.workunit.client.0.smithi135.stderr:opendir: No such file or directory 2024-04-26T22:02:22.107 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-26T22:02:22.110 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-26T22:02:22.177 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-26T22:02:22.180 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-26T22:02:22.251 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-26T22:02:22.253 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-26T22:02:22.360 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-26T22:02:22.363 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr:In file included from parser.h:22, 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr: from parser.c:25: 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c: In function ‘init_filesys’: 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.390 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.391 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.392 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.393 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.394 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.395 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.396 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.397 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: | ^ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: 36 | container_of(ptr, type, member) 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-26T22:02:22.398 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c: In function ‘get_optstr’: 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:200:14: note: source object declared here 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr: 200 | char temp[BUFSIZE]; 2024-04-26T22:02:22.438 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~ 2024-04-26T22:02:22.444 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c: In function ‘size64_convert’: 2024-04-26T22:02:22.444 INFO:tasks.workunit.client.1.smithi139.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-26T22:02:22.444 INFO:tasks.workunit.client.1.smithi139.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-26T22:02:22.445 INFO:tasks.workunit.client.1.smithi139.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-26T22:02:22.769 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-26T22:02:22.776 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-26T22:02:22.878 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-26T22:02:22.883 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-26T22:02:22.980 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-26T22:02:22.983 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-26T22:02:23.012 INFO:tasks.workunit.client.1.smithi139.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-26T22:02:23.015 INFO:tasks.workunit.client.1.smithi139.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-26T22:02:23.047 INFO:tasks.workunit.client.1.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb' 2024-04-26T22:02:23.048 INFO:tasks.workunit.client.1.smithi139.stderr:+ cd .. 2024-04-26T22:02:23.048 INFO:tasks.workunit.client.1.smithi139.stderr:+ mkdir tmp 2024-04-26T22:02:23.049 INFO:tasks.workunit.client.1.smithi139.stderr:+ cd tmp 2024-04-26T22:02:23.049 INFO:tasks.workunit.client.1.smithi139.stderr:+ for f in $mydir/*.ffsb 2024-04-26T22:02:23.049 INFO:tasks.workunit.client.1.smithi139.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/random_write.32.ffsb 2024-04-26T22:02:23.050 INFO:tasks.workunit.client.1.smithi139.stdout:FFSB version 6.0-RC2 started 2024-04-26T22:02:23.050 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout:benchmark time = 300 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout:ThreadGroup 0 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout:================ 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: num_threads = 32 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: read_random = off 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: read_size = 0 (0B) 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: read_blocksize = 0 (0B) 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: read_skip = off 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: read_skipsize = 0 (0B) 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.053 INFO:tasks.workunit.client.1.smithi139.stdout: write_random = on 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: write_size = 5242880 (5MB) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: fsync_file = 0 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: write_blocksize = 4096 (4KB) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: wait time = 0 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: op weights 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: read = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: readall = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: write = 1 (100.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: create = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: append = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: delete = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: metaop = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: createdir = 0 (0.00%) 2024-04-26T22:02:23.054 INFO:tasks.workunit.client.1.smithi139.stdout: stat = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: writeall = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: writeall_fsync = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: open_close = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: write_fsync = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: create_fsync = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: append_fsync = 0 (0.00%) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout:FileSystem . 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout:========== 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: num_dirs = 0 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: starting files = 128 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: min file size = 104857600 (100MB) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: max file size = 104857600 (100MB) 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: directio = off 2024-04-26T22:02:23.055 INFO:tasks.workunit.client.1.smithi139.stdout: alignedio = on 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout: bufferedio = off 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout: aging is off 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout: current utilization = 0.00% 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout:checking existing fs: . 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stdout:recreating new fileset 2024-04-26T22:02:23.056 INFO:tasks.workunit.client.1.smithi139.stderr:opendir: No such file or directory 2024-04-26T22:02:28.429 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:02:28.433 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:02:47.406 INFO:tasks.workunit.client.0.smithi135.stdout:fs setup took 25 secs 2024-04-26T22:02:50.540 INFO:tasks.workunit.client.0.smithi135.stdout:Syncing()...3 sec 2024-04-26T22:02:50.540 INFO:tasks.workunit.client.0.smithi135.stdout:Starting Actual Benchmark At: Fri Apr 26 22:02:50 2024 2024-04-26T22:02:50.540 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:02:51.756 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:52.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.005+0000 7f319cba4640 1 Processor -- start 2024-04-26T22:02:52.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.007+0000 7f319cba4640 1 -- start start 2024-04-26T22:02:52.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.009+0000 7f319cba4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f319815e8d0 0x7f319815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.009+0000 7f319cba4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f319815b7e0 0x7f319815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.009+0000 7f319cba4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31980ad6c0 con 0x7f319806ebd0 2024-04-26T22:02:52.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.009+0000 7f319cba4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f319806e740 con 0x7f319815e8d0 2024-04-26T22:02:52.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.009+0000 7f319cba4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f319806e3f0 con 0x7f319815b7e0 2024-04-26T22:02:52.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.011+0000 7f3196d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f319806ebd0 0x7f319806f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:45866/0 (socket says 172.21.15.135:45866) 2024-04-26T22:02:52.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.011+0000 7f3196d9d640 1 -- 172.21.15.135:0/1153238802 learned_addr learned my addr 172.21.15.135:0/1153238802 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:02:52.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.012+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905998643 0 0) 0x7f31980ad6c0 con 0x7f319806ebd0 2024-04-26T22:02:52.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.012+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3180003520 con 0x7f319806ebd0 2024-04-26T22:02:52.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.012+0000 7f319659c640 1 --2- 172.21.15.135:0/1153238802 >> v2:172.21.15.139:3300/0 conn(0x7f319815e8d0 0x7f319815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f3195d9b640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f319815b7e0 0x7f319815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2441639503 0 0) 0x7f3180003520 con 0x7f319806ebd0 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f319815b7e0 msgr2=0x7f319815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f319815b7e0 0x7f319815bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 >> v2:172.21.15.139:3300/0 conn(0x7f319815e8d0 msgr2=0x7f319815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 --2- 172.21.15.135:0/1153238802 >> v2:172.21.15.139:3300/0 conn(0x7f319815e8d0 0x7f319815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31980ac100 con 0x7f319806ebd0 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f3188001670 con 0x7f319806ebd0 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 >> v1:172.21.15.139:6789/0 conn(0x7f319806ebd0 legacy=0x7f319806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31800039b0 0x7f3180003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3180004800 0x7f3180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.013+0000 7f319559a640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31800051c0 0x7f318000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f3196d9d640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31800051c0 0x7f318000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f319806e740 con 0x7f31800051c0 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f319806e3f0 con 0x7f3180004800 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f318000c0b0 con 0x7f31800039b0 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f319659c640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31800039b0 0x7f3180003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f3196d9d640 1 -- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31800039b0 msgr2=0x7f3180003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f3196d9d640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31800039b0 0x7f3180003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f3196d9d640 1 -- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3180004800 msgr2=0x7f3180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f3196d9d640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3180004800 0x7f3180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.014+0000 7f3196d9d640 1 -- 172.21.15.135:0/1153238802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3180003520 con 0x7f31800051c0 2024-04-26T22:02:52.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.016+0000 7f3196d9d640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31800051c0 0x7f318000b9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f318800c580 tx=0x7f3180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6792f26e228a1048 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.016+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188007ca0 con 0x7f31800051c0 2024-04-26T22:02:52.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.016+0000 7f319559a640 1 -- 172.21.15.135:0/1153238802 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3188034070 con 0x7f31800051c0 2024-04-26T22:02:52.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.017+0000 7f319cba4640 1 -- 172.21.15.135:0/1153238802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31800051c0 msgr2=0x7f318000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.017+0000 7f319cba4640 1 --2- 172.21.15.135:0/1153238802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31800051c0 0x7f318000b9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f318800c580 tx=0x7f3180005830 comp rx=0 tx=0).stop 2024-04-26T22:02:52.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.017+0000 7f3196d9d640 1 -- 172.21.15.135:0/1153238802 reap_dead start 2024-04-26T22:02:52.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.020+0000 7f319cba4640 1 -- 172.21.15.135:0/1153238802 shutdown_connections 2024-04-26T22:02:52.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.020+0000 7f319cba4640 1 -- 172.21.15.135:0/1153238802 >> 172.21.15.135:0/1153238802 conn(0x7f31980c82d0 msgr2=0x7f31980c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.021+0000 7f319cba4640 1 -- 172.21.15.135:0/1153238802 shutdown_connections 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.022+0000 7f319cba4640 1 -- 172.21.15.135:0/1153238802 wait complete. 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.024+0000 7f319cba4640 1 Processor -- start 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.024+0000 7f319cba4640 1 -- start start 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.025+0000 7f319cba4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3198146470 0x7f3198157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.025+0000 7f319cba4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3198144170 0x7f3198146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.025+0000 7f319cba4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 0x7f3198152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.025+0000 7f319cba4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31980abd90 con 0x7f3198144170 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.025+0000 7f319cba4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f31980ac6c0 con 0x7f3198152800 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.025+0000 7f319cba4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f319806f730 con 0x7f3198146470 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.026+0000 7f319659c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3198146470 0x7f3198157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.026+0000 7f3196d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 0x7f3198152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.026+0000 7f3196d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 0x7f3198152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56680/0 (socket says 172.21.15.135:56680) 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.026+0000 7f3196d9d640 1 -- 172.21.15.135:0/41644426 learned_addr learned my addr 172.21.15.135:0/41644426 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.026+0000 7f3195d9b640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3198144170 0x7f3198146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.027+0000 7f3196d9d640 1 -- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3198146470 msgr2=0x7f3198157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.027+0000 7f3196d9d640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3198146470 0x7f3198157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.027+0000 7f3196d9d640 1 -- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3198144170 msgr2=0x7f3198146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.027+0000 7f3196d9d640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3198144170 0x7f3198146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.027+0000 7f3196d9d640 1 -- 172.21.15.135:0/41644426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3198072020 con 0x7f3198152800 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.027+0000 7f3195d9b640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3198144170 0x7f3198146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.028+0000 7f3196d9d640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 0x7f3198152c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f318800c580 tx=0x7f3188001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.030+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188007ca0 con 0x7f3198152800 2024-04-26T22:02:52.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.030+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3198072300 con 0x7f3198152800 2024-04-26T22:02:52.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.030+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3198152000 con 0x7f3198152800 2024-04-26T22:02:52.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.030+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318803e070 con 0x7f3198152800 2024-04-26T22:02:52.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.030+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188007680 con 0x7f3198152800 2024-04-26T22:02:52.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.033+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f318802f480 con 0x7f3198152800 2024-04-26T22:02:52.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.034+0000 7f3186ffd640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f318007f0d0 0x7f3180081590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.034+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f31880acbc0 con 0x7f3198152800 2024-04-26T22:02:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.036+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c005290 con 0x7f3198152800 2024-04-26T22:02:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.037+0000 7f319659c640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f318007f0d0 0x7f3180081590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.038+0000 7f319659c640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f318007f0d0 0x7f3180081590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3190009fc0 tx=0x7f319000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.047+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f318807abf0 con 0x7f3198152800 2024-04-26T22:02:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.555+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f317c005580 con 0x7f3198152800 2024-04-26T22:02:52.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.559+0000 7f3186ffd640 1 -- 172.21.15.135:0/41644426 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7f318807a4f0 con 0x7f3198152800 2024-04-26T22:02:52.563 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:02:52.563 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:02:52.563 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.564+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f318007f0d0 msgr2=0x7f3180081590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.564+0000 7f319cba4640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f318007f0d0 0x7f3180081590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3190009fc0 tx=0x7f319000b040 comp rx=0 tx=0).stop 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.564+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 msgr2=0x7f3198152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.564+0000 7f319cba4640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 0x7f3198152c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f318800c580 tx=0x7f3188001970 comp rx=0 tx=0).stop 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 shutdown_connections 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f318007f0d0 0x7f3180081590 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198152800 0x7f3198152c90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3198144170 0x7f3198146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 --2- 172.21.15.135:0/41644426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3198146470 0x7f3198157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 >> 172.21.15.135:0/41644426 conn(0x7f31980c82d0 msgr2=0x7f319815c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 shutdown_connections 2024-04-26T22:02:52.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.565+0000 7f319cba4640 1 -- 172.21.15.135:0/41644426 wait complete. 2024-04-26T22:02:52.597 INFO:tasks.mds_thrash.fs.[a]:kill mds.d (rank=0) 2024-04-26T22:02:52.598 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T22:02:52.598 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:02:52.626 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T22:02:52.626 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:52.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.825+0000 7f7ab359e640 1 Processor -- start 2024-04-26T22:02:52.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.825+0000 7f7ab359e640 1 -- start start 2024-04-26T22:02:52.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac04a9e0 0x7f7aac007bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab359e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7aac047240 0x7f7aac04adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab359e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aac04a250 con 0x7f7aac0db000 2024-04-26T22:02:52.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab259c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac04a9e0 0x7f7aac007bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab359e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7aac04ba30 con 0x7f7aac047240 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab359e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac04c5d0 con 0x7f7aac04a9e0 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab1d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7aac047240 0x7f7aac04adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab1d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7aac047240 0x7f7aac04adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58888/0 (socket says 172.21.15.135:58888) 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.827+0000 7f7ab1d9b640 1 -- 172.21.15.135:0/1721895954 learned_addr learned my addr 172.21.15.135:0/1721895954 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.828+0000 7f7ab1d9b640 1 -- 172.21.15.135:0/1721895954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac04a9e0 msgr2=0x7f7aac007bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.828+0000 7f7ab1d9b640 1 --2- 172.21.15.135:0/1721895954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac04a9e0 0x7f7aac007bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.828+0000 7f7ab1d9b640 1 -- 172.21.15.135:0/1721895954 >> v1:172.21.15.135:6789/0 conn(0x7f7aac0db000 legacy=0x7f7aac04bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:52.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.828+0000 7f7ab1d9b640 1 -- 172.21.15.135:0/1721895954 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0476d0 con 0x7f7aac047240 2024-04-26T22:02:52.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.829+0000 7f7ab1d9b640 1 --2- 172.21.15.135:0/1721895954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac047240 0x7f7aac04adc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009870 tx=0x7f7a9c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13367cf0efafc606 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.830+0000 7f7ab159a640 1 -- 172.21.15.135:0/1721895954 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c037070 con 0x7f7aac047240 2024-04-26T22:02:52.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.830+0000 7f7ab159a640 1 -- 172.21.15.135:0/1721895954 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9c032440 con 0x7f7aac047240 2024-04-26T22:02:52.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.830+0000 7f7ab159a640 1 -- 172.21.15.135:0/1721895954 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c032810 con 0x7f7aac047240 2024-04-26T22:02:52.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.831+0000 7f7ab359e640 1 -- 172.21.15.135:0/1721895954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac047240 msgr2=0x7f7aac04adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.831+0000 7f7ab359e640 1 --2- 172.21.15.135:0/1721895954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac047240 0x7f7aac04adc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009870 tx=0x7f7a9c02a250 comp rx=0 tx=0).stop 2024-04-26T22:02:52.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.832+0000 7f7ab359e640 1 -- 172.21.15.135:0/1721895954 shutdown_connections 2024-04-26T22:02:52.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.832+0000 7f7ab359e640 1 --2- 172.21.15.135:0/1721895954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac047240 0x7f7aac04adc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.832+0000 7f7ab359e640 1 --2- 172.21.15.135:0/1721895954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac04a9e0 0x7f7aac007bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.832+0000 7f7ab359e640 1 -- 172.21.15.135:0/1721895954 >> 172.21.15.135:0/1721895954 conn(0x7f7aac049340 msgr2=0x7f7aac049750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:52.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.832+0000 7f7ab359e640 1 -- 172.21.15.135:0/1721895954 shutdown_connections 2024-04-26T22:02:52.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.832+0000 7f7ab359e640 1 -- 172.21.15.135:0/1721895954 wait complete. 2024-04-26T22:02:52.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.834+0000 7f7ab359e640 1 Processor -- start 2024-04-26T22:02:52.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.834+0000 7f7ab359e640 1 -- start start 2024-04-26T22:02:52.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.835+0000 7f7ab359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 0x7f7aac0d69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.836+0000 7f7ab259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 0x7f7aac0d69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.836+0000 7f7ab259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 0x7f7aac0d69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58900/0 (socket says 172.21.15.135:58900) 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.836+0000 7f7ab259c640 1 -- 172.21.15.135:0/75030427 learned_addr learned my addr 172.21.15.135:0/75030427 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.836+0000 7f7ab359e640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0db000 0x7f7aac0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab359e640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac0d6f20 0x7f7aac0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab1d9b640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0db000 0x7f7aac0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab359e640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac0db490 con 0x7f7aac04a9e0 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab359e640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac04b6c0 con 0x7f7aac0db000 2024-04-26T22:02:52.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab359e640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac047c90 con 0x7f7aac0d6f20 2024-04-26T22:02:52.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab2d9d640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac0d6f20 0x7f7aac0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab259c640 1 -- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac0d6f20 msgr2=0x7f7aac0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab259c640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac0d6f20 0x7f7aac0cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab259c640 1 -- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0db000 msgr2=0x7f7aac0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:52.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab259c640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0db000 0x7f7aac0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:52.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.837+0000 7f7ab259c640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac032420 con 0x7f7aac04a9e0 2024-04-26T22:02:52.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.838+0000 7f7ab259c640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 0x7f7aac0d69e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400be70 tx=0x7f7aa400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.847+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa401d070 con 0x7f7aac04a9e0 2024-04-26T22:02:52.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.847+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4018640 con 0x7f7aac04a9e0 2024-04-26T22:02:52.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.847+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40189f0 con 0x7f7aac04a9e0 2024-04-26T22:02:52.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.847+0000 7f7ab359e640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac0d5210 con 0x7f7aac04a9e0 2024-04-26T22:02:52.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.847+0000 7f7ab359e640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac0d4f30 con 0x7f7aac04a9e0 2024-04-26T22:02:52.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.849+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac000bc0 con 0x7f7aac04a9e0 2024-04-26T22:02:52.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.851+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7aa4020070 con 0x7f7aac04a9e0 2024-04-26T22:02:52.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.852+0000 7f7aaaffd640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a9406c9c0 0x7f7a9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:52.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.852+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f7aa408cb10 con 0x7f7aac04a9e0 2024-04-26T22:02:52.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.854+0000 7f7ab1d9b640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a9406c9c0 0x7f7a9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:52.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.855+0000 7f7ab1d9b640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a9406c9c0 0x7f7a9406ee80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c034000 tx=0x7f7a9c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:52.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:52.858+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7aac000bc0 con 0x7f7aac04a9e0 2024-04-26T22:02:53.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.359+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aac0057e0 con 0x7f7aac04a9e0 2024-04-26T22:02:53.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.361+0000 7f7aaaffd640 1 -- 172.21.15.135:0/75030427 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7f7aa4027020 con 0x7f7aac04a9e0 2024-04-26T22:02:53.365 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:02:53.365 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:02:53.365 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:02:53.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.372+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a9406c9c0 msgr2=0x7f7a9406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:53.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.372+0000 7f7aa8ff9640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a9406c9c0 0x7f7a9406ee80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c034000 tx=0x7f7a9c02e000 comp rx=0 tx=0).stop 2024-04-26T22:02:53.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.372+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 msgr2=0x7f7aac0d69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:53.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.372+0000 7f7aa8ff9640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 0x7f7aac0d69e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400be70 tx=0x7f7aa400e730 comp rx=0 tx=0).stop 2024-04-26T22:02:53.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 shutdown_connections 2024-04-26T22:02:53.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7a9406c9c0 0x7f7a9406ee80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:53.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7aac0d6f20 0x7f7aac0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:53.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7aac0db000 0x7f7aac0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:53.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 --2- 172.21.15.135:0/75030427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aac04a9e0 0x7f7aac0d69e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:53.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 >> 172.21.15.135:0/75030427 conn(0x7f7aac049340 msgr2=0x7f7aac03f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:53.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 shutdown_connections 2024-04-26T22:02:53.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:53.373+0000 7f7aa8ff9640 1 -- 172.21.15.135:0/75030427 wait complete. 2024-04-26T22:02:53.405 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:02:55.406 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:55.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.603+0000 7fc117f9e640 1 Processor -- start 2024-04-26T22:02:55.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.605+0000 7fc117f9e640 1 -- start start 2024-04-26T22:02:55.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc117f9e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc11006d590 0x7fc1101539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc117f9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1100c5560 0x7fc1101574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc117f9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc110157a10 0x7fc110150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc117f9e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc110157e80 con 0x7fc11006d590 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc117f9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc110153ff0 con 0x7fc110157a10 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc117f9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc1101542d0 con 0x7fc1100c5560 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc115d3a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc11006d590 0x7fc1101539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc115d3a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc11006d590 0x7fc1101539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58906/0 (socket says 172.21.15.135:58906) 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc115d3a640 1 -- 172.21.15.135:0/1683175152 learned_addr learned my addr 172.21.15.135:0/1683175152 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.607+0000 7fc115539640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1100c5560 0x7fc1101574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:55.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.608+0000 7fc115d3a640 1 -- 172.21.15.135:0/1683175152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1100c5560 msgr2=0x7fc1101574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:55.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.608+0000 7fc115d3a640 1 --2- 172.21.15.135:0/1683175152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1100c5560 0x7fc1101574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.608+0000 7fc115d3a640 1 -- 172.21.15.135:0/1683175152 >> v2:172.21.15.139:3300/0 conn(0x7fc110157a10 msgr2=0x7fc110150cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:55.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.608+0000 7fc115d3a640 1 --2- 172.21.15.135:0/1683175152 >> v2:172.21.15.139:3300/0 conn(0x7fc110157a10 0x7fc110150cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.608+0000 7fc115d3a640 1 -- 172.21.15.135:0/1683175152 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1101545b0 con 0x7fc11006d590 2024-04-26T22:02:55.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.609+0000 7fc115d3a640 1 --2- 172.21.15.135:0/1683175152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc11006d590 0x7fc1101539a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc110069150 tx=0x7fc10c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc6cb760b97a50f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:55.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.610+0000 7fc114d38640 1 -- 172.21.15.135:0/1683175152 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c037070 con 0x7fc11006d590 2024-04-26T22:02:55.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.610+0000 7fc114d38640 1 -- 172.21.15.135:0/1683175152 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c032440 con 0x7fc11006d590 2024-04-26T22:02:55.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.610+0000 7fc114d38640 1 -- 172.21.15.135:0/1683175152 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0327f0 con 0x7fc11006d590 2024-04-26T22:02:55.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.611+0000 7fc117f9e640 1 -- 172.21.15.135:0/1683175152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc11006d590 msgr2=0x7fc1101539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:55.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.611+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1683175152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc11006d590 0x7fc1101539a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc110069150 tx=0x7fc10c02a180 comp rx=0 tx=0).stop 2024-04-26T22:02:55.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.614+0000 7fc117f9e640 1 -- 172.21.15.135:0/1683175152 shutdown_connections 2024-04-26T22:02:55.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.614+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1683175152 >> v2:172.21.15.139:3300/0 conn(0x7fc110157a10 0x7fc110150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.614+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1683175152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1100c5560 0x7fc1101574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.614+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1683175152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc11006d590 0x7fc1101539a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.614+0000 7fc117f9e640 1 -- 172.21.15.135:0/1683175152 >> 172.21.15.135:0/1683175152 conn(0x7fc1100c5ce0 msgr2=0x7fc1100c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:55.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.616+0000 7fc117f9e640 1 -- 172.21.15.135:0/1683175152 shutdown_connections 2024-04-26T22:02:55.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.617+0000 7fc117f9e640 1 -- 172.21.15.135:0/1683175152 wait complete. 2024-04-26T22:02:55.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.619+0000 7fc117f9e640 1 Processor -- start 2024-04-26T22:02:55.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.622+0000 7fc117f9e640 1 -- start start 2024-04-26T22:02:55.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.623+0000 7fc117f9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11006d590 0x7fc1100acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.623+0000 7fc117f9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 0x7fc11014d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.623+0000 7fc117f9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110157a10 0x7fc1101f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.623+0000 7fc117f9e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc110074b30 con 0x7fc1100c5560 2024-04-26T22:02:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.623+0000 7fc117f9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc110156fb0 con 0x7fc110157a10 2024-04-26T22:02:55.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.623+0000 7fc117f9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100c3c70 con 0x7fc11006d590 2024-04-26T22:02:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.625+0000 7fc115539640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 0x7fc11014d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.625+0000 7fc115539640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 0x7fc11014d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58922/0 (socket says 172.21.15.135:58922) 2024-04-26T22:02:55.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.625+0000 7fc115539640 1 -- 172.21.15.135:0/1082336255 learned_addr learned my addr 172.21.15.135:0/1082336255 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:55.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.626+0000 7fc115539640 1 -- 172.21.15.135:0/1082336255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11006d590 msgr2=0x7fc1100acaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:55.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.626+0000 7fc115539640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11006d590 0x7fc1100acaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.626+0000 7fc115539640 1 -- 172.21.15.135:0/1082336255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110157a10 msgr2=0x7fc1101f0a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.626+0000 7fc115539640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110157a10 0x7fc1101f0a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.626+0000 7fc115539640 1 -- 172.21.15.135:0/1082336255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110153960 con 0x7fc1100c5560 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.627+0000 7fc115539640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 0x7fc11014d490 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc10000bfd0 tx=0x7fc10000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.628+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10000de00 con 0x7fc1100c5560 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.628+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc100010440 con 0x7fc1100c5560 2024-04-26T22:02:55.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.628+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100010830 con 0x7fc1100c5560 2024-04-26T22:02:55.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.630+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1100ae750 con 0x7fc1100c5560 2024-04-26T22:02:55.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.630+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc110153c00 con 0x7fc1100c5560 2024-04-26T22:02:55.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.633+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1000109d0 con 0x7fc1100c5560 2024-04-26T22:02:55.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.633+0000 7fc1067fc640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc0f406c940 0x7fc0f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:55.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.634+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7fc10008c880 con 0x7fc1100c5560 2024-04-26T22:02:55.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.634+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1100696c0 con 0x7fc1100c5560 2024-04-26T22:02:55.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.637+0000 7fc115d3a640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc0f406c940 0x7fc0f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:55.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.646+0000 7fc115d3a640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc0f406c940 0x7fc0f406ee00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc10c034000 tx=0x7fc10c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:55.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:55.646+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc100014030 con 0x7fc1100c5560 2024-04-26T22:02:56.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.139+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1100507a0 con 0x7fc1100c5560 2024-04-26T22:02:56.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.142+0000 7fc1067fc640 1 -- 172.21.15.135:0/1082336255 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7fc10005a8b0 con 0x7fc1100c5560 2024-04-26T22:02:56.157 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:02:56.158 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:02:56.158 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:02:56.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc0f406c940 msgr2=0x7fc0f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:56.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc0f406c940 0x7fc0f406ee00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc10c034000 tx=0x7fc10c02e000 comp rx=0 tx=0).stop 2024-04-26T22:02:56.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 msgr2=0x7fc11014d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 0x7fc11014d490 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc10000bfd0 tx=0x7fc10000ee60 comp rx=0 tx=0).stop 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 shutdown_connections 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc0f406c940 0x7fc0f406ee00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110157a10 0x7fc1101f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1100c5560 0x7fc11014d490 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 --2- 172.21.15.135:0/1082336255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11006d590 0x7fc1100acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.158+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 >> 172.21.15.135:0/1082336255 conn(0x7fc1100c5ce0 msgr2=0x7fc1100c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:56.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.159+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 shutdown_connections 2024-04-26T22:02:56.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:56.161+0000 7fc117f9e640 1 -- 172.21.15.135:0/1082336255 wait complete. 2024-04-26T22:02:56.189 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:02:58.190 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:58.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.400+0000 7f0de186b640 1 Processor -- start 2024-04-26T22:02:58.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.400+0000 7f0de186b640 1 -- start start 2024-04-26T22:02:58.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0de186b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dd4047260 0x7f0dd40df1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:58.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0de186b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd4047020 con 0x7f0dd404baa0 2024-04-26T22:02:58.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0de186b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd4046d40 con 0x7f0dd403f2b0 2024-04-26T22:02:58.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0de186b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd40df700 con 0x7f0dd4047260 2024-04-26T22:02:58.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0ddb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dd4047260 0x7f0dd40df1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:58.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0ddb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dd4047260 0x7f0dd40df1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56732/0 (socket says 172.21.15.135:56732) 2024-04-26T22:02:58.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.401+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/3816683210 learned_addr learned my addr 172.21.15.135:0/3816683210 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:58.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.402+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/3816683210 >> v1:172.21.15.139:6789/0 conn(0x7f0dd403f2b0 legacy=0x7f0dd40deab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:02:58.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.402+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/3816683210 >> v1:172.21.15.135:6789/0 conn(0x7f0dd404baa0 legacy=0x7f0dd40e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:58.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.402+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/3816683210 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd404b040 con 0x7f0dd4047260 2024-04-26T22:02:58.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.403+0000 7f0ddb7fe640 1 --2- 172.21.15.135:0/3816683210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd40df1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc065e50 tx=0x7f0ddc0a71b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff4a8060eb76c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:58.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.404+0000 7f0dd9ffb640 1 -- 172.21.15.135:0/3816683210 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b4070 con 0x7f0dd4047260 2024-04-26T22:02:58.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.405+0000 7f0dd9ffb640 1 -- 172.21.15.135:0/3816683210 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0af440 con 0x7f0dd4047260 2024-04-26T22:02:58.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.406+0000 7f0dd9ffb640 1 -- 172.21.15.135:0/3816683210 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0af870 con 0x7f0dd4047260 2024-04-26T22:02:58.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.406+0000 7f0de186b640 1 -- 172.21.15.135:0/3816683210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 msgr2=0x7f0dd40df1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:58.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.406+0000 7f0de186b640 1 --2- 172.21.15.135:0/3816683210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd40df1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc065e50 tx=0x7f0ddc0a71b0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.407+0000 7f0de186b640 1 -- 172.21.15.135:0/3816683210 shutdown_connections 2024-04-26T22:02:58.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.407+0000 7f0de186b640 1 --2- 172.21.15.135:0/3816683210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd40df1c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.407+0000 7f0de186b640 1 -- 172.21.15.135:0/3816683210 >> 172.21.15.135:0/3816683210 conn(0x7f0dd4048af0 msgr2=0x7f0dd4048f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:58.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.413+0000 7f0de186b640 1 -- 172.21.15.135:0/3816683210 shutdown_connections 2024-04-26T22:02:58.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.413+0000 7f0de186b640 1 -- 172.21.15.135:0/3816683210 wait complete. 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.415+0000 7f0de186b640 1 Processor -- start 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.415+0000 7f0de186b640 1 -- start start 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.416+0000 7f0de186b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0dd403f2b0 0x7f0dd40e2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.416+0000 7f0de186b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd402e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.416+0000 7f0de186b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 0x7f0dd402e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0de186b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd40469f0 con 0x7f0dd403f2b0 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0de186b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd404b600 con 0x7f0dd4047260 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0de186b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd403f690 con 0x7f0dd404baa0 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 0x7f0dd402e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 0x7f0dd402e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:39800/0 (socket says 172.21.15.135:39800) 2024-04-26T22:02:58.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/589148895 learned_addr learned my addr 172.21.15.135:0/589148895 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0dda7fc640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd402e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddaffd640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0dd403f2b0 0x7f0dd40e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 msgr2=0x7f0dd402e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd402e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0dd403f2b0 msgr2=0x7f0dd40e2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0dd403f2b0 0x7f0dd40e2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.417+0000 7f0ddb7fe640 1 -- 172.21.15.135:0/589148895 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd40dc7e0 con 0x7f0dd404baa0 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.418+0000 7f0ddaffd640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0dd403f2b0 0x7f0dd40e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:02:58.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.418+0000 7f0ddb7fe640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 0x7f0dd402e6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc06acf0 tx=0x7f0ddc0661d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:58.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.419+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b4040 con 0x7f0dd404baa0 2024-04-26T22:02:58.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.419+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0b9950 con 0x7f0dd404baa0 2024-04-26T22:02:58.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.419+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd4180b40 con 0x7f0dd404baa0 2024-04-26T22:02:58.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.419+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd4181100 con 0x7f0dd404baa0 2024-04-26T22:02:58.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.420+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b9d00 con 0x7f0dd404baa0 2024-04-26T22:02:58.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.421+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc005290 con 0x7f0dd404baa0 2024-04-26T22:02:58.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.422+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ddc0b7070 con 0x7f0dd404baa0 2024-04-26T22:02:58.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.423+0000 7f0dc3fff640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0dc806c8f0 0x7f0dc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:58.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.423+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f0ddc1294f0 con 0x7f0dd404baa0 2024-04-26T22:02:58.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.423+0000 7f0ddaffd640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0dc806c8f0 0x7f0dc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:58.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.424+0000 7f0ddaffd640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0dc806c8f0 0x7f0dc806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0006fd0 tx=0x7f0dd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:58.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.432+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0dd40dc7e0 con 0x7f0dd404baa0 2024-04-26T22:02:58.532 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:02:58.535 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:02:58.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.942+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dbc005580 con 0x7f0dd404baa0 2024-04-26T22:02:58.944 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.944+0000 7f0dc3fff640 1 -- 172.21.15.135:0/589148895 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7f0ddc0bd020 con 0x7f0dd404baa0 2024-04-26T22:02:58.945 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:02:58.945 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:02:58.945 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0dc806c8f0 msgr2=0x7f0dc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0dc806c8f0 0x7f0dc806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0006fd0 tx=0x7f0dd0008040 comp rx=0 tx=0).stop 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 msgr2=0x7f0dd402e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 0x7f0dd402e6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc06acf0 tx=0x7f0ddc0661d0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 shutdown_connections 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0dc806c8f0 0x7f0dc806edb0 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0006fd0 tx=0x7f0dd0008040 comp rx=0 tx=0).stop 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dd404baa0 0x7f0dd402e6d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd4047260 0x7f0dd402e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 --2- 172.21.15.135:0/589148895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0dd403f2b0 0x7f0dd40e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:58.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 >> 172.21.15.135:0/589148895 conn(0x7f0dd4048af0 msgr2=0x7f0dd40dcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:58.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.950+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 shutdown_connections 2024-04-26T22:02:58.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:02:58.951+0000 7f0de186b640 1 -- 172.21.15.135:0/589148895 wait complete. 2024-04-26T22:02:58.971 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:03:00.972 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:01.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.181+0000 7f6822f39640 1 Processor -- start 2024-04-26T22:03:01.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.182+0000 7f6822f39640 1 -- start start 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6822f39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 0x7f681c153570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6822f39640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f681c0c3c70 0x7f681c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6822f39640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f681c0ad3a0 con 0x7f681c06d530 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6822f39640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f681c0ab9d0 con 0x7f681c0c3c70 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6822f39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f681c0ad680 con 0x7f681c14cae0 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6820cd5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 0x7f681c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6820cd5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 0x7f681c153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:44792/0 (socket says 172.21.15.135:44792) 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f6820cd5640 1 -- 172.21.15.135:0/441832521 learned_addr learned my addr 172.21.15.135:0/441832521 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:01.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.183+0000 7f68214d6640 1 --2- 172.21.15.135:0/441832521 >> v2:172.21.15.135:3300/0 conn(0x7f681c0c3c70 0x7f681c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.184+0000 7f6820cd5640 1 -- 172.21.15.135:0/441832521 >> v2:172.21.15.135:3300/0 conn(0x7f681c0c3c70 msgr2=0x7f681c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.184+0000 7f6820cd5640 1 --2- 172.21.15.135:0/441832521 >> v2:172.21.15.135:3300/0 conn(0x7f681c0c3c70 0x7f681c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.184+0000 7f6820cd5640 1 -- 172.21.15.135:0/441832521 >> v1:172.21.15.139:6789/0 conn(0x7f681c06d530 legacy=0x7f681c0abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.184+0000 7f6820cd5640 1 -- 172.21.15.135:0/441832521 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c0ab380 con 0x7f681c14cae0 2024-04-26T22:03:01.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.185+0000 7f6820cd5640 1 --2- 172.21.15.135:0/441832521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 0x7f681c153570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6810008870 tx=0x7f681002c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e651dcbe7a210f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.186+0000 7f681b7fe640 1 -- 172.21.15.135:0/441832521 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681003a070 con 0x7f681c14cae0 2024-04-26T22:03:01.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.187+0000 7f681b7fe640 1 -- 172.21.15.135:0/441832521 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6810032480 con 0x7f681c14cae0 2024-04-26T22:03:01.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.188+0000 7f6822f39640 1 -- 172.21.15.135:0/441832521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 msgr2=0x7f681c153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.188+0000 7f6822f39640 1 --2- 172.21.15.135:0/441832521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 0x7f681c153570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6810008870 tx=0x7f681002c6f0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.188+0000 7f6822f39640 1 -- 172.21.15.135:0/441832521 shutdown_connections 2024-04-26T22:03:01.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.188+0000 7f6822f39640 1 --2- 172.21.15.135:0/441832521 >> v2:172.21.15.135:3300/0 conn(0x7f681c0c3c70 0x7f681c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.188+0000 7f6822f39640 1 --2- 172.21.15.135:0/441832521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c14cae0 0x7f681c153570 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6810008870 tx=0x7f681002c6f0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.189 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.188+0000 7f6822f39640 1 -- 172.21.15.135:0/441832521 >> 172.21.15.135:0/441832521 conn(0x7f681c0c4d60 msgr2=0x7f681c0c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.189+0000 7f6822f39640 1 -- 172.21.15.135:0/441832521 shutdown_connections 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.189+0000 7f6822f39640 1 -- 172.21.15.135:0/441832521 wait complete. 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.191+0000 7f6822f39640 1 Processor -- start 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.191+0000 7f6822f39640 1 -- start start 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.192+0000 7f6822f39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06d530 0x7f681c072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.192+0000 7f6822f39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c0c3c70 0x7f681c1f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.192+0000 7f6822f39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c071d70 0x7f681c149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.192+0000 7f6822f39640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f681c14cec0 con 0x7f681c0c3c70 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.192+0000 7f6822f39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f681c06d9b0 con 0x7f681c06d530 2024-04-26T22:03:01.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.192+0000 7f6822f39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f681c0aca60 con 0x7f681c071d70 2024-04-26T22:03:01.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.194+0000 7f6820cd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06d530 0x7f681c072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.194+0000 7f6820cd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06d530 0x7f681c072720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:44122/0 (socket says 172.21.15.135:44122) 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.194+0000 7f6820cd5640 1 -- 172.21.15.135:0/3978893893 learned_addr learned my addr 172.21.15.135:0/3978893893 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.195+0000 7f681bfff640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c0c3c70 0x7f681c1f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.195+0000 7f68214d6640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c071d70 0x7f681c149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.196+0000 7f68214d6640 1 -- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06d530 msgr2=0x7f681c072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.196+0000 7f68214d6640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06d530 0x7f681c072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.196+0000 7f68214d6640 1 -- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c0c3c70 msgr2=0x7f681c1f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.196+0000 7f68214d6640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c0c3c70 0x7f681c1f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.196+0000 7f68214d6640 1 -- 172.21.15.135:0/3978893893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c073050 con 0x7f681c071d70 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.196+0000 7f68214d6640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c071d70 0x7f681c149520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d580 tx=0x7f680c00da40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.197+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c018020 con 0x7f681c071d70 2024-04-26T22:03:01.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.197+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680c014620 con 0x7f681c071d70 2024-04-26T22:03:01.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.199+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c018020 con 0x7f681c071d70 2024-04-26T22:03:01.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.200+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c073330 con 0x7f681c071d70 2024-04-26T22:03:01.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.201+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c072220 con 0x7f681c071d70 2024-04-26T22:03:01.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.202+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681c0506c0 con 0x7f681c071d70 2024-04-26T22:03:01.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.204+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f680c01b020 con 0x7f681c071d70 2024-04-26T22:03:01.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.204+0000 7f68197fa640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680806ccf0 0x7f680806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.207+0000 7f6820cd5640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680806ccf0 0x7f680806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.208+0000 7f6820cd5640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680806ccf0 0x7f680806f1b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f681002c730 tx=0x7f681000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.208+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f680c00f040 con 0x7f681c071d70 2024-04-26T22:03:01.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.220+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f680c022020 con 0x7f681c071d70 2024-04-26T22:03:01.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.696+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f681c0ac640 con 0x7f681c071d70 2024-04-26T22:03:01.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.698+0000 7f68197fa640 1 -- 172.21.15.135:0/3978893893 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7f680c022020 con 0x7f681c071d70 2024-04-26T22:03:01.702 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:01.702 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:01.703 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:03:01.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680806ccf0 msgr2=0x7f680806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680806ccf0 0x7f680806f1b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f681002c730 tx=0x7f681000b040 comp rx=0 tx=0).stop 2024-04-26T22:03:01.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c071d70 msgr2=0x7f681c149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c071d70 0x7f681c149520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d580 tx=0x7f680c00da40 comp rx=0 tx=0).stop 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 shutdown_connections 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680806ccf0 0x7f680806f1b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c071d70 0x7f681c149520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c0c3c70 0x7f681c1f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 --2- 172.21.15.135:0/3978893893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06d530 0x7f681c072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.704+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 >> 172.21.15.135:0/3978893893 conn(0x7f681c0c4d60 msgr2=0x7f681c0c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.705+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 shutdown_connections 2024-04-26T22:03:01.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:01.705+0000 7f6822f39640 1 -- 172.21.15.135:0/3978893893 wait complete. 2024-04-26T22:03:01.731 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:03:03.732 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:03.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.919+0000 7f2a4a338640 1 Processor -- start 2024-04-26T22:03:03.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.919+0000 7f2a4a338640 1 -- start start 2024-04-26T22:03:03.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a4a338640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a44070120 0x7f2a440aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a4a338640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a440aef70 0x7f2a440c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a4a338640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a44154e10 con 0x7f2a4406d590 2024-04-26T22:03:03.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a4a338640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2a440af400 con 0x7f2a44070120 2024-04-26T22:03:03.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a4a338640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a441559e0 con 0x7f2a440aef70 2024-04-26T22:03:03.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a43fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2a4406d590 0x7f2a44158d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57168/0 (socket says 172.21.15.135:57168) 2024-04-26T22:03:03.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a43fff640 1 -- 172.21.15.135:0/1169789754 learned_addr learned my addr 172.21.15.135:0/1169789754 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:03.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a488d5640 1 --2- 172.21.15.135:0/1169789754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a440aef70 0x7f2a440c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a488d5640 1 -- 172.21.15.135:0/1169789754 >> v2:172.21.15.139:3300/0 conn(0x7f2a44070120 msgr2=0x7f2a440aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:03.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a488d5640 1 --2- 172.21.15.135:0/1169789754 >> v2:172.21.15.139:3300/0 conn(0x7f2a44070120 0x7f2a440aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a488d5640 1 -- 172.21.15.135:0/1169789754 >> v1:172.21.15.139:6789/0 conn(0x7f2a4406d590 legacy=0x7f2a44158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.921+0000 7f2a488d5640 1 -- 172.21.15.135:0/1169789754 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441550b0 con 0x7f2a440aef70 2024-04-26T22:03:03.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.922+0000 7f2a488d5640 1 --2- 172.21.15.135:0/1169789754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a440aef70 0x7f2a440c3be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a380097a0 tx=0x7f2a3802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe611c95b9b6dc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.923+0000 7f2a42ffd640 1 -- 172.21.15.135:0/1169789754 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38037070 con 0x7f2a440aef70 2024-04-26T22:03:03.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.923+0000 7f2a42ffd640 1 -- 172.21.15.135:0/1169789754 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3802ae40 con 0x7f2a440aef70 2024-04-26T22:03:03.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.923+0000 7f2a42ffd640 1 -- 172.21.15.135:0/1169789754 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38032660 con 0x7f2a440aef70 2024-04-26T22:03:03.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.924+0000 7f2a4a338640 1 -- 172.21.15.135:0/1169789754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a440aef70 msgr2=0x7f2a440c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.924+0000 7f2a4a338640 1 --2- 172.21.15.135:0/1169789754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a440aef70 0x7f2a440c3be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a380097a0 tx=0x7f2a3802a0f0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.924+0000 7f2a4a338640 1 -- 172.21.15.135:0/1169789754 shutdown_connections 2024-04-26T22:03:03.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.924+0000 7f2a4a338640 1 --2- 172.21.15.135:0/1169789754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a440aef70 0x7f2a440c3be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.924+0000 7f2a4a338640 1 --2- 172.21.15.135:0/1169789754 >> v2:172.21.15.139:3300/0 conn(0x7f2a44070120 0x7f2a440aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.924+0000 7f2a4a338640 1 -- 172.21.15.135:0/1169789754 >> 172.21.15.135:0/1169789754 conn(0x7f2a440c5ce0 msgr2=0x7f2a440c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.926+0000 7f2a4a338640 1 -- 172.21.15.135:0/1169789754 shutdown_connections 2024-04-26T22:03:03.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.926+0000 7f2a4a338640 1 -- 172.21.15.135:0/1169789754 wait complete. 2024-04-26T22:03:03.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.928+0000 7f2a4a338640 1 Processor -- start 2024-04-26T22:03:03.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.929+0000 7f2a4a338640 1 -- start start 2024-04-26T22:03:03.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.929+0000 7f2a4a338640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a4406d590 0x7f2a44157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.929+0000 7f2a4a338640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a44070120 0x7f2a44157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.929+0000 7f2a4a338640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 0x7f2a44146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a4a338640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4406d970 con 0x7f2a440aef70 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a4a338640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44155670 con 0x7f2a4406d590 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a4a338640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4406daf0 con 0x7f2a44070120 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 0x7f2a44146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 0x7f2a44146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47684/0 (socket says 172.21.15.135:47684) 2024-04-26T22:03:03.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 -- 172.21.15.135:0/2515070067 learned_addr learned my addr 172.21.15.135:0/2515070067 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a43fff640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a4406d590 0x7f2a44157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a437fe640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a44070120 0x7f2a44157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 -- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a44070120 msgr2=0x7f2a44157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a44070120 0x7f2a44157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 -- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a4406d590 msgr2=0x7f2a44157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a4406d590 0x7f2a44157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a488d5640 1 -- 172.21.15.135:0/2515070067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a440be290 con 0x7f2a440aef70 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.930+0000 7f2a43fff640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a4406d590 0x7f2a44157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:03.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.931+0000 7f2a488d5640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 0x7f2a44146c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2a38035040 tx=0x7f2a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.931+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38042020 con 0x7f2a440aef70 2024-04-26T22:03:03.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.931+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3803d690 con 0x7f2a440aef70 2024-04-26T22:03:03.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.931+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38037040 con 0x7f2a440aef70 2024-04-26T22:03:03.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.931+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a440be810 con 0x7f2a440aef70 2024-04-26T22:03:03.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.931+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a440be530 con 0x7f2a440aef70 2024-04-26T22:03:03.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.934+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a38045020 con 0x7f2a440aef70 2024-04-26T22:03:03.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.935+0000 7f2a40ff9640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2a3006c9c0 0x7f2a3006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.935+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a24005290 con 0x7f2a440aef70 2024-04-26T22:03:03.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.935+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f2a380ac660 con 0x7f2a440aef70 2024-04-26T22:03:03.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.936+0000 7f2a43fff640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2a3006c9c0 0x7f2a3006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.936+0000 7f2a43fff640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2a3006c9c0 0x7f2a3006ee80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005c00 tx=0x7f2a3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:03.946+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a3804c030 con 0x7f2a440aef70 2024-04-26T22:03:04.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.426+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a24005580 con 0x7f2a440aef70 2024-04-26T22:03:04.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.427+0000 7f2a40ff9640 1 -- 172.21.15.135:0/2515070067 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7f2a3807a160 con 0x7f2a440aef70 2024-04-26T22:03:04.429 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:04.429 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:04.429 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:03:04.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2a3006c9c0 msgr2=0x7f2a3006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2a3006c9c0 0x7f2a3006ee80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005c00 tx=0x7f2a3c009040 comp rx=0 tx=0).stop 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 msgr2=0x7f2a44146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 0x7f2a44146c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2a38035040 tx=0x7f2a380046c0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 shutdown_connections 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2a3006c9c0 0x7f2a3006ee80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a440aef70 0x7f2a44146c10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a44070120 0x7f2a44157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 --2- 172.21.15.135:0/2515070067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a4406d590 0x7f2a44157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.434+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 >> 172.21.15.135:0/2515070067 conn(0x7f2a440c5ce0 msgr2=0x7f2a440c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.438+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 shutdown_connections 2024-04-26T22:03:04.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:04.438+0000 7f2a4a338640 1 -- 172.21.15.135:0/2515070067 wait complete. 2024-04-26T22:03:04.467 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:03:06.468 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:06.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.671+0000 7f467eebd640 1 Processor -- start 2024-04-26T22:03:06.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.671+0000 7f467eebd640 1 -- start start 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.672+0000 7f467eebd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780abfc0 0x7f467815e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.672+0000 7f467eebd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f467815edd0 con 0x7f46780c3750 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.672+0000 7f467eebd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f467815f0b0 con 0x7f467815e170 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.672+0000 7f467eebd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f467815eba0 con 0x7f46780abfc0 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f467cc59640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780abfc0 0x7f467815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f467d45a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f467815e170 0x7f467806eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57178/0 (socket says 172.21.15.135:57178) 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f467d45a640 1 -- 172.21.15.135:0/3681637266 learned_addr learned my addr 172.21.15.135:0/3681637266 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f46777fe640 1 -- 172.21.15.135:0/3681637266 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819610795 0 0) 0x7f467815f0b0 con 0x7f467815e170 2024-04-26T22:03:06.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f46777fe640 1 -- 172.21.15.135:0/3681637266 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4664003520 con 0x7f467815e170 2024-04-26T22:03:06.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f467cc59640 1 -- 172.21.15.135:0/3681637266 >> v1:172.21.15.139:6789/0 conn(0x7f467815e170 legacy=0x7f467806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f467cc59640 1 -- 172.21.15.135:0/3681637266 >> v1:172.21.15.135:6789/0 conn(0x7f46780c3750 legacy=0x7f467815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.673+0000 7f467cc59640 1 -- 172.21.15.135:0/3681637266 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467815f390 con 0x7f46780abfc0 2024-04-26T22:03:06.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.676+0000 7f467cc59640 1 --2- 172.21.15.135:0/3681637266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780abfc0 0x7f467815e660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46600097d0 tx=0x7f466002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf79d000aeb22cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.676+0000 7f46777fe640 1 -- 172.21.15.135:0/3681637266 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660037070 con 0x7f46780abfc0 2024-04-26T22:03:06.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.676+0000 7f46777fe640 1 -- 172.21.15.135:0/3681637266 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4660032440 con 0x7f46780abfc0 2024-04-26T22:03:06.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.676+0000 7f46777fe640 1 -- 172.21.15.135:0/3681637266 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600327f0 con 0x7f46780abfc0 2024-04-26T22:03:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.678+0000 7f467eebd640 1 -- 172.21.15.135:0/3681637266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780abfc0 msgr2=0x7f467815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.678+0000 7f467eebd640 1 --2- 172.21.15.135:0/3681637266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780abfc0 0x7f467815e660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46600097d0 tx=0x7f466002a1c0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.679+0000 7f467eebd640 1 -- 172.21.15.135:0/3681637266 shutdown_connections 2024-04-26T22:03:06.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.679+0000 7f467eebd640 1 --2- 172.21.15.135:0/3681637266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780abfc0 0x7f467815e660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.679+0000 7f467eebd640 1 -- 172.21.15.135:0/3681637266 >> 172.21.15.135:0/3681637266 conn(0x7f467806c310 msgr2=0x7f467806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:06.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.679+0000 7f467eebd640 1 -- 172.21.15.135:0/3681637266 shutdown_connections 2024-04-26T22:03:06.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.679+0000 7f467eebd640 1 -- 172.21.15.135:0/3681637266 wait complete. 2024-04-26T22:03:06.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.681+0000 7f467eebd640 1 Processor -- start 2024-04-26T22:03:06.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.681+0000 7f467eebd640 1 -- start start 2024-04-26T22:03:06.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467eebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46780abfc0 0x7f46781f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467eebd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780c3750 0x7f4678073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467eebd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 0x7f46781e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467eebd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f467815c560 con 0x7f467815e170 2024-04-26T22:03:06.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467eebd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f467806e3d0 con 0x7f46780abfc0 2024-04-26T22:03:06.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467eebd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f46780ac3a0 con 0x7f46780c3750 2024-04-26T22:03:06.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467d45a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 0x7f46781e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467cc59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46780abfc0 0x7f46781f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467d45a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 0x7f46781e4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47700/0 (socket says 172.21.15.135:47700) 2024-04-26T22:03:06.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f467d45a640 1 -- 172.21.15.135:0/492724362 learned_addr learned my addr 172.21.15.135:0/492724362 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:06.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.682+0000 7f4677fff640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780c3750 0x7f4678073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.683+0000 7f467d45a640 1 -- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780c3750 msgr2=0x7f4678073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.683+0000 7f467d45a640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780c3750 0x7f4678073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.683+0000 7f467d45a640 1 -- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46780abfc0 msgr2=0x7f46781f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.683+0000 7f467d45a640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46780abfc0 0x7f46781f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.683+0000 7f467d45a640 1 -- 172.21.15.135:0/492724362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781e7fd0 con 0x7f467815e170 2024-04-26T22:03:06.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.683+0000 7f467d45a640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 0x7f46781e4de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f467000c410 tx=0x7f467000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.684+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700074e0 con 0x7f467815e170 2024-04-26T22:03:06.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.684+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46700047e0 con 0x7f467815e170 2024-04-26T22:03:06.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.684+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781e82b0 con 0x7f467815e170 2024-04-26T22:03:06.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.684+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781e78c0 con 0x7f467815e170 2024-04-26T22:03:06.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.687+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670004b90 con 0x7f467815e170 2024-04-26T22:03:06.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.687+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4670011070 con 0x7f467815e170 2024-04-26T22:03:06.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.687+0000 7f466effd640 1 -- 172.21.15.135:0/492724362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c005290 con 0x7f467815e170 2024-04-26T22:03:06.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.688+0000 7f46757fa640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f465806c9c0 0x7f465806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.689+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6838+0+0 (secure 0 0 0) 0x7f467008bad0 con 0x7f467815e170 2024-04-26T22:03:06.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.698+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4670059c80 con 0x7f467815e170 2024-04-26T22:03:06.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.701+0000 7f467cc59640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f465806c9c0 0x7f465806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:06.704+0000 7f467cc59640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f465806c9c0 0x7f465806ee80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4660034000 tx=0x7f466002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.166+0000 7f466effd640 1 -- 172.21.15.135:0/492724362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f465c005580 con 0x7f467815e170 2024-04-26T22:03:07.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.169+0000 7f46757fa640 1 -- 172.21.15.135:0/492724362 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6503 (secure 0 0 0) 0x7f4670059580 con 0x7f467815e170 2024-04-26T22:03:07.169 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:07.169 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:01:30.775227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":28,"addr":"172.21.15.139:6835/2608264407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2608264407},{"type":"v1","addr":"172.21.15.139:6835","nonce":2608264407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:07.169 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 23 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.172+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f465806c9c0 msgr2=0x7f465806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.172+0000 7f467eebd640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f465806c9c0 0x7f465806ee80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4660034000 tx=0x7f466002e000 comp rx=0 tx=0).stop 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.172+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 msgr2=0x7f46781e4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.172+0000 7f467eebd640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 0x7f46781e4de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f467000c410 tx=0x7f467000c8d0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 shutdown_connections 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f465806c9c0 0x7f465806ee80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f467815e170 0x7f46781e4de0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46780c3750 0x7f4678073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 --2- 172.21.15.135:0/492724362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46780abfc0 0x7f46781f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 >> 172.21.15.135:0/492724362 conn(0x7f467806c310 msgr2=0x7f46780c5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.173+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 shutdown_connections 2024-04-26T22:03:07.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:07.174+0000 7f467eebd640 1 -- 172.21.15.135:0/492724362 wait complete. 2024-04-26T22:03:07.199 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:03:09.199 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:09.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.429+0000 7f5c75664640 1 Processor -- start 2024-04-26T22:03:09.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.429+0000 7f5c75664640 1 -- start start 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c75664640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5c7013ffb0 0x7f5c700c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c75664640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c70158180 0x7f5c701585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c75664640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c700c8a30 con 0x7f5c7015a280 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c75664640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5c70072be0 con 0x7f5c7013ffb0 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c75664640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c700c9640 con 0x7f5c70158180 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c6e7fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5c7015a280 0x7f5c70072ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39502/0 (socket says 172.21.15.135:39502) 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.430+0000 7f5c6e7fc640 1 -- 172.21.15.135:0/1487933888 learned_addr learned my addr 172.21.15.135:0/1487933888 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6f7fe640 1 --2- 172.21.15.135:0/1487933888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c70158180 0x7f5c701585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6effd640 1 --2- 172.21.15.135:0/1487933888 >> v2:172.21.15.135:3300/0 conn(0x7f5c7013ffb0 0x7f5c700c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:09.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6dffb640 1 -- 172.21.15.135:0/1487933888 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3156533636 0 0) 0x7f5c700c8a30 con 0x7f5c7015a280 2024-04-26T22:03:09.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6dffb640 1 -- 172.21.15.135:0/1487933888 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c58003520 con 0x7f5c7015a280 2024-04-26T22:03:09.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6f7fe640 1 -- 172.21.15.135:0/1487933888 >> v2:172.21.15.135:3300/0 conn(0x7f5c7013ffb0 msgr2=0x7f5c700c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:09.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6f7fe640 1 --2- 172.21.15.135:0/1487933888 >> v2:172.21.15.135:3300/0 conn(0x7f5c7013ffb0 0x7f5c700c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6f7fe640 1 -- 172.21.15.135:0/1487933888 >> v1:172.21.15.135:6789/0 conn(0x7f5c7015a280 legacy=0x7f5c70072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:09.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.431+0000 7f5c6f7fe640 1 -- 172.21.15.135:0/1487933888 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c700c8d10 con 0x7f5c70158180 2024-04-26T22:03:09.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.432+0000 7f5c6f7fe640 1 --2- 172.21.15.135:0/1487933888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c70158180 0x7f5c701585d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009d90 tx=0x7f5c6802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3934db9b15a889c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:09.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.433+0000 7f5c6dffb640 1 -- 172.21.15.135:0/1487933888 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68031600 con 0x7f5c70158180 2024-04-26T22:03:09.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.433+0000 7f5c6dffb640 1 -- 172.21.15.135:0/1487933888 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c680317a0 con 0x7f5c70158180 2024-04-26T22:03:09.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.433+0000 7f5c6dffb640 1 -- 172.21.15.135:0/1487933888 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68031600 con 0x7f5c70158180 2024-04-26T22:03:09.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.434+0000 7f5c75664640 1 -- 172.21.15.135:0/1487933888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c70158180 msgr2=0x7f5c701585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:09.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.434+0000 7f5c75664640 1 --2- 172.21.15.135:0/1487933888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c70158180 0x7f5c701585d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009d90 tx=0x7f5c6802a5d0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.435+0000 7f5c75664640 1 -- 172.21.15.135:0/1487933888 shutdown_connections 2024-04-26T22:03:09.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.435+0000 7f5c75664640 1 --2- 172.21.15.135:0/1487933888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c70158180 0x7f5c701585d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.435+0000 7f5c75664640 1 --2- 172.21.15.135:0/1487933888 >> v2:172.21.15.135:3300/0 conn(0x7f5c7013ffb0 0x7f5c700c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.435+0000 7f5c75664640 1 -- 172.21.15.135:0/1487933888 >> 172.21.15.135:0/1487933888 conn(0x7f5c7006dc10 msgr2=0x7f5c7006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:09.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.435+0000 7f5c75664640 1 -- 172.21.15.135:0/1487933888 shutdown_connections 2024-04-26T22:03:09.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.435+0000 7f5c75664640 1 -- 172.21.15.135:0/1487933888 wait complete. 2024-04-26T22:03:09.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.438+0000 7f5c75664640 1 Processor -- start 2024-04-26T22:03:09.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.438+0000 7f5c75664640 1 -- start start 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.438+0000 7f5c75664640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c7013ffb0 0x7f5c70155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.438+0000 7f5c75664640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 0x7f5c700acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c75664640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c7015a280 0x7f5c701efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 0x7f5c700acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 0x7f5c700acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55006/0 (socket says 172.21.15.135:55006) 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6e7fc640 1 -- 172.21.15.135:0/2646424307 learned_addr learned my addr 172.21.15.135:0/2646424307 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700c92d0 con 0x7f5c70158180 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700c87e0 con 0x7f5c7015a280 2024-04-26T22:03:09.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c70073700 con 0x7f5c7013ffb0 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6effd640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c7013ffb0 0x7f5c70155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6f7fe640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c7015a280 0x7f5c701efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6effd640 1 -- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c7015a280 msgr2=0x7f5c701efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6effd640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c7015a280 0x7f5c701efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6effd640 1 -- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 msgr2=0x7f5c700acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.439+0000 7f5c6effd640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 0x7f5c700acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.440+0000 7f5c6effd640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c701f0710 con 0x7f5c7013ffb0 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.440+0000 7f5c6f7fe640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c7015a280 0x7f5c701efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.440+0000 7f5c6e7fc640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 0x7f5c700acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:09.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.440+0000 7f5c6effd640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c7013ffb0 0x7f5c70155570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000e8a0 tx=0x7f5c6000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:09.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.441+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60017070 con 0x7f5c7013ffb0 2024-04-26T22:03:09.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.441+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6000c940 con 0x7f5c7013ffb0 2024-04-26T22:03:09.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.441+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60009e60 con 0x7f5c7013ffb0 2024-04-26T22:03:09.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.441+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c70137160 con 0x7f5c7013ffb0 2024-04-26T22:03:09.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.442+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c700ad450 con 0x7f5c7013ffb0 2024-04-26T22:03:09.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.443+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c70136f50 con 0x7f5c7013ffb0 2024-04-26T22:03:09.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.444+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c6001a070 con 0x7f5c7013ffb0 2024-04-26T22:03:09.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.445+0000 7f5c577fe640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c5c06c9c0 0x7f5c5c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:09.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.445+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7f5c6008c090 con 0x7f5c7013ffb0 2024-04-26T22:03:09.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.446+0000 7f5c6e7fc640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c5c06c9c0 0x7f5c5c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:09.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.447+0000 7f5c6e7fc640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c5c06c9c0 0x7f5c5c06ee80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b13b0 tx=0x7f5c64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:09.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.454+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c60059a20 con 0x7f5c7013ffb0 2024-04-26T22:03:09.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.958+0000 7f5c75664640 1 -- 172.21.15.135:0/2646424307 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c70076650 con 0x7f5c7013ffb0 2024-04-26T22:03:09.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.959+0000 7f5c577fe640 1 -- 172.21.15.135:0/2646424307 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5667 (secure 0 0 0) 0x7f5c6005dd20 con 0x7f5c7013ffb0 2024-04-26T22:03:09.961 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:09.961 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:07.785630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:09.961 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 24 2024-04-26T22:03:09.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 -- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c5c06c9c0 msgr2=0x7f5c5c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c5c06c9c0 0x7f5c5c06ee80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b13b0 tx=0x7f5c64009040 comp rx=0 tx=0).stop 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 -- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c7013ffb0 msgr2=0x7f5c70155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c7013ffb0 0x7f5c70155570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000e8a0 tx=0x7f5c6000ed60 comp rx=0 tx=0).stop 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 -- 172.21.15.135:0/2646424307 shutdown_connections 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c5c06c9c0 0x7f5c5c06ee80 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b13b0 tx=0x7f5c64009040 comp rx=0 tx=0).stop 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c7015a280 0x7f5c701efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c70158180 0x7f5c700acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 --2- 172.21.15.135:0/2646424307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c7013ffb0 0x7f5c70155570 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 -- 172.21.15.135:0/2646424307 >> 172.21.15.135:0/2646424307 conn(0x7f5c7006dc10 msgr2=0x7f5c7013dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:09.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 -- 172.21.15.135:0/2646424307 shutdown_connections 2024-04-26T22:03:09.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:09.966+0000 7f5c557fa640 1 -- 172.21.15.135:0/2646424307 wait complete. 2024-04-26T22:03:10.006 INFO:tasks.mds_thrash.fs.[a]:mds.d down, removed from mdsmap 2024-04-26T22:03:10.006 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-04-26T22:03:10.006 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:10.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.195+0000 7fad35221640 1 Processor -- start 2024-04-26T22:03:10.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.195+0000 7fad35221640 1 -- start start 2024-04-26T22:03:10.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.196+0000 7fad35221640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad300c69b0 0x7fad3015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:10.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.196+0000 7fad35221640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad300c50d0 con 0x7fad30154ed0 2024-04-26T22:03:10.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.196+0000 7fad35221640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad300c53b0 con 0x7fad3015c490 2024-04-26T22:03:10.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.196+0000 7fad35221640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fad300c5ce0 con 0x7fad300c69b0 2024-04-26T22:03:10.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.196+0000 7fad2f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad300c69b0 0x7fad3015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:10.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.197+0000 7fad2f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad300c69b0 0x7fad3015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55012/0 (socket says 172.21.15.135:55012) 2024-04-26T22:03:10.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.197+0000 7fad2f7fe640 1 -- 172.21.15.135:0/4061354872 learned_addr learned my addr 172.21.15.135:0/4061354872 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:10.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.198+0000 7fad2f7fe640 1 -- 172.21.15.135:0/4061354872 >> v1:172.21.15.139:6789/0 conn(0x7fad3015c490 legacy=0x7fad3015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:10.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.198+0000 7fad2f7fe640 1 -- 172.21.15.135:0/4061354872 >> v1:172.21.15.135:6789/0 conn(0x7fad30154ed0 legacy=0x7fad30074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:10.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.198+0000 7fad2f7fe640 1 -- 172.21.15.135:0/4061354872 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad300c5690 con 0x7fad300c69b0 2024-04-26T22:03:10.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.199+0000 7fad2f7fe640 1 --2- 172.21.15.135:0/4061354872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad300c69b0 0x7fad3015afa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fad24009870 tx=0x7fad2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d308fd429a35e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:10.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.200+0000 7fad2effd640 1 -- 172.21.15.135:0/4061354872 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24037070 con 0x7fad300c69b0 2024-04-26T22:03:10.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.200+0000 7fad2effd640 1 -- 172.21.15.135:0/4061354872 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad24032440 con 0x7fad300c69b0 2024-04-26T22:03:10.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.200+0000 7fad2effd640 1 -- 172.21.15.135:0/4061354872 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24032810 con 0x7fad300c69b0 2024-04-26T22:03:10.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.201+0000 7fad35221640 1 -- 172.21.15.135:0/4061354872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad300c69b0 msgr2=0x7fad3015afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:10.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.201+0000 7fad35221640 1 --2- 172.21.15.135:0/4061354872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad300c69b0 0x7fad3015afa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fad24009870 tx=0x7fad2402a210 comp rx=0 tx=0).stop 2024-04-26T22:03:10.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.202+0000 7fad35221640 1 -- 172.21.15.135:0/4061354872 shutdown_connections 2024-04-26T22:03:10.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.203+0000 7fad35221640 1 --2- 172.21.15.135:0/4061354872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad300c69b0 0x7fad3015afa0 secure :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fad24009870 tx=0x7fad2402a210 comp rx=0 tx=0).stop 2024-04-26T22:03:10.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.203+0000 7fad35221640 1 -- 172.21.15.135:0/4061354872 >> 172.21.15.135:0/4061354872 conn(0x7fad30073610 msgr2=0x7fad30073a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:10.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.205+0000 7fad35221640 1 -- 172.21.15.135:0/4061354872 shutdown_connections 2024-04-26T22:03:10.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.206+0000 7fad35221640 1 -- 172.21.15.135:0/4061354872 wait complete. 2024-04-26T22:03:10.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.208+0000 7fad35221640 1 Processor -- start 2024-04-26T22:03:10.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.208+0000 7fad35221640 1 -- start start 2024-04-26T22:03:10.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.209+0000 7fad35221640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad30154ed0 0x7fad3014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:10.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.209+0000 7fad35221640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad3015c490 0x7fad30074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:10.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.209+0000 7fad35221640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3014d6a0 0x7fad3014db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.209+0000 7fad35221640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad300c4cf0 con 0x7fad30154ed0 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.209+0000 7fad35221640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad300c6170 con 0x7fad3014d6a0 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.209+0000 7fad35221640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad300c4e70 con 0x7fad3015c490 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad30154ed0 0x7fad3014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad30154ed0 0x7fad3014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55018/0 (socket says 172.21.15.135:55018) 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2ffff640 1 -- 172.21.15.135:0/1678512603 learned_addr learned my addr 172.21.15.135:0/1678512603 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:10.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2f7fe640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad3015c490 0x7fad30074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:10.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2f7fe640 1 -- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3014d6a0 msgr2=0x7fad3014db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:10.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2f7fe640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3014d6a0 0x7fad3014db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.210+0000 7fad2f7fe640 1 -- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad30154ed0 msgr2=0x7fad3014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:10.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.211+0000 7fad2f7fe640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad30154ed0 0x7fad3014e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.211+0000 7fad2f7fe640 1 -- 172.21.15.135:0/1678512603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad300c9ef0 con 0x7fad3015c490 2024-04-26T22:03:10.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.211+0000 7fad2f7fe640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad3015c490 0x7fad30074170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad24031000 tx=0x7fad240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:10.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.211+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24041070 con 0x7fad3015c490 2024-04-26T22:03:10.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.212+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2403c8b0 con 0x7fad3015c490 2024-04-26T22:03:10.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.212+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24037040 con 0x7fad3015c490 2024-04-26T22:03:10.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.214+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad300c9c10 con 0x7fad3015c490 2024-04-26T22:03:10.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.214+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3015ba80 con 0x7fad3015c490 2024-04-26T22:03:10.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.216+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad300c8e80 con 0x7fad3015c490 2024-04-26T22:03:10.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.218+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad24044070 con 0x7fad3015c490 2024-04-26T22:03:10.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.219+0000 7fad2cff9640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad1c06c8f0 0x7fad1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:10.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.219+0000 7fad2ffff640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad1c06c8f0 0x7fad1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:10.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.221+0000 7fad2ffff640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad1c06c8f0 0x7fad1c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad3014f860 tx=0x7fad20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:10.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.221+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7fad240ace50 con 0x7fad3015c490 2024-04-26T22:03:10.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.226+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad300c8e80 con 0x7fad3015c490 2024-04-26T22:03:10.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.685+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad300506c0 con 0x7fad3015c490 2024-04-26T22:03:10.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.686+0000 7fad2cff9640 1 -- 172.21.15.135:0/1678512603 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5667 (secure 0 0 0) 0x7fad2407a7f0 con 0x7fad3015c490 2024-04-26T22:03:10.686 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:10.687 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:07.785630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:10.687 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 24 2024-04-26T22:03:10.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.690+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad1c06c8f0 msgr2=0x7fad1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:10.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.690+0000 7fad35221640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad1c06c8f0 0x7fad1c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad3014f860 tx=0x7fad20009040 comp rx=0 tx=0).stop 2024-04-26T22:03:10.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.690+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad3015c490 msgr2=0x7fad30074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:10.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.690+0000 7fad35221640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad3015c490 0x7fad30074170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad24031000 tx=0x7fad240046c0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 shutdown_connections 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad1c06c8f0 0x7fad1c06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3014d6a0 0x7fad3014db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad3015c490 0x7fad30074170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 --2- 172.21.15.135:0/1678512603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad30154ed0 0x7fad3014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 >> 172.21.15.135:0/1678512603 conn(0x7fad30073610 msgr2=0x7fad3015e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 shutdown_connections 2024-04-26T22:03:10.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:10.691+0000 7fad35221640 1 -- 172.21.15.135:0/1678512603 wait complete. 2024-04-26T22:03:10.714 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 24, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 24, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5268": { "addr": "172.21.15.135:6835/2910412510", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 2910412510, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 2910412510, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5268, "incarnation": 24, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:03:07.785630+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5268 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4241": { "addr": "172.21.15.139:6833/3183513686", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3183513686, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3183513686, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 18, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 30 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:01:40.794279+0000", "qdb_cluster": [ 4241 ], "qdb_leader": 4241, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4241 } } } ], "standbys": [ { "addr": "172.21.15.135:6837/3652200818", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 3652200818, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 3652200818, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 23, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:03:12.715 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:12.892 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:12.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.918+0000 7f6b2fbb4640 1 Processor -- start 2024-04-26T22:03:12.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.919+0000 7f6b2fbb4640 1 -- start start 2024-04-26T22:03:12.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.920+0000 7f6b2fbb4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b2815a280 0x7f6b280c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.920+0000 7f6b2fbb4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b280c8a30 con 0x7f6b2813ffb0 2024-04-26T22:03:12.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.920+0000 7f6b2fbb4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b280c8d10 con 0x7f6b28158180 2024-04-26T22:03:12.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.920+0000 7f6b2fbb4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6b28072be0 con 0x7f6b2815a280 2024-04-26T22:03:12.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.923+0000 7f6b2e151640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6b28158180 0x7f6b28161610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54906/0 (socket says 172.21.15.135:54906) 2024-04-26T22:03:12.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.923+0000 7f6b2e151640 1 -- 172.21.15.135:0/2168599219 learned_addr learned my addr 172.21.15.135:0/2168599219 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:12.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.923+0000 7f6b2d950640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f6b2813ffb0 0x7f6b28072ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39508/0 (socket says 172.21.15.135:39508) 2024-04-26T22:03:12.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.923+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356916039 0 0) 0x7f6b280c8a30 con 0x7f6b2813ffb0 2024-04-26T22:03:12.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.923+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b14003520 con 0x7f6b2813ffb0 2024-04-26T22:03:12.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452834352 0 0) 0x7f6b280c8d10 con 0x7f6b28158180 2024-04-26T22:03:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b280c8a30 con 0x7f6b28158180 2024-04-26T22:03:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.923+0000 7f6b2d14f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b2815a280 0x7f6b280c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3595212798 0 0) 0x7f6b14003520 con 0x7f6b2813ffb0 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 >> v2:172.21.15.139:3300/0 conn(0x7f6b2815a280 msgr2=0x7f6b280c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 --2- 172.21.15.135:0/2168599219 >> v2:172.21.15.139:3300/0 conn(0x7f6b2815a280 0x7f6b280c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 >> v1:172.21.15.139:6789/0 conn(0x7f6b28158180 legacy=0x7f6b28161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b280c8ff0 con 0x7f6b2813ffb0 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f6b24001670 con 0x7f6b2813ffb0 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 >> v1:172.21.15.135:6789/0 conn(0x7f6b2813ffb0 legacy=0x7f6b28072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b14003b40 0x7f6b14003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14004800 0x7f6b14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b14005210 0x7f6b1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.924+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1400c110 con 0x7f6b14004800 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.925+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1400c2c0 con 0x7f6b14005210 2024-04-26T22:03:12.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.925+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1400c4c0 con 0x7f6b14003b40 2024-04-26T22:03:12.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.925+0000 7f6b2e151640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b14005210 0x7f6b1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.925+0000 7f6b2d14f640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14004800 0x7f6b14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.926+0000 7f6b2d950640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b14003b40 0x7f6b14003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.926+0000 7f6b2d14f640 1 -- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b14003b40 msgr2=0x7f6b14003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:12.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.926+0000 7f6b2d14f640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b14003b40 0x7f6b14003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:12.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.926+0000 7f6b2d14f640 1 -- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b14005210 msgr2=0x7f6b1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.926+0000 7f6b2d14f640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b14005210 0x7f6b1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.926+0000 7f6b2d14f640 1 -- 172.21.15.135:0/2168599219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b280c8ff0 con 0x7f6b14004800 2024-04-26T22:03:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.928+0000 7f6b2d14f640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14004800 0x7f6b14004c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ba10 tx=0x7f6b14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb04ee317553f204 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.928+0000 7f6b2e151640 1 -- 172.21.15.135:0/2168599219 reap_dead start 2024-04-26T22:03:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.928+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18030070 con 0x7f6b14004800 2024-04-26T22:03:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.929+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1802ae10 con 0x7f6b14004800 2024-04-26T22:03:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.929+0000 7f6b2c94e640 1 -- 172.21.15.135:0/2168599219 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800b5e0 con 0x7f6b14004800 2024-04-26T22:03:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.930+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2168599219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14004800 msgr2=0x7f6b14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.931+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14004800 0x7f6b14004c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ba10 tx=0x7f6b14005830 comp rx=0 tx=0).stop 2024-04-26T22:03:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.931+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2168599219 shutdown_connections 2024-04-26T22:03:12.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.931+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2168599219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14004800 0x7f6b14004c80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:12.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.931+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2168599219 >> 172.21.15.135:0/2168599219 conn(0x7f6b2806dc10 msgr2=0x7f6b2806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:12.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.931+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2168599219 shutdown_connections 2024-04-26T22:03:12.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.933+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2168599219 wait complete. 2024-04-26T22:03:12.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.934+0000 7f6b2fbb4640 1 Processor -- start 2024-04-26T22:03:12.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.935+0000 7f6b2fbb4640 1 -- start start 2024-04-26T22:03:12.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2fbb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28157000 0x7f6b28070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2fbb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b281573e0 0x7f6b28070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2fbb4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 0x7f6b28139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2fbb4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28140390 con 0x7f6b2813a900 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2fbb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b280c9640 con 0x7f6b28157000 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2fbb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6b28140510 con 0x7f6b281573e0 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2e151640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 0x7f6b28139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2e151640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 0x7f6b28139cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55044/0 (socket says 172.21.15.135:55044) 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2e151640 1 -- 172.21.15.135:0/2733175200 learned_addr learned my addr 172.21.15.135:0/2733175200 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:12.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.936+0000 7f6b2d14f640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b281573e0 0x7f6b28070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.937+0000 7f6b2e151640 1 -- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b281573e0 msgr2=0x7f6b28070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:12.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.937+0000 7f6b2e151640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b281573e0 0x7f6b28070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:12.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.937+0000 7f6b2e151640 1 -- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28157000 msgr2=0x7f6b28070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:12.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.937+0000 7f6b2e151640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28157000 0x7f6b28070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:12.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.937+0000 7f6b2e151640 1 -- 172.21.15.135:0/2733175200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2806f310 con 0x7f6b2813a900 2024-04-26T22:03:12.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.937+0000 7f6b2e151640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 0x7f6b28139cb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b2815b3a0 tx=0x7f6b2000f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.938+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000fd30 con 0x7f6b2813a900 2024-04-26T22:03:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.938+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20018070 con 0x7f6b2813a900 2024-04-26T22:03:12.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.939+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20013630 con 0x7f6b2813a900 2024-04-26T22:03:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.940+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b28154300 con 0x7f6b2813a900 2024-04-26T22:03:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.941+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b28071240 con 0x7f6b2813a900 2024-04-26T22:03:12.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.942+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b28076650 con 0x7f6b2813a900 2024-04-26T22:03:12.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.950+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b200137d0 con 0x7f6b2813a900 2024-04-26T22:03:12.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.951+0000 7f6b1e7fc640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b0c06c9c0 0x7f6b0c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:12.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.952+0000 7f6b2d950640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b0c06c9c0 0x7f6b0c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:12.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.953+0000 7f6b2d950640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b0c06c9c0 0x7f6b0c06ee80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b24001eb0 tx=0x7f6b24000f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:12.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.953+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7f6b2008cce0 con 0x7f6b2813a900 2024-04-26T22:03:12.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:12.955+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b2005acc0 con 0x7f6b2813a900 2024-04-26T22:03:13.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.091+0000 7ff5c15af640 1 Processor -- start 2024-04-26T22:03:13.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.091+0000 7ff5c15af640 1 -- start start 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5c15af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5bc0ab380 0x7ff5bc152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5c15af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5bc0c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5c15af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0c3e00 con 0x7ff5bc06f990 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5c15af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc06e3d0 con 0x7ff5bc0ab380 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5c15af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc06e6b0 con 0x7ff5bc150270 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5baffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5bc0ab380 0x7ff5bc152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5ba7fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff5bc06f990 0x7ff5bc0c3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39512/0 (socket says 172.21.15.135:39512) 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.092+0000 7ff5ba7fc640 1 -- 172.21.15.135:0/3591250597 learned_addr learned my addr 172.21.15.135:0/3591250597 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:13.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5bb7fe640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5bc0c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437438876 0 0) 0x7ff5bc0c3e00 con 0x7ff5bc06f990 2024-04-26T22:03:13.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a8003520 con 0x7ff5bc06f990 2024-04-26T22:03:13.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4023781307 0 0) 0x7ff5a8003520 con 0x7ff5bc06f990 2024-04-26T22:03:13.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 msgr2=0x7ff5bc0c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:13.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5b9ffb640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5bc0c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.093+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 >> v2:172.21.15.135:3300/0 conn(0x7ff5bc0ab380 msgr2=0x7ff5bc152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:13.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.094+0000 7ff5b9ffb640 1 --2- 172.21.15.135:0/3591250597 >> v2:172.21.15.135:3300/0 conn(0x7ff5bc0ab380 0x7ff5bc152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.094+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc0c40e0 con 0x7ff5bc06f990 2024-04-26T22:03:13.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.094+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7ff5ac0019f0 con 0x7ff5bc06f990 2024-04-26T22:03:13.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.094+0000 7ff5baffd640 1 --2- 172.21.15.135:0/3591250597 >> v2:172.21.15.135:3300/0 conn(0x7ff5bc0ab380 0x7ff5bc152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:13.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.095+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 >> v1:172.21.15.135:6789/0 conn(0x7ff5bc06f990 legacy=0x7ff5bc0c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.097+0000 7ff5b9ffb640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5a8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5b9ffb640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a8003fc0 0x7ff5a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5ba7fc640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a8003fc0 0x7ff5a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5baffd640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5a8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5b9ffb640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5a8004d40 0x7ff5a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc06e3d0 con 0x7ff5a8003fc0 2024-04-26T22:03:13.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc06e6b0 con 0x7ff5a8004d40 2024-04-26T22:03:13.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.098+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a800bb30 con 0x7ff5bc150270 2024-04-26T22:03:13.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.100+0000 7ff5ba7fc640 1 -- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 msgr2=0x7ff5a8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:13.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.100+0000 7ff5ba7fc640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5a8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.100+0000 7ff5ba7fc640 1 -- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5a8004d40 msgr2=0x7ff5a80051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:13.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.100+0000 7ff5ba7fc640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5a8004d40 0x7ff5a80051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.100+0000 7ff5ba7fc640 1 -- 172.21.15.135:0/3591250597 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a8003520 con 0x7ff5a8003fc0 2024-04-26T22:03:13.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.101+0000 7ff5ba7fc640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a8003fc0 0x7ff5a8004800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00d190 tx=0x7ff5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae61d9e27ae1f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:13.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.102+0000 7ff5baffd640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc150270 0x7ff5a8003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:13.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.102+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac005660 con 0x7ff5a8003fc0 2024-04-26T22:03:13.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.102+0000 7ff5b9ffb640 1 -- 172.21.15.135:0/3591250597 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac008330 con 0x7ff5a8003fc0 2024-04-26T22:03:13.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.103+0000 7ff5c15af640 1 -- 172.21.15.135:0/3591250597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a8003fc0 msgr2=0x7ff5a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.103+0000 7ff5c15af640 1 --2- 172.21.15.135:0/3591250597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5a8003fc0 0x7ff5a8004800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00d190 tx=0x7ff5a8005830 comp rx=0 tx=0).stop 2024-04-26T22:03:13.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.108+0000 7ff5bb7fe640 1 -- 172.21.15.135:0/3591250597 reap_dead start 2024-04-26T22:03:13.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.108+0000 7ff5c15af640 1 -- 172.21.15.135:0/3591250597 shutdown_connections 2024-04-26T22:03:13.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.108+0000 7ff5c15af640 1 -- 172.21.15.135:0/3591250597 >> 172.21.15.135:0/3591250597 conn(0x7ff5bc06c310 msgr2=0x7ff5bc06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:13.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.109+0000 7ff5c15af640 1 -- 172.21.15.135:0/3591250597 shutdown_connections 2024-04-26T22:03:13.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.111+0000 7ff5c15af640 1 -- 172.21.15.135:0/3591250597 wait complete. 2024-04-26T22:03:13.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.113+0000 7ff5c15af640 1 Processor -- start 2024-04-26T22:03:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.114+0000 7ff5c15af640 1 -- start start 2024-04-26T22:03:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.115+0000 7ff5c15af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc0c7850 0x7ff5bc151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.115+0000 7ff5c15af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 0x7ff5bc1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.115+0000 7ff5c15af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 0x7ff5bc0abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5ba7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 0x7ff5bc1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5ba7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 0x7ff5bc1515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55060/0 (socket says 172.21.15.135:55060) 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5c15af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc0c4c90 con 0x7ff5bc140470 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5c15af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc150700 con 0x7ff5bc0ab760 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5c15af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0c4e10 con 0x7ff5bc0c7850 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 0x7ff5bc0abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 0x7ff5bc0abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40618/0 (socket says 172.21.15.135:40618) 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 -- 172.21.15.135:0/252612561 learned_addr learned my addr 172.21.15.135:0/252612561 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 -- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc0c7850 msgr2=0x7ff5bc151080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:13.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc0c7850 0x7ff5bc151080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 -- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 msgr2=0x7ff5bc1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.116+0000 7ff5bb7fe640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 0x7ff5bc1515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.117+0000 7ff5bb7fe640 1 -- 172.21.15.135:0/252612561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc152a70 con 0x7ff5bc0ab760 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.117+0000 7ff5ba7fc640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 0x7ff5bc1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.117+0000 7ff5bb7fe640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 0x7ff5bc0abc10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc070520 tx=0x7ff5b400d760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.117+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40125f0 con 0x7ff5bc0ab760 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.118+0000 7ff5c15af640 1 -- 172.21.15.135:0/252612561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc164a60 con 0x7ff5bc0ab760 2024-04-26T22:03:13.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.118+0000 7ff5c15af640 1 -- 172.21.15.135:0/252612561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc141ca0 con 0x7ff5bc0ab760 2024-04-26T22:03:13.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.119+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a0005290 con 0x7ff5bc0ab760 2024-04-26T22:03:13.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.119+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b4019070 con 0x7ff5bc0ab760 2024-04-26T22:03:13.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.119+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40129b0 con 0x7ff5bc0ab760 2024-04-26T22:03:13.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.120+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5b4012b50 con 0x7ff5bc0ab760 2024-04-26T22:03:13.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.121+0000 7ff5b37fe640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff59c06c920 0x7ff59c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.121+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7ff5b408bcc0 con 0x7ff5bc0ab760 2024-04-26T22:03:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.121+0000 7ff5baffd640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff59c06c920 0x7ff59c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.122+0000 7ff5baffd640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff59c06c920 0x7ff59c06ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc074ef0 tx=0x7ff5a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:13.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.130+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5b4059c20 con 0x7ff5bc0ab760 2024-04-26T22:03:13.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.479+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b280507a0 con 0x7f6b2813a900 2024-04-26T22:03:13.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.482+0000 7f6b1e7fc640 1 -- 172.21.15.135:0/2733175200 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5671 (secure 0 0 0) 0x7f6b2005a5c0 con 0x7f6b2813a900 2024-04-26T22:03:13.483 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:13.483 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:12.563780+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:13.483 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 25 2024-04-26T22:03:13.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.486+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b0c06c9c0 msgr2=0x7f6b0c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.486+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b0c06c9c0 0x7f6b0c06ee80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b24001eb0 tx=0x7f6b24000f40 comp rx=0 tx=0).stop 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.486+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 msgr2=0x7f6b28139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.486+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 0x7f6b28139cb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b2815b3a0 tx=0x7f6b2000f5a0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 shutdown_connections 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b0c06c9c0 0x7f6b0c06ee80 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b24001eb0 tx=0x7f6b24000f40 comp rx=0 tx=0).stop 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2813a900 0x7f6b28139cb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b281573e0 0x7f6b28070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 --2- 172.21.15.135:0/2733175200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b28157000 0x7f6b28070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 >> 172.21.15.135:0/2733175200 conn(0x7f6b2806dc10 msgr2=0x7f6b281551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 shutdown_connections 2024-04-26T22:03:13.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.487+0000 7f6b2fbb4640 1 -- 172.21.15.135:0/2733175200 wait complete. 2024-04-26T22:03:13.506 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:03:13.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.649+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5a0005580 con 0x7ff5bc0ab760 2024-04-26T22:03:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.651+0000 7ff5b37fe640 1 -- 172.21.15.135:0/252612561 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5671 (secure 0 0 0) 0x7ff5b4059520 con 0x7ff5bc0ab760 2024-04-26T22:03:13.651 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:13.651 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:12.563780+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:13.651 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 25 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff59c06c920 msgr2=0x7ff59c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff59c06c920 0x7ff59c06ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc074ef0 tx=0x7ff5a4008040 comp rx=0 tx=0).stop 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 msgr2=0x7ff5bc0abc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 0x7ff5bc0abc10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc070520 tx=0x7ff5b400d760 comp rx=0 tx=0).stop 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 shutdown_connections 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff59c06c920 0x7ff59c06ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5bc0ab760 0x7ff5bc0abc10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5bc140470 0x7ff5bc1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 --2- 172.21.15.135:0/252612561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5bc0c7850 0x7ff5bc151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.655+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 >> 172.21.15.135:0/252612561 conn(0x7ff5bc06c310 msgr2=0x7ff5bc13f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.656+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 shutdown_connections 2024-04-26T22:03:13.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.656+0000 7ff5b17fa640 1 -- 172.21.15.135:0/252612561 wait complete. 2024-04-26T22:03:13.681 INFO:tasks.mds_thrash.fs.[b]:kill mds.b (rank=0) 2024-04-26T22:03:13.681 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T22:03:13.681 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:03:13.707 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T22:03:13.707 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:13.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.891+0000 7ff8fa6d1640 1 Processor -- start 2024-04-26T22:03:13.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.891+0000 7ff8fa6d1640 1 -- start start 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8fa6d1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff8f40c4710 0x7ff8f40718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8fa6d1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f406f0e0 con 0x7ff8f4071e20 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8fa6d1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f40abe70 con 0x7ff8f4167bb0 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8fa6d1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f406f3c0 con 0x7ff8f40c4710 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8f3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff8f4167bb0 0x7ff8f40ab380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54908/0 (socket says 172.21.15.135:54908) 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8f3fff640 1 -- 172.21.15.135:0/1061351893 learned_addr learned my addr 172.21.15.135:0/1061351893 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:13.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.892+0000 7ff8f37fe640 1 --2- 172.21.15.135:0/1061351893 >> v2:172.21.15.135:3300/0 conn(0x7ff8f40c4710 0x7ff8f40718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.893+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1824770867 0 0) 0x7ff8f40abe70 con 0x7ff8f4167bb0 2024-04-26T22:03:13.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.893+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8dc003520 con 0x7ff8f4167bb0 2024-04-26T22:03:13.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.893+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716884873 0 0) 0x7ff8f406f0e0 con 0x7ff8f4071e20 2024-04-26T22:03:13.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.893+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f40abe70 con 0x7ff8f4071e20 2024-04-26T22:03:13.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.894+0000 7ff8f37fe640 1 -- 172.21.15.135:0/1061351893 >> v1:172.21.15.139:6789/0 conn(0x7ff8f4167bb0 legacy=0x7ff8f40ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.894+0000 7ff8f37fe640 1 -- 172.21.15.135:0/1061351893 >> v1:172.21.15.135:6789/0 conn(0x7ff8f4071e20 legacy=0x7ff8f416e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.894+0000 7ff8f37fe640 1 -- 172.21.15.135:0/1061351893 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f406f6a0 con 0x7ff8f40c4710 2024-04-26T22:03:13.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.896+0000 7ff8f37fe640 1 --2- 172.21.15.135:0/1061351893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f40c4710 0x7ff8f40718e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009720 tx=0x7ff8e802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b9770451851041 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:13.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.897+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8037070 con 0x7ff8f40c4710 2024-04-26T22:03:13.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.897+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e802abf0 con 0x7ff8f40c4710 2024-04-26T22:03:13.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.897+0000 7ff8f2ffd640 1 -- 172.21.15.135:0/1061351893 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0018f0 con 0x7ff8f40c4710 2024-04-26T22:03:13.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.898+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1061351893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f40c4710 msgr2=0x7ff8f40718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:13.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.898+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1061351893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f40c4710 0x7ff8f40718e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009720 tx=0x7ff8e802a160 comp rx=0 tx=0).stop 2024-04-26T22:03:13.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.898+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1061351893 shutdown_connections 2024-04-26T22:03:13.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.898+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1061351893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f40c4710 0x7ff8f40718e0 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009720 tx=0x7ff8e802a160 comp rx=0 tx=0).stop 2024-04-26T22:03:13.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.898+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1061351893 >> 172.21.15.135:0/1061351893 conn(0x7ff8f40c82d0 msgr2=0x7ff8f40c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:13.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.899+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1061351893 shutdown_connections 2024-04-26T22:03:13.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.899+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1061351893 wait complete. 2024-04-26T22:03:13.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.901+0000 7ff8fa6d1640 1 Processor -- start 2024-04-26T22:03:13.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.901+0000 7ff8fa6d1640 1 -- start start 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.902+0000 7ff8fa6d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8f4071e20 0x7ff8f408c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.902+0000 7ff8fa6d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4167bb0 0x7ff8f40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.902+0000 7ff8fa6d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 0x7ff8f408ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.902+0000 7ff8fa6d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f40c4b30 con 0x7ff8f4167bb0 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.902+0000 7ff8fa6d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f406fc60 con 0x7ff8f4074730 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.902+0000 7ff8fa6d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f40ac040 con 0x7ff8f4071e20 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 0x7ff8f408ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 0x7ff8f408ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40628/0 (socket says 172.21.15.135:40628) 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 -- 172.21.15.135:0/1109218708 learned_addr learned my addr 172.21.15.135:0/1109218708 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 -- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8f4071e20 msgr2=0x7ff8f408c740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8f4071e20 0x7ff8f408c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 -- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4167bb0 msgr2=0x7ff8f40734c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:13.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4167bb0 0x7ff8f40734c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:13.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.903+0000 7ff8f8c6e640 1 -- 172.21.15.135:0/1109218708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f40bdda0 con 0x7ff8f4074730 2024-04-26T22:03:13.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.904+0000 7ff8f8c6e640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 0x7ff8f408ba80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00caf0 tx=0x7ff8ec002150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:13.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.904+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec017070 con 0x7ff8f4074730 2024-04-26T22:03:13.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.904+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f40be040 con 0x7ff8f4074730 2024-04-26T22:03:13.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.904+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f4203810 con 0x7ff8f4074730 2024-04-26T22:03:13.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.904+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ec003d80 con 0x7ff8f4074730 2024-04-26T22:03:13.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.905+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec00a940 con 0x7ff8f4074730 2024-04-26T22:03:13.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.907+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8ec01a070 con 0x7ff8f4074730 2024-04-26T22:03:13.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.907+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f40ac6e0 con 0x7ff8f4074730 2024-04-26T22:03:13.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.908+0000 7ff8f0ff9640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff8dc06cff0 0x7ff8dc06f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:13.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.909+0000 7ff8f3fff640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff8dc06cff0 0x7ff8dc06f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:13.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.910+0000 7ff8f3fff640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff8dc06cff0 0x7ff8dc06f4b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4001640 tx=0x7ff8e4001450 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:13.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.910+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7ff8ec08b7e0 con 0x7ff8f4074730 2024-04-26T22:03:13.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:13.916+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8f40ac6e0 con 0x7ff8f4074730 2024-04-26T22:03:14.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.405+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8f40abc00 con 0x7ff8f4074730 2024-04-26T22:03:14.407 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.407+0000 7ff8f0ff9640 1 -- 172.21.15.135:0/1109218708 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5668 (secure 0 0 0) 0x7ff8ec0597c0 con 0x7ff8f4074730 2024-04-26T22:03:14.408 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:14.408 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:13.021770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:14.408 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 26 2024-04-26T22:03:14.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.414+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff8dc06cff0 msgr2=0x7ff8dc06f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.414+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff8dc06cff0 0x7ff8dc06f4b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4001640 tx=0x7ff8e4001450 comp rx=0 tx=0).stop 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.414+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 msgr2=0x7ff8f408ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.414+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 0x7ff8f408ba80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00caf0 tx=0x7ff8ec002150 comp rx=0 tx=0).stop 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.415+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 shutdown_connections 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.415+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff8dc06cff0 0x7ff8dc06f4b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.415+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8f4074730 0x7ff8f408ba80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.415+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4167bb0 0x7ff8f40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.415+0000 7ff8fa6d1640 1 --2- 172.21.15.135:0/1109218708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8f4071e20 0x7ff8f408c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:14.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.415+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 >> 172.21.15.135:0/1109218708 conn(0x7ff8f40c82d0 msgr2=0x7ff8f4166e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:14.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.416+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 shutdown_connections 2024-04-26T22:03:14.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:14.416+0000 7ff8fa6d1640 1 -- 172.21.15.135:0/1109218708 wait complete. 2024-04-26T22:03:14.446 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:03:15.507 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:15.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.706+0000 7f8fbaf38640 1 Processor -- start 2024-04-26T22:03:15.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.707+0000 7f8fbaf38640 1 -- start start 2024-04-26T22:03:15.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fbaf38640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8fb4150200 0x7f8fb40ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fbaf38640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8fb40c2cd0 0x7f8fb41533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fbaf38640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ac680 0x7f8fb41538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fbaf38640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb40c4bd0 con 0x7f8fb4150200 2024-04-26T22:03:15.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fbaf38640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb406ff20 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fbaf38640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb406f4e0 con 0x7f8fb40ac680 2024-04-26T22:03:15.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fb8cd4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8fb4150200 0x7f8fb40ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fb8cd4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8fb4150200 0x7f8fb40ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55070/0 (socket says 172.21.15.135:55070) 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.708+0000 7f8fb8cd4640 1 -- 172.21.15.135:0/2280135454 learned_addr learned my addr 172.21.15.135:0/2280135454 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.709+0000 7f8fb94d5640 1 --2- 172.21.15.135:0/2280135454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ac680 0x7f8fb41538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.709+0000 7f8fb8cd4640 1 -- 172.21.15.135:0/2280135454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ac680 msgr2=0x7f8fb41538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.709+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/2280135454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ac680 0x7f8fb41538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.709+0000 7f8fb8cd4640 1 -- 172.21.15.135:0/2280135454 >> v2:172.21.15.139:3300/0 conn(0x7f8fb40c2cd0 msgr2=0x7f8fb41533b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.709+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/2280135454 >> v2:172.21.15.139:3300/0 conn(0x7f8fb40c2cd0 0x7f8fb41533b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.709+0000 7f8fb8cd4640 1 -- 172.21.15.135:0/2280135454 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb40c48f0 con 0x7f8fb4150200 2024-04-26T22:03:15.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.711+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/2280135454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb4150200 0x7f8fb40ac140 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40c8210 tx=0x7f8fa8032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10e7325731bea16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.712+0000 7f8fb37fe640 1 -- 172.21.15.135:0/2280135454 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8039070 con 0x7f8fb4150200 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.712+0000 7f8fb37fe640 1 -- 172.21.15.135:0/2280135454 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa80339c0 con 0x7f8fb4150200 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.712+0000 7f8fb37fe640 1 -- 172.21.15.135:0/2280135454 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8033d90 con 0x7f8fb4150200 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 -- 172.21.15.135:0/2280135454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb4150200 msgr2=0x7f8fb40ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/2280135454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb4150200 0x7f8fb40ac140 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40c8210 tx=0x7f8fa8032c70 comp rx=0 tx=0).stop 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 -- 172.21.15.135:0/2280135454 shutdown_connections 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/2280135454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ac680 0x7f8fb41538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/2280135454 >> v2:172.21.15.139:3300/0 conn(0x7f8fb40c2cd0 0x7f8fb41533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/2280135454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb4150200 0x7f8fb40ac140 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.714+0000 7f8fbaf38640 1 -- 172.21.15.135:0/2280135454 >> 172.21.15.135:0/2280135454 conn(0x7f8fb40c82d0 msgr2=0x7f8fb40c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.719+0000 7f8fbaf38640 1 -- 172.21.15.135:0/2280135454 shutdown_connections 2024-04-26T22:03:15.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.719+0000 7f8fbaf38640 1 -- 172.21.15.135:0/2280135454 wait complete. 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.721+0000 7f8fbaf38640 1 Processor -- start 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.721+0000 7f8fbaf38640 1 -- start start 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fbaf38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb40ac680 0x7f8fb4156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fbaf38640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 0x7f8fb4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fbaf38640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb4150200 0x7f8fb40c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fbaf38640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb4150760 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fbaf38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb41505e0 con 0x7f8fb40ac680 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fbaf38640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40c8dc0 con 0x7f8fb4150200 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fb3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 0x7f8fb4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fb3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 0x7f8fb4145cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55082/0 (socket says 172.21.15.135:55082) 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fb3fff640 1 -- 172.21.15.135:0/72818017 learned_addr learned my addr 172.21.15.135:0/72818017 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.722+0000 7f8fb94d5640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb4150200 0x7f8fb40c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:15.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb40ac680 0x7f8fb4156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb3fff640 1 -- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb4150200 msgr2=0x7f8fb40c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb3fff640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb4150200 0x7f8fb40c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb3fff640 1 -- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb40ac680 msgr2=0x7f8fb4156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb3fff640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb40ac680 0x7f8fb4156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb3fff640 1 -- 172.21.15.135:0/72818017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb408d760 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb94d5640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb4150200 0x7f8fb40c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb40ac680 0x7f8fb4156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.723+0000 7f8fb3fff640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 0x7f8fb4145cd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b9e0 tx=0x7f8fac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.724+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac019070 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.725+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb4154270 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.725+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb408da20 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.725+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac00cb40 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.725+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac014440 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.727+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94005290 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.729+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fac01c360 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.729+0000 7f8fb17fa640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8fa006c9f0 0x7f8fa006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:15.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.729+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7f8fac08bb20 con 0x7f8fb40c2cd0 2024-04-26T22:03:15.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.730+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8fa006c9f0 0x7f8fa006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:15.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.732+0000 7f8fb8cd4640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8fa006c9f0 0x7f8fa006eeb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8fa803e000 tx=0x7f8fa8012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:15.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:15.737+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8fac0594b0 con 0x7f8fb40c2cd0 2024-04-26T22:03:16.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.202+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f94005580 con 0x7f8fb40c2cd0 2024-04-26T22:03:16.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.203+0000 7f8fb17fa640 1 -- 172.21.15.135:0/72818017 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5675 (secure 0 0 0) 0x7f8fac05d7b0 con 0x7f8fb40c2cd0 2024-04-26T22:03:16.204 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:16.204 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:16.205 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 27 2024-04-26T22:03:16.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8fa006c9f0 msgr2=0x7f8fa006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8fa006c9f0 0x7f8fa006eeb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8fa803e000 tx=0x7f8fa8012040 comp rx=0 tx=0).stop 2024-04-26T22:03:16.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 msgr2=0x7f8fb4145cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 0x7f8fb4145cd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b9e0 tx=0x7f8fac00bea0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 shutdown_connections 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8fa006c9f0 0x7f8fa006eeb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb4150200 0x7f8fb40c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fb40c2cd0 0x7f8fb4145cd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 --2- 172.21.15.135:0/72818017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb40ac680 0x7f8fb4156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 >> 172.21.15.135:0/72818017 conn(0x7f8fb40c82d0 msgr2=0x7f8fb414e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 shutdown_connections 2024-04-26T22:03:16.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.207+0000 7f8fbaf38640 1 -- 172.21.15.135:0/72818017 wait complete. 2024-04-26T22:03:16.227 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=4244 2024-04-26T22:03:16.227 INFO:tasks.mds_thrash.fs.[a]:waiting for 5 secs before reviving mds.d 2024-04-26T22:03:16.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:16.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.573+0000 7fd3ccdaf640 1 Processor -- start 2024-04-26T22:03:16.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.573+0000 7fd3ccdaf640 1 -- start start 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.574+0000 7fd3ccdaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.574+0000 7fd3ccdaf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd3c815a1a0 0x7fd3c80c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.574+0000 7fd3ccdaf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c80c8df0 con 0x7fd3c8154160 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.574+0000 7fd3ccdaf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c80c9100 con 0x7fd3c815a1a0 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.574+0000 7fd3ccdaf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c8154540 con 0x7fd3c80ca5a0 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c659c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd3c8154160 0x7fd3c80c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39530/0 (socket says 172.21.15.135:39530) 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c659c640 1 -- 172.21.15.135:0/254731758 learned_addr learned my addr 172.21.15.135:0/254731758 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c6d9d640 1 --2- 172.21.15.135:0/254731758 >> v2:172.21.15.139:3300/0 conn(0x7fd3c815a1a0 0x7fd3c80c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c5d9b640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c559a640 1 -- 172.21.15.135:0/254731758 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673457126 0 0) 0x7fd3b8001110 con 0x7fd3c8154160 2024-04-26T22:03:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c559a640 1 -- 172.21.15.135:0/254731758 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3b0003520 con 0x7fd3c8154160 2024-04-26T22:03:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 msgr2=0x7fd3c8156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c6d9d640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c8156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/254731758 >> v1:172.21.15.135:6789/0 conn(0x7fd3c8154160 legacy=0x7fd3c80c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/254731758 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c815a910 con 0x7fd3c815a1a0 2024-04-26T22:03:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.575+0000 7fd3c5d9b640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c8156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:16.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.576+0000 7fd3c6d9d640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c815a1a0 0x7fd3c80c9590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009d90 tx=0x7fd3c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2554e5f33da093 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:16.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.576+0000 7fd3c559a640 1 -- 172.21.15.135:0/254731758 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0007740 con 0x7fd3c815a1a0 2024-04-26T22:03:16.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.576+0000 7fd3c559a640 1 -- 172.21.15.135:0/254731758 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c00078e0 con 0x7fd3c815a1a0 2024-04-26T22:03:16.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.576+0000 7fd3c559a640 1 -- 172.21.15.135:0/254731758 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0007c90 con 0x7fd3c815a1a0 2024-04-26T22:03:16.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c815a1a0 msgr2=0x7fd3c80c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c815a1a0 0x7fd3c80c9590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009d90 tx=0x7fd3c002a660 comp rx=0 tx=0).stop 2024-04-26T22:03:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/254731758 shutdown_connections 2024-04-26T22:03:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c815a1a0 0x7fd3c80c9590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/254731758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c8156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/254731758 >> 172.21.15.135:0/254731758 conn(0x7fd3c80ae010 msgr2=0x7fd3c80ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/254731758 shutdown_connections 2024-04-26T22:03:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.577+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/254731758 wait complete. 2024-04-26T22:03:16.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.578+0000 7fd3ccdaf640 1 Processor -- start 2024-04-26T22:03:16.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.578+0000 7fd3ccdaf640 1 -- start start 2024-04-26T22:03:16.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3ccdaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:16.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3ccdaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c8154160 0x7fd3c81ec940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3ccdaf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 0x7fd3c8155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3ccdaf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c815a630 con 0x7fd3c815a1a0 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3ccdaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c815aed0 con 0x7fd3c8154160 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3ccdaf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c80c7510 con 0x7fd3c80ca5a0 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 0x7fd3c8155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 0x7fd3c8155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55096/0 (socket says 172.21.15.135:55096) 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/3362286360 learned_addr learned my addr 172.21.15.135:0/3362286360 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c659c640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 msgr2=0x7fd3c814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c8154160 msgr2=0x7fd3c81ec940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c8154160 0x7fd3c81ec940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 -- 172.21.15.135:0/3362286360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c814a600 con 0x7fd3c815a1a0 2024-04-26T22:03:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.579+0000 7fd3c6d9d640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 0x7fd3c8155e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c002ac50 tx=0x7fd3c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.580+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c003f070 con 0x7fd3c815a1a0 2024-04-26T22:03:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.580+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c0031c60 con 0x7fd3c815a1a0 2024-04-26T22:03:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.580+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c814ae60 con 0x7fd3c815a1a0 2024-04-26T22:03:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.581+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0009450 con 0x7fd3c815a1a0 2024-04-26T22:03:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.581+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c81f00c0 con 0x7fd3c815a1a0 2024-04-26T22:03:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.581+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3c003b410 con 0x7fd3c815a1a0 2024-04-26T22:03:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.582+0000 7fd3b6ffd640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd3a806c920 0x7fd3a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.582+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7fd3c00b1060 con 0x7fd3c815a1a0 2024-04-26T22:03:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.582+0000 7fd3c659c640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd3a806c920 0x7fd3a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.583+0000 7fd3c659c640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd3a806c920 0x7fd3a806ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005880 tx=0x7fd3b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.583+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c81f02c0 con 0x7fd3c815a1a0 2024-04-26T22:03:16.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.587+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3c81f02c0 con 0x7fd3c815a1a0 2024-04-26T22:03:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.843+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3c81efeb0 con 0x7fd3c815a1a0 2024-04-26T22:03:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.844+0000 7fd3b6ffd640 1 -- 172.21.15.135:0/3362286360 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5675 (secure 0 0 0) 0x7fd3c007a800 con 0x7fd3c815a1a0 2024-04-26T22:03:16.844 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:16.845 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 27 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd3a806c920 msgr2=0x7fd3a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd3a806c920 0x7fd3a806ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005880 tx=0x7fd3b8009040 comp rx=0 tx=0).stop 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 msgr2=0x7fd3c8155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 0x7fd3c8155e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c002ac50 tx=0x7fd3c0004a40 comp rx=0 tx=0).stop 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 shutdown_connections 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd3a806c920 0x7fd3a806ede0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3c815a1a0 0x7fd3c8155e10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3c8154160 0x7fd3c81ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 --2- 172.21.15.135:0/3362286360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3c80ca5a0 0x7fd3c814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 >> 172.21.15.135:0/3362286360 conn(0x7fd3c80ae010 msgr2=0x7fd3c80cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.846+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 shutdown_connections 2024-04-26T22:03:16.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:16.847+0000 7fd3ccdaf640 1 -- 172.21.15.135:0/3362286360 wait complete. 2024-04-26T22:03:16.862 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:03:17.826 INFO:tasks.ceph.mds.a.smithi135.stderr:2024-04-26T22:03:17.825+0000 7f1502651640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:03:18.863 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:19.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.135+0000 7f2e3fdda640 1 Processor -- start 2024-04-26T22:03:19.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.135+0000 7f2e3fdda640 1 -- start start 2024-04-26T22:03:19.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3fdda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3806e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:19.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3fdda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3815c0d0 con 0x7f2e380c3650 2024-04-26T22:03:19.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3fdda640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e380ab4b0 con 0x7f2e380abf50 2024-04-26T22:03:19.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3fdda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3815bdf0 con 0x7f2e380aa8c0 2024-04-26T22:03:19.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3e377640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2e380abf50 0x7f2e3815a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:44098/0 (socket says 172.21.15.135:44098) 2024-04-26T22:03:19.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3e377640 1 -- 172.21.15.135:0/1748263557 learned_addr learned my addr 172.21.15.135:0/1748263557 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:19.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.137+0000 7f2e3d375640 1 --2- 172.21.15.135:0/1748263557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3806e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:19.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.138+0000 7f2e3d375640 1 -- 172.21.15.135:0/1748263557 >> v1:172.21.15.139:6789/0 conn(0x7f2e380abf50 legacy=0x7f2e3815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:19.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.138+0000 7f2e3d375640 1 -- 172.21.15.135:0/1748263557 >> v1:172.21.15.135:6789/0 conn(0x7f2e380c3650 legacy=0x7f2e380c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:19.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.138+0000 7f2e3d375640 1 -- 172.21.15.135:0/1748263557 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e380ab750 con 0x7f2e380aa8c0 2024-04-26T22:03:19.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.139+0000 7f2e3d375640 1 --2- 172.21.15.135:0/1748263557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3806e760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e280097d0 tx=0x7f2e2802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f583fe6f83778a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:19.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.139+0000 7f2e3cb74640 1 -- 172.21.15.135:0/1748263557 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28037070 con 0x7f2e380aa8c0 2024-04-26T22:03:19.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.140+0000 7f2e3cb74640 1 -- 172.21.15.135:0/1748263557 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2802ae40 con 0x7f2e380aa8c0 2024-04-26T22:03:19.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.140+0000 7f2e3cb74640 1 -- 172.21.15.135:0/1748263557 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28032660 con 0x7f2e380aa8c0 2024-04-26T22:03:19.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.141+0000 7f2e3fdda640 1 -- 172.21.15.135:0/1748263557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 msgr2=0x7f2e3806e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:19.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.141+0000 7f2e3fdda640 1 --2- 172.21.15.135:0/1748263557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3806e760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e280097d0 tx=0x7f2e2802a0f0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.141+0000 7f2e3fdda640 1 -- 172.21.15.135:0/1748263557 shutdown_connections 2024-04-26T22:03:19.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.141+0000 7f2e3fdda640 1 --2- 172.21.15.135:0/1748263557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3806e760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.141+0000 7f2e3fdda640 1 -- 172.21.15.135:0/1748263557 >> 172.21.15.135:0/1748263557 conn(0x7f2e380c82d0 msgr2=0x7f2e380c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:19.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.143+0000 7f2e3fdda640 1 -- 172.21.15.135:0/1748263557 shutdown_connections 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.143+0000 7f2e3fdda640 1 -- 172.21.15.135:0/1748263557 wait complete. 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.144+0000 7f2e3fdda640 1 Processor -- start 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.145+0000 7f2e3fdda640 1 -- start start 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3fdda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3820dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3fdda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e380abf50 0x7f2e3814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3fdda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 0x7f2e3814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3fdda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e380c42c0 con 0x7f2e380c3650 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3fdda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e380c4140 con 0x7f2e380abf50 2024-04-26T22:03:19.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3fdda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3815adb0 con 0x7f2e380aa8c0 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3e377640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 0x7f2e3814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3e377640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 0x7f2e3814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45638/0 (socket says 172.21.15.135:45638) 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3e377640 1 -- 172.21.15.135:0/4251121227 learned_addr learned my addr 172.21.15.135:0/4251121227 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.146+0000 7f2e3db76640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3820dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.147+0000 7f2e3e377640 1 -- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 msgr2=0x7f2e3820dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.147+0000 7f2e3e377640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3820dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.147+0000 7f2e3e377640 1 -- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e380abf50 msgr2=0x7f2e3814e3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.147+0000 7f2e3e377640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e380abf50 0x7f2e3814e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.147+0000 7f2e3e377640 1 -- 172.21.15.135:0/4251121227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3820e6d0 con 0x7f2e380c3650 2024-04-26T22:03:19.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.147+0000 7f2e3e377640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 0x7f2e3814e900 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c450 tx=0x7f2e3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:19.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.148+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34017070 con 0x7f2e380c3650 2024-04-26T22:03:19.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.148+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34005ce0 con 0x7f2e380c3650 2024-04-26T22:03:19.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.149+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34007270 con 0x7f2e380c3650 2024-04-26T22:03:19.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.149+0000 7f2e3fdda640 1 -- 172.21.15.135:0/4251121227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e38071b80 con 0x7f2e380c3650 2024-04-26T22:03:19.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.149+0000 7f2e3fdda640 1 -- 172.21.15.135:0/4251121227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e380725f0 con 0x7f2e380c3650 2024-04-26T22:03:19.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.153+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e3401a070 con 0x7f2e380c3650 2024-04-26T22:03:19.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.154+0000 7f2e2e7fc640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2e1c06c9c0 0x7f2e1c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:19.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.154+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7f2e3408b1e0 con 0x7f2e380c3650 2024-04-26T22:03:19.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.154+0000 7f2e3fdda640 1 -- 172.21.15.135:0/4251121227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3820e8a0 con 0x7f2e380c3650 2024-04-26T22:03:19.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.157+0000 7f2e3db76640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2e1c06c9c0 0x7f2e1c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:19.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.157+0000 7f2e3db76640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2e1c06c9c0 0x7f2e1c06ee80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2e300012e0 tx=0x7f2e30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:19.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.166+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e3820e8a0 con 0x7f2e380c3650 2024-04-26T22:03:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.713+0000 7f2e3fdda640 1 -- 172.21.15.135:0/4251121227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e38071970 con 0x7f2e380c3650 2024-04-26T22:03:19.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.714+0000 7f2e2e7fc640 1 -- 172.21.15.135:0/4251121227 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5675 (secure 0 0 0) 0x7f2e34058b70 con 0x7f2e380c3650 2024-04-26T22:03:19.716 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:19.716 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:19.716 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 27 2024-04-26T22:03:19.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.719+0000 7f2e1bfff640 1 -- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2e1c06c9c0 msgr2=0x7f2e1c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:19.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.719+0000 7f2e1bfff640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2e1c06c9c0 0x7f2e1c06ee80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2e300012e0 tx=0x7f2e30009040 comp rx=0 tx=0).stop 2024-04-26T22:03:19.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.719+0000 7f2e1bfff640 1 -- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 msgr2=0x7f2e3814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:19.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.719+0000 7f2e1bfff640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 0x7f2e3814e900 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c450 tx=0x7f2e3400c910 comp rx=0 tx=0).stop 2024-04-26T22:03:19.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 -- 172.21.15.135:0/4251121227 shutdown_connections 2024-04-26T22:03:19.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2e1c06c9c0 0x7f2e1c06ee80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e380c3650 0x7f2e3814e900 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e380abf50 0x7f2e3814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 --2- 172.21.15.135:0/4251121227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e380aa8c0 0x7f2e3820dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:19.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 -- 172.21.15.135:0/4251121227 >> 172.21.15.135:0/4251121227 conn(0x7f2e380c82d0 msgr2=0x7f2e38157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:19.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 -- 172.21.15.135:0/4251121227 shutdown_connections 2024-04-26T22:03:19.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:19.720+0000 7f2e1bfff640 1 -- 172.21.15.135:0/4251121227 wait complete. 2024-04-26T22:03:19.747 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:03:21.228 INFO:tasks.mds_thrash.fs.[a]:reviving mds.d 2024-04-26T22:03:21.228 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T22:03:21.228 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T22:03:21.232 INFO:tasks.ceph.mds.d:Started 2024-04-26T22:03:21.232 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:21.360 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-04-26T22:03:21.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.435+0000 7f933d464640 1 Processor -- start 2024-04-26T22:03:21.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.435+0000 7f933d464640 1 -- start start 2024-04-26T22:03:21.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.436+0000 7f933d464640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93380c68f0 0x7f933806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.436+0000 7f933d464640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9338154e70 con 0x7f93380c6e00 2024-04-26T22:03:21.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.436+0000 7f933d464640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9338155150 con 0x7f93381520d0 2024-04-26T22:03:21.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.436+0000 7f933d464640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f93380adbc0 con 0x7f93380c68f0 2024-04-26T22:03:21.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.437+0000 7f93377fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93380c68f0 0x7f933806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:21.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.437+0000 7f93367fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f93380c6e00 0x7f933806f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35736/0 (socket says 172.21.15.135:35736) 2024-04-26T22:03:21.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.437+0000 7f93367fc640 1 -- 172.21.15.135:0/893652465 learned_addr learned my addr 172.21.15.135:0/893652465 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:21.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.438+0000 7f9335ffb640 1 -- 172.21.15.135:0/893652465 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713373688 0 0) 0x7f9338154e70 con 0x7f93380c6e00 2024-04-26T22:03:21.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.438+0000 7f9335ffb640 1 -- 172.21.15.135:0/893652465 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9320003520 con 0x7f93380c6e00 2024-04-26T22:03:21.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.438+0000 7f93377fe640 1 -- 172.21.15.135:0/893652465 >> v1:172.21.15.139:6789/0 conn(0x7f93381520d0 legacy=0x7f93380addb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:21.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.438+0000 7f93377fe640 1 -- 172.21.15.135:0/893652465 >> v1:172.21.15.135:6789/0 conn(0x7f93380c6e00 legacy=0x7f933806f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:21.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.438+0000 7f93377fe640 1 -- 172.21.15.135:0/893652465 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9338155430 con 0x7f93380c68f0 2024-04-26T22:03:21.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.440+0000 7f93377fe640 1 --2- 172.21.15.135:0/893652465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93380c68f0 0x7f933806fe80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f932c0097d0 tx=0x7f932c02a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=181428d9c6bbb75e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:21.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.443+0000 7f9335ffb640 1 -- 172.21.15.135:0/893652465 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c037070 con 0x7f93380c68f0 2024-04-26T22:03:21.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.443+0000 7f9335ffb640 1 -- 172.21.15.135:0/893652465 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932c02ae30 con 0x7f93380c68f0 2024-04-26T22:03:21.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.443+0000 7f9335ffb640 1 -- 172.21.15.135:0/893652465 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c032640 con 0x7f93380c68f0 2024-04-26T22:03:21.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.448+0000 7f933d464640 1 -- 172.21.15.135:0/893652465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93380c68f0 msgr2=0x7f933806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:21.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.448+0000 7f933d464640 1 --2- 172.21.15.135:0/893652465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93380c68f0 0x7f933806fe80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f932c0097d0 tx=0x7f932c02a120 comp rx=0 tx=0).stop 2024-04-26T22:03:21.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.450+0000 7f933d464640 1 -- 172.21.15.135:0/893652465 shutdown_connections 2024-04-26T22:03:21.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.450+0000 7f933d464640 1 --2- 172.21.15.135:0/893652465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93380c68f0 0x7f933806fe80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:21.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.450+0000 7f933d464640 1 -- 172.21.15.135:0/893652465 >> 172.21.15.135:0/893652465 conn(0x7f93380c89f0 msgr2=0x7f93380c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:21.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.454+0000 7f933d464640 1 -- 172.21.15.135:0/893652465 shutdown_connections 2024-04-26T22:03:21.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.454+0000 7f933d464640 1 -- 172.21.15.135:0/893652465 wait complete. 2024-04-26T22:03:21.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.456+0000 7f933d464640 1 Processor -- start 2024-04-26T22:03:21.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.456+0000 7f933d464640 1 -- start start 2024-04-26T22:03:21.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.457+0000 7f933d464640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93380c68f0 0x7f9338151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.457+0000 7f933d464640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 0x7f93380c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f933d464640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93381520d0 0x7f93381ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f933d464640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9338154b20 con 0x7f93380c68f0 2024-04-26T22:03:21.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f933d464640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f93380703c0 con 0x7f93380c6e00 2024-04-26T22:03:21.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f933d464640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f93380ae4c0 con 0x7f93381520d0 2024-04-26T22:03:21.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 0x7f93380c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:21.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93380c68f0 0x7f9338151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:21.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 0x7f93380c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:52578/0 (socket says 172.21.15.135:52578) 2024-04-26T22:03:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 -- 172.21.15.135:0/361284380 learned_addr learned my addr 172.21.15.135:0/361284380 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93377fe640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93381520d0 0x7f93381ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 -- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93381520d0 msgr2=0x7f93381ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93381520d0 0x7f93381ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 -- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93380c68f0 msgr2=0x7f9338151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93380c68f0 0x7f9338151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.458+0000 7f93367fc640 1 -- 172.21.15.135:0/361284380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93381f9660 con 0x7f93380c6e00 2024-04-26T22:03:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.459+0000 7f93377fe640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93381520d0 0x7f93381ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.459+0000 7f93367fc640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 0x7f93380c7320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f932800c460 tx=0x7f932800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.460+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328017070 con 0x7f93380c6e00 2024-04-26T22:03:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.460+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9328005ce0 con 0x7f93380c6e00 2024-04-26T22:03:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.460+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328007270 con 0x7f93380c6e00 2024-04-26T22:03:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.460+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93381f9940 con 0x7f93380c6e00 2024-04-26T22:03:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.461+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93381f9f00 con 0x7f93380c6e00 2024-04-26T22:03:21.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.463+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f932801a070 con 0x7f93380c6e00 2024-04-26T22:03:21.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.465+0000 7f931f7fe640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f932406c9c0 0x7f932406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.465+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93380765e0 con 0x7f93380c6e00 2024-04-26T22:03:21.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.466+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7f932808b240 con 0x7f93380c6e00 2024-04-26T22:03:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.469+0000 7f9336ffd640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f932406c9c0 0x7f932406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.470+0000 7f9336ffd640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f932406c9c0 0x7f932406ee80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9330009fd0 tx=0x7f933000d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:21.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.475+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9328058bd0 con 0x7f93380c6e00 2024-04-26T22:03:21.748 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:21.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.980+0000 7fc337fe9640 1 Processor -- start 2024-04-26T22:03:21.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.980+0000 7fc337fe9640 1 -- start start 2024-04-26T22:03:21.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.981+0000 7fc337fe9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3300c2520 0x7fc33015b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.981+0000 7fc337fe9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 0x7fc3300cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:21.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.981+0000 7fc337fe9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc33015bff0 con 0x7fc330158790 2024-04-26T22:03:21.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.981+0000 7fc337fe9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc3300ca500 con 0x7fc3300c2520 2024-04-26T22:03:21.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.981+0000 7fc337fe9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc33015b750 con 0x7fc3300ca990 2024-04-26T22:03:21.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.983+0000 7fc336586640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 0x7fc3300cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:21.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.983+0000 7fc336586640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 0x7fc3300cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:45668/0 (socket says 172.21.15.135:45668) 2024-04-26T22:03:21.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.983+0000 7fc336586640 1 -- 172.21.15.135:0/2873717845 learned_addr learned my addr 172.21.15.135:0/2873717845 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:21.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.984+0000 7fc336586640 1 -- 172.21.15.135:0/2873717845 >> v2:172.21.15.139:3300/0 conn(0x7fc3300c2520 msgr2=0x7fc33015b050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:21.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.984+0000 7fc336586640 1 --2- 172.21.15.135:0/2873717845 >> v2:172.21.15.139:3300/0 conn(0x7fc3300c2520 0x7fc33015b050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:21.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.984+0000 7fc336586640 1 -- 172.21.15.135:0/2873717845 >> v1:172.21.15.139:6789/0 conn(0x7fc330158790 legacy=0x7fc3300ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:21.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.984+0000 7fc336586640 1 -- 172.21.15.135:0/2873717845 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33015ba30 con 0x7fc3300ca990 2024-04-26T22:03:21.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.986+0000 7fc336586640 1 --2- 172.21.15.135:0/2873717845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 0x7fc3300cae20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009e50 tx=0x7fc32c02a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4122545f657826 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:21.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.986+0000 7fc334d83640 1 -- 172.21.15.135:0/2873717845 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c036070 con 0x7fc3300ca990 2024-04-26T22:03:21.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.986+0000 7fc334d83640 1 -- 172.21.15.135:0/2873717845 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32c031600 con 0x7fc3300ca990 2024-04-26T22:03:21.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.987+0000 7fc337fe9640 1 -- 172.21.15.135:0/2873717845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 msgr2=0x7fc3300cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:21.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.987+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2873717845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 0x7fc3300cae20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009e50 tx=0x7fc32c02a630 comp rx=0 tx=0).stop 2024-04-26T22:03:21.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.987+0000 7fc337fe9640 1 -- 172.21.15.135:0/2873717845 shutdown_connections 2024-04-26T22:03:21.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.987+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2873717845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300ca990 0x7fc3300cae20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:21.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.987+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2873717845 >> v2:172.21.15.139:3300/0 conn(0x7fc3300c2520 0x7fc33015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:21.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.987+0000 7fc337fe9640 1 -- 172.21.15.135:0/2873717845 >> 172.21.15.135:0/2873717845 conn(0x7fc3300cc4d0 msgr2=0x7fc3300cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:21.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.989+0000 7fc337fe9640 1 -- 172.21.15.135:0/2873717845 shutdown_connections 2024-04-26T22:03:21.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.990+0000 7fc337fe9640 1 -- 172.21.15.135:0/2873717845 wait complete. 2024-04-26T22:03:21.993 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:21.994 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:21.995 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 27 2024-04-26T22:03:21.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.990+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93380507a0 con 0x7f93380c6e00 2024-04-26T22:03:21.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.991+0000 7f931f7fe640 1 -- 172.21.15.135:0/361284380 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5675 (secure 0 0 0) 0x7f932805ced0 con 0x7f93380c6e00 2024-04-26T22:03:22.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.992+0000 7fc337fe9640 1 Processor -- start 2024-04-26T22:03:22.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.992+0000 7fc337fe9640 1 -- start start 2024-04-26T22:03:22.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.993+0000 7fc337fe9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 0x7fc3301480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:22.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.993+0000 7fc337fe9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc330158790 0x7fc3300add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:22.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.993+0000 7fc337fe9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3301fc700 0x7fc3301fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:22.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.993+0000 7fc337fe9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3300ae780 con 0x7fc330158790 2024-04-26T22:03:22.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.993+0000 7fc337fe9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc3300cc280 con 0x7fc3301fc700 2024-04-26T22:03:22.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.993+0000 7fc337fe9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc3300ae900 con 0x7fc3300c2520 2024-04-26T22:03:22.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335d85640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 0x7fc3301480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:22.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc336586640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3301fc700 0x7fc3301fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:22.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335d85640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 0x7fc3301480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:45680/0 (socket says 172.21.15.135:45680) 2024-04-26T22:03:22.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335d85640 1 -- 172.21.15.135:0/2987565839 learned_addr learned my addr 172.21.15.135:0/2987565839 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:22.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335d85640 1 -- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3301fc700 msgr2=0x7fc3301fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:22.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335d85640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3301fc700 0x7fc3301fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335d85640 1 -- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc330158790 msgr2=0x7fc3300add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:22.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.994+0000 7fc335584640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc330158790 0x7fc3300add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:22.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.995+0000 7fc335d85640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc330158790 0x7fc3300add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.995+0000 7fc335d85640 1 -- 172.21.15.135:0/2987565839 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33013ce50 con 0x7fc3300c2520 2024-04-26T22:03:22.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.995+0000 7fc335584640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc330158790 0x7fc3300add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:22.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.995+0000 7fc335d85640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 0x7fc3301480e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c7b0 tx=0x7fc32000cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:22.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.995+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320017070 con 0x7fc3300c2520 2024-04-26T22:03:22.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.996+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f932406c9c0 msgr2=0x7f932406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:22.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.996+0000 7f933d464640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f932406c9c0 0x7f932406ee80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9330009fd0 tx=0x7f933000d040 comp rx=0 tx=0).stop 2024-04-26T22:03:22.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.996+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 msgr2=0x7f93380c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:22.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.996+0000 7f933d464640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 0x7f93380c7320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f932800c460 tx=0x7f932800c920 comp rx=0 tx=0).stop 2024-04-26T22:03:22.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 shutdown_connections 2024-04-26T22:03:22.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f932406c9c0 0x7f932406ee80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93381520d0 0x7f93381ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c6e00 0x7f93380c7320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 --2- 172.21.15.135:0/361284380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93380c68f0 0x7f9338151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 >> 172.21.15.135:0/361284380 conn(0x7f93380c89f0 msgr2=0x7f93380c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:22.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 shutdown_connections 2024-04-26T22:03:22.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.997+0000 7f933d464640 1 -- 172.21.15.135:0/361284380 wait complete. 2024-04-26T22:03:22.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.996+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3300acce0 con 0x7fc3300c2520 2024-04-26T22:03:22.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.996+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3300ad2a0 con 0x7fc3300c2520 2024-04-26T22:03:22.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.998+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc320005270 con 0x7fc3300c2520 2024-04-26T22:03:22.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:21.998+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32000a640 con 0x7fc3300c2520 2024-04-26T22:03:22.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.001+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc318005290 con 0x7fc3300c2520 2024-04-26T22:03:22.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.001+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc32001a070 con 0x7fc3300c2520 2024-04-26T22:03:22.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.002+0000 7fc3267fc640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc31406c9f0 0x7fc31406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:22.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.002+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7fc32008b320 con 0x7fc3300c2520 2024-04-26T22:03:22.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.002+0000 7fc335584640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc31406c9f0 0x7fc31406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:22.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.003+0000 7fc335584640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc31406c9f0 0x7fc31406eeb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc32800a8b0 tx=0x7fc328008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:22.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.008+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc320058c20 con 0x7fc3300c2520 2024-04-26T22:03:22.028 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-04-26T22:03:22.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.496+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc318005580 con 0x7fc3300c2520 2024-04-26T22:03:22.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.498+0000 7fc3267fc640 1 -- 172.21.15.135:0/2987565839 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6503 (secure 0 0 0) 0x7fc32005cf20 con 0x7fc3300c2520 2024-04-26T22:03:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:22.502 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:22.502 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 28 2024-04-26T22:03:22.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.503+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc31406c9f0 msgr2=0x7fc31406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:22.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.503+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc31406c9f0 0x7fc31406eeb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc32800a8b0 tx=0x7fc328008040 comp rx=0 tx=0).stop 2024-04-26T22:03:22.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.503+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 msgr2=0x7fc3301480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:22.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.503+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 0x7fc3301480e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c7b0 tx=0x7fc32000cc70 comp rx=0 tx=0).stop 2024-04-26T22:03:22.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.504+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 shutdown_connections 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.504+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc31406c9f0 0x7fc31406eeb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.504+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3301fc700 0x7fc3301fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.504+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc330158790 0x7fc3300add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.504+0000 7fc337fe9640 1 --2- 172.21.15.135:0/2987565839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3300c2520 0x7fc3301480e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.504+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 >> 172.21.15.135:0/2987565839 conn(0x7fc3300cc4d0 msgr2=0x7fc330165fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.505+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 shutdown_connections 2024-04-26T22:03:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:22.505+0000 7fc337fe9640 1 -- 172.21.15.135:0/2987565839 wait complete. 2024-04-26T22:03:22.527 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:03:24.029 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:24.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.295+0000 7fbd3442c640 1 Processor -- start 2024-04-26T22:03:24.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.295+0000 7fbd3442c640 1 -- start start 2024-04-26T22:03:24.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.297+0000 7fbd3442c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbd240f5840 0x7fbd241038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.297+0000 7fbd3442c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd2400c3a0 con 0x7fbd240f9420 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.297+0000 7fbd3442c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd240f8eb0 con 0x7fbd240463c0 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.297+0000 7fbd3442c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbd24006ae0 con 0x7fbd240f5840 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.299+0000 7fbd319c7640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbd240f9420 0x7fbd240f5130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35744/0 (socket says 172.21.15.135:35744) 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.299+0000 7fbd319c7640 1 -- 172.21.15.135:0/81874965 learned_addr learned my addr 172.21.15.135:0/81874965 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45817774 0 0) 0x7fbd2400c3a0 con 0x7fbd240f9420 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd18003500 con 0x7fbd240f9420 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1477949058 0 0) 0x7fbd18003500 con 0x7fbd240f9420 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 >> v2:172.21.15.139:3300/0 conn(0x7fbd240f5840 msgr2=0x7fbd241038c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 --2- 172.21.15.135:0/81874965 >> v2:172.21.15.139:3300/0 conn(0x7fbd240f5840 0x7fbd241038c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 >> v1:172.21.15.139:6789/0 conn(0x7fbd240463c0 legacy=0x7fbd240f87a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:24.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd24006dc0 con 0x7fbd240f9420 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fbd1c001310 con 0x7fbd240f9420 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 >> v1:172.21.15.135:6789/0 conn(0x7fbd240f9420 legacy=0x7fbd240f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd18003b60 0x7fbd18003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd180068c0 0x7fbd18008d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd18009aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24006ae0 con 0x7fbd180095f0 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1800a380 con 0x7fbd180068c0 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.300+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1800a580 con 0x7fbd18003b60 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd329c9640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd18009aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd319c7640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd180068c0 0x7fbd18008d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd329c9640 1 -- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd18003b60 msgr2=0x7fbd18003f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd329c9640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd18003b60 0x7fbd18003f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd329c9640 1 -- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd180068c0 msgr2=0x7fbd18008d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd329c9640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd180068c0 0x7fbd18008d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.301+0000 7fbd329c9640 1 -- 172.21.15.135:0/81874965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd24006dc0 con 0x7fbd180095f0 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.304+0000 7fbd329c9640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd18009aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c075190 tx=0x7fbd180037f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a773b908a278af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:24.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.304+0000 7fbd329c9640 1 -- 172.21.15.135:0/81874965 reap_dead start 2024-04-26T22:03:24.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.309+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c063d80 con 0x7fbd180095f0 2024-04-26T22:03:24.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.309+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c0b8da0 con 0x7fbd180095f0 2024-04-26T22:03:24.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.309+0000 7fbd311c6640 1 -- 172.21.15.135:0/81874965 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0c0640 con 0x7fbd180095f0 2024-04-26T22:03:24.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.314+0000 7fbd3442c640 1 -- 172.21.15.135:0/81874965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 msgr2=0x7fbd18009aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.314+0000 7fbd3442c640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd18009aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c075190 tx=0x7fbd180037f0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.318+0000 7fbd3442c640 1 -- 172.21.15.135:0/81874965 shutdown_connections 2024-04-26T22:03:24.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.318+0000 7fbd3442c640 1 --2- 172.21.15.135:0/81874965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd18009aa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.318+0000 7fbd3442c640 1 -- 172.21.15.135:0/81874965 >> 172.21.15.135:0/81874965 conn(0x7fbd24005e90 msgr2=0x7fbd240062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:24.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.322+0000 7fbd3442c640 1 -- 172.21.15.135:0/81874965 shutdown_connections 2024-04-26T22:03:24.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.325+0000 7fbd3442c640 1 -- 172.21.15.135:0/81874965 wait complete. 2024-04-26T22:03:24.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.326+0000 7fbd3442c640 1 Processor -- start 2024-04-26T22:03:24.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.327+0000 7fbd3442c640 1 -- start start 2024-04-26T22:03:24.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.328+0000 7fbd3442c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd2402c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.328+0000 7fbd3442c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd24182120 0x7fbd241825a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.328+0000 7fbd321c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd2402c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.328+0000 7fbd321c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd2402c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45674/0 (socket says 172.21.15.135:45674) 2024-04-26T22:03:24.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd3442c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd24043e10 0x7fbd24044290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd3442c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240f7f30 con 0x7fbd180095f0 2024-04-26T22:03:24.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd3442c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240f8280 con 0x7fbd24182120 2024-04-26T22:03:24.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd3442c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd24009610 con 0x7fbd24043e10 2024-04-26T22:03:24.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd321c8640 1 -- 172.21.15.135:0/1187401159 learned_addr learned my addr 172.21.15.135:0/1187401159 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:24.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd319c7640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd24182120 0x7fbd241825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd321c8640 1 -- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd24043e10 msgr2=0x7fbd24044290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd321c8640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd24043e10 0x7fbd24044290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd321c8640 1 -- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd24182120 msgr2=0x7fbd241825a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd321c8640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd24182120 0x7fbd241825a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.329+0000 7fbd321c8640 1 -- 172.21.15.135:0/1187401159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2418e330 con 0x7fbd180095f0 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.330+0000 7fbd321c8640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd2402c430 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800b4c0 tx=0x7fbd2800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.330+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28015020 con 0x7fbd180095f0 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.330+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28011830 con 0x7fbd180095f0 2024-04-26T22:03:24.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.330+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28015020 con 0x7fbd180095f0 2024-04-26T22:03:24.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.330+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2418e610 con 0x7fbd180095f0 2024-04-26T22:03:24.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.331+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2418ebd0 con 0x7fbd180095f0 2024-04-26T22:03:24.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.333+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd28018020 con 0x7fbd180095f0 2024-04-26T22:03:24.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.334+0000 7fbd22ffd640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbd1006cd20 0x7fbd1006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.334+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7fbd28011630 con 0x7fbd180095f0 2024-04-26T22:03:24.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.334+0000 7fbd319c7640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbd1006cd20 0x7fbd1006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.335+0000 7fbd319c7640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbd1006cd20 0x7fbd1006f1e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c004910 tx=0x7fbd1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:24.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.338+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd24046b60 con 0x7fbd180095f0 2024-04-26T22:03:24.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.349+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd24046b60 con 0x7fbd180095f0 2024-04-26T22:03:24.528 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:24.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.832+0000 7fb644904640 1 Processor -- start 2024-04-26T22:03:24.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.832+0000 7fb644904640 1 -- start start 2024-04-26T22:03:24.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.834+0000 7fb644904640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb6400abe20 0x7fb6400c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.834+0000 7fb644904640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb640167940 0x7fb6400ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.834+0000 7fb644904640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64006f3c0 con 0x7fb6400ab8c0 2024-04-26T22:03:24.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.834+0000 7fb644904640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb64006f6a0 con 0x7fb640167940 2024-04-26T22:03:24.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.834+0000 7fb644904640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb64006f980 con 0x7fb6400abe20 2024-04-26T22:03:24.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.835+0000 7fb63dd9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb640167940 0x7fb6400ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.835+0000 7fb63dd9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb640167940 0x7fb6400ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45680/0 (socket says 172.21.15.135:45680) 2024-04-26T22:03:24.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.835+0000 7fb63dd9b640 1 -- 172.21.15.135:0/3280914756 learned_addr learned my addr 172.21.15.135:0/3280914756 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:24.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.835+0000 7fb63e59c640 1 --2- 172.21.15.135:0/3280914756 >> v2:172.21.15.139:3300/0 conn(0x7fb6400abe20 0x7fb6400c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.837+0000 7fb63d59a640 1 -- 172.21.15.135:0/3280914756 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208043905 0 0) 0x7fb64006f3c0 con 0x7fb6400ab8c0 2024-04-26T22:03:24.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.837+0000 7fb63dd9b640 1 -- 172.21.15.135:0/3280914756 >> v2:172.21.15.139:3300/0 conn(0x7fb6400abe20 msgr2=0x7fb6400c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:24.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.837+0000 7fb63dd9b640 1 --2- 172.21.15.135:0/3280914756 >> v2:172.21.15.139:3300/0 conn(0x7fb6400abe20 0x7fb6400c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.837+0000 7fb63dd9b640 1 -- 172.21.15.135:0/3280914756 >> v1:172.21.15.135:6789/0 conn(0x7fb6400ab8c0 legacy=0x7fb6400718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.837+0000 7fb63dd9b640 1 -- 172.21.15.135:0/3280914756 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64006fc60 con 0x7fb640167940 2024-04-26T22:03:24.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.847+0000 7fb63dd9b640 1 --2- 172.21.15.135:0/3280914756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb640167940 0x7fb6400ab380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb640068970 tx=0x7fb63002a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5917709d3f688803 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:24.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.848+0000 7fb63d59a640 1 -- 172.21.15.135:0/3280914756 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630037070 con 0x7fb640167940 2024-04-26T22:03:24.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.848+0000 7fb63d59a640 1 -- 172.21.15.135:0/3280914756 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb630032640 con 0x7fb640167940 2024-04-26T22:03:24.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.848+0000 7fb63d59a640 1 -- 172.21.15.135:0/3280914756 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63002ae60 con 0x7fb640167940 2024-04-26T22:03:24.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.850+0000 7fb644904640 1 -- 172.21.15.135:0/3280914756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb640167940 msgr2=0x7fb6400ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.850+0000 7fb644904640 1 --2- 172.21.15.135:0/3280914756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb640167940 0x7fb6400ab380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb640068970 tx=0x7fb63002a310 comp rx=0 tx=0).stop 2024-04-26T22:03:24.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.850+0000 7fb644904640 1 -- 172.21.15.135:0/3280914756 shutdown_connections 2024-04-26T22:03:24.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.850+0000 7fb644904640 1 --2- 172.21.15.135:0/3280914756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb640167940 0x7fb6400ab380 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.850+0000 7fb644904640 1 --2- 172.21.15.135:0/3280914756 >> v2:172.21.15.139:3300/0 conn(0x7fb6400abe20 0x7fb6400c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.850+0000 7fb644904640 1 -- 172.21.15.135:0/3280914756 >> 172.21.15.135:0/3280914756 conn(0x7fb6400c82d0 msgr2=0x7fb6400c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:24.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.851+0000 7fb644904640 1 -- 172.21.15.135:0/3280914756 shutdown_connections 2024-04-26T22:03:24.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.853+0000 7fb644904640 1 -- 172.21.15.135:0/3280914756 wait complete. 2024-04-26T22:03:24.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.855+0000 7fb644904640 1 Processor -- start 2024-04-26T22:03:24.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.855+0000 7fb644904640 1 -- start start 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb644904640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6400ab8c0 0x7fb6400732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb644904640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6400abe20 0x7fb6400bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb644904640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb640167940 0x7fb6400bff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb644904640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6400ac620 con 0x7fb6400ab8c0 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb644904640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6400ac4a0 con 0x7fb640167940 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb644904640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb640167d60 con 0x7fb6400abe20 2024-04-26T22:03:24.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb63dd9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6400abe20 0x7fb6400bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb63dd9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6400abe20 0x7fb6400bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:45714/0 (socket says 172.21.15.135:45714) 2024-04-26T22:03:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.857+0000 7fb63dd9b640 1 -- 172.21.15.135:0/75168068 learned_addr learned my addr 172.21.15.135:0/75168068 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:24.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.858+0000 7fb63ed9d640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb640167940 0x7fb6400bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.861+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd2402e210 con 0x7fbd180095f0 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.859+0000 7fb63ed9d640 1 -- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6400abe20 msgr2=0x7fb6400bdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.859+0000 7fb63ed9d640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6400abe20 0x7fb6400bdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.859+0000 7fb63ed9d640 1 -- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6400ab8c0 msgr2=0x7fb6400732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.859+0000 7fb63ed9d640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6400ab8c0 0x7fb6400732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.859+0000 7fb63ed9d640 1 -- 172.21.15.135:0/75168068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6400be7e0 con 0x7fb640167940 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.860+0000 7fb63ed9d640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb640167940 0x7fb6400bff50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb63400c3e0 tx=0x7fb63400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:24.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.861+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634017070 con 0x7fb640167940 2024-04-26T22:03:24.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.861+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6400beac0 con 0x7fb640167940 2024-04-26T22:03:24.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.861+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6400bfa30 con 0x7fb640167940 2024-04-26T22:03:24.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.861+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6340035f0 con 0x7fb640167940 2024-04-26T22:03:24.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.861+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6340039e0 con 0x7fb640167940 2024-04-26T22:03:24.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.863+0000 7fbd22ffd640 1 -- 172.21.15.135:0/1187401159 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6503 (secure 0 0 0) 0x7fbd2402e210 con 0x7fbd180095f0 2024-04-26T22:03:24.863 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:24.864 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:24.864 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 28 2024-04-26T22:03:24.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.865+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb63401a070 con 0x7fb640167940 2024-04-26T22:03:24.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.866+0000 7fb62effd640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb62006c9c0 0x7fb62006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.867+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbd1006cd20 msgr2=0x7fbd1006f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.867+0000 7fbd3442c640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbd1006cd20 0x7fbd1006f1e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c004910 tx=0x7fbd1c009040 comp rx=0 tx=0).stop 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.867+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 msgr2=0x7fbd2402c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.867+0000 7fbd3442c640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd2402c430 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800b4c0 tx=0x7fbd2800b980 comp rx=0 tx=0).stop 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.867+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb624005290 con 0x7fb640167940 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fb63e59c640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb62006c9c0 0x7fb62006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:24.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7fb63408b320 con 0x7fb640167940 2024-04-26T22:03:24.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 shutdown_connections 2024-04-26T22:03:24.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fbd3442c640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fbd1006cd20 0x7fbd1006f1e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fbd3442c640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd24043e10 0x7fbd24044290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fbd3442c640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd24182120 0x7fbd241825a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fbd3442c640 1 --2- 172.21.15.135:0/1187401159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd180095f0 0x7fbd2402c430 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:24.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 >> 172.21.15.135:0/1187401159 conn(0x7fbd24005e90 msgr2=0x7fbd240f6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:24.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.868+0000 7fb63e59c640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb62006c9c0 0x7fb62006ee80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb6380070b0 tx=0x7fb638007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:24.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.869+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 shutdown_connections 2024-04-26T22:03:24.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.869+0000 7fbd3442c640 1 -- 172.21.15.135:0/1187401159 wait complete. 2024-04-26T22:03:24.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:24.876+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb634058c20 con 0x7fb640167940 2024-04-26T22:03:24.896 INFO:tasks.mds_thrash.fs.[a]:mds.d reported in up:standby state 2024-04-26T22:03:24.896 INFO:tasks.mds_thrash.fs.[a]:waiting for 56 secs before thrashing 2024-04-26T22:03:25.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.391+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb624005580 con 0x7fb640167940 2024-04-26T22:03:25.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.393+0000 7fb62effd640 1 -- 172.21.15.135:0/75168068 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6503 (secure 0 0 0) 0x7fb63405cf20 con 0x7fb640167940 2024-04-26T22:03:25.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:25.393 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:01:40.794279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.139:6833/3183513686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3183513686},{"type":"v1","addr":"172.21.15.139:6833","nonce":3183513686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4241,"qdb_cluster":[4241]},"id":2}]} 2024-04-26T22:03:25.393 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 28 2024-04-26T22:03:25.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.396+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb62006c9c0 msgr2=0x7fb62006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb62006c9c0 0x7fb62006ee80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb6380070b0 tx=0x7fb638007040 comp rx=0 tx=0).stop 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb640167940 msgr2=0x7fb6400bff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb640167940 0x7fb6400bff50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb63400c3e0 tx=0x7fb63400c8a0 comp rx=0 tx=0).stop 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 shutdown_connections 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb62006c9c0 0x7fb62006ee80 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb6380070b0 tx=0x7fb638007040 comp rx=0 tx=0).stop 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb640167940 0x7fb6400bff50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6400abe20 0x7fb6400bdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 --2- 172.21.15.135:0/75168068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6400ab8c0 0x7fb6400732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 >> 172.21.15.135:0/75168068 conn(0x7fb6400c82d0 msgr2=0x7fb6400c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 shutdown_connections 2024-04-26T22:03:25.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:25.397+0000 7fb644904640 1 -- 172.21.15.135:0/75168068 wait complete. 2024-04-26T22:03:25.419 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:03:27.420 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:27.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.636+0000 7f645f2bb640 1 Processor -- start 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.637+0000 7f645f2bb640 1 -- start start 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.638+0000 7f645f2bb640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64580c5880 0x7f645806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.638+0000 7f645f2bb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f645806e720 0x7f64581657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.638+0000 7f645f2bb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64580ac1c0 con 0x7f6458073eb0 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.638+0000 7f645f2bb640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f64580ab620 con 0x7f64580c5880 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.638+0000 7f645f2bb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f645806d6d0 con 0x7f645806e720 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.639+0000 7f645c856640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64580c5880 0x7f645806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.639+0000 7f645c856640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64580c5880 0x7f645806e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45700/0 (socket says 172.21.15.135:45700) 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.639+0000 7f645c856640 1 -- 172.21.15.135:0/655437110 learned_addr learned my addr 172.21.15.135:0/655437110 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.640+0000 7f645c856640 1 -- 172.21.15.135:0/655437110 >> v2:172.21.15.139:3300/0 conn(0x7f645806e720 msgr2=0x7f64581657d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.640+0000 7f645c856640 1 --2- 172.21.15.135:0/655437110 >> v2:172.21.15.139:3300/0 conn(0x7f645806e720 0x7f64581657d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.640+0000 7f645c856640 1 -- 172.21.15.135:0/655437110 >> v1:172.21.15.139:6789/0 conn(0x7f6458073eb0 legacy=0x7f64580abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:27.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.640+0000 7f645c856640 1 -- 172.21.15.135:0/655437110 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645806d9b0 con 0x7f64580c5880 2024-04-26T22:03:27.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.644+0000 7f645c856640 1 --2- 172.21.15.135:0/655437110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64580c5880 0x7f645806e1e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f64580714c0 tx=0x7f645002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ea660fbfff28d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:27.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.646+0000 7f644ffff640 1 -- 172.21.15.135:0/655437110 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64500090d0 con 0x7f64580c5880 2024-04-26T22:03:27.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.646+0000 7f644ffff640 1 -- 172.21.15.135:0/655437110 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64500313f0 con 0x7f64580c5880 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.647+0000 7f645f2bb640 1 -- 172.21.15.135:0/655437110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64580c5880 msgr2=0x7f645806e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.647+0000 7f645f2bb640 1 --2- 172.21.15.135:0/655437110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64580c5880 0x7f645806e1e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f64580714c0 tx=0x7f645002a8e0 comp rx=0 tx=0).stop 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.649+0000 7f645f2bb640 1 -- 172.21.15.135:0/655437110 shutdown_connections 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.649+0000 7f645f2bb640 1 --2- 172.21.15.135:0/655437110 >> v2:172.21.15.139:3300/0 conn(0x7f645806e720 0x7f64581657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.649+0000 7f645f2bb640 1 --2- 172.21.15.135:0/655437110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64580c5880 0x7f645806e1e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.649+0000 7f645f2bb640 1 -- 172.21.15.135:0/655437110 >> 172.21.15.135:0/655437110 conn(0x7f64580c4d60 msgr2=0x7f64580c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:27.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.649+0000 7f645f2bb640 1 -- 172.21.15.135:0/655437110 shutdown_connections 2024-04-26T22:03:27.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.649+0000 7f645f2bb640 1 -- 172.21.15.135:0/655437110 wait complete. 2024-04-26T22:03:27.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.651+0000 7f645f2bb640 1 Processor -- start 2024-04-26T22:03:27.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.651+0000 7f645f2bb640 1 -- start start 2024-04-26T22:03:27.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.652+0000 7f645f2bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645806e720 0x7f645814cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.652+0000 7f645f2bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 0x7f64580bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.652+0000 7f645f2bb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645815bdd0 0x7f64580c0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.652+0000 7f645f2bb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64580ac6b0 con 0x7f645815bdd0 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.652+0000 7f645f2bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64580c4870 con 0x7f645806e720 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.652+0000 7f645f2bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64580ac830 con 0x7f6458073eb0 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 0x7f64580bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 0x7f64580bfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:45718/0 (socket says 172.21.15.135:45718) 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 -- 172.21.15.135:0/2413939595 learned_addr learned my addr 172.21.15.135:0/2413939595 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645d858640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645815bdd0 0x7f64580c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 -- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645806e720 msgr2=0x7f645814cb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645806e720 0x7f645814cb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 -- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645815bdd0 msgr2=0x7f64580c0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645815bdd0 0x7f64580c0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:27.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.653+0000 7f645c856640 1 -- 172.21.15.135:0/2413939595 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64580c2700 con 0x7f6458073eb0 2024-04-26T22:03:27.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.654+0000 7f645d858640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645815bdd0 0x7f64580c0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:27.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.654+0000 7f645c856640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 0x7f64580bfd10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f645000b290 tx=0x7f6450004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:27.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.655+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64500090d0 con 0x7f6458073eb0 2024-04-26T22:03:27.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.655+0000 7f645f2bb640 1 -- 172.21.15.135:0/2413939595 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64580bf830 con 0x7f6458073eb0 2024-04-26T22:03:27.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.655+0000 7f645f2bb640 1 -- 172.21.15.135:0/2413939595 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64580be5d0 con 0x7f6458073eb0 2024-04-26T22:03:27.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.657+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64500313f0 con 0x7f6458073eb0 2024-04-26T22:03:27.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.657+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64500046d0 con 0x7f6458073eb0 2024-04-26T22:03:27.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.659+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438005290 con 0x7f6458073eb0 2024-04-26T22:03:27.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.660+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f645003b410 con 0x7f6458073eb0 2024-04-26T22:03:27.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.660+0000 7f644dffb640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f643c06c940 0x7f643c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:27.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.660+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 6924+0+0 (secure 0 0 0) 0x7f64500aca40 con 0x7f6458073eb0 2024-04-26T22:03:27.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.668+0000 7f645d057640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f643c06c940 0x7f643c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:27.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.668+0000 7f645d057640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f643c06c940 0x7f643c06ee00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6448001540 tx=0x7f6448007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:27.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:27.674+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f645003f070 con 0x7f6458073eb0 2024-04-26T22:03:28.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.234+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6438005580 con 0x7f6458073eb0 2024-04-26T22:03:28.240 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:28.240 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:27.791651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:03:28.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.237+0000 7f644dffb640 1 -- 172.21.15.135:0/2413939595 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5667 (secure 0 0 0) 0x7f645007a3d0 con 0x7f6458073eb0 2024-04-26T22:03:28.240 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 29 2024-04-26T22:03:28.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.242+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f643c06c940 msgr2=0x7f643c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:28.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.242+0000 7f64437fe640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f643c06c940 0x7f643c06ee00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6448001540 tx=0x7f6448007990 comp rx=0 tx=0).stop 2024-04-26T22:03:28.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.242+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 msgr2=0x7f64580bfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:28.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.242+0000 7f64437fe640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 0x7f64580bfd10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f645000b290 tx=0x7f6450004a40 comp rx=0 tx=0).stop 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 shutdown_connections 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f643c06c940 0x7f643c06ee00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645815bdd0 0x7f64580c0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6458073eb0 0x7f64580bfd10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 --2- 172.21.15.135:0/2413939595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645806e720 0x7f645814cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 >> 172.21.15.135:0/2413939595 conn(0x7f64580c4d60 msgr2=0x7f645814a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:28.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 shutdown_connections 2024-04-26T22:03:28.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.245+0000 7f64437fe640 1 -- 172.21.15.135:0/2413939595 wait complete. 2024-04-26T22:03:28.271 INFO:tasks.mds_thrash.fs.[b]:mds.b down, removed from mdsmap 2024-04-26T22:03:28.271 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:03:28.271 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:28.483 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.484+0000 7f9dace7b640 1 Processor -- start 2024-04-26T22:03:28.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.486+0000 7f9dace7b640 1 -- start start 2024-04-26T22:03:28.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9dace7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9da806d590 0x7f9da815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:28.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9dace7b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9da8073be0 0x7f9da8074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:28.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9dace7b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da815e7d0 con 0x7f9da80705c0 2024-04-26T22:03:28.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9dace7b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9da80701f0 con 0x7f9da8073be0 2024-04-26T22:03:28.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9dace7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9da815f0c0 con 0x7f9da806d590 2024-04-26T22:03:28.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9da5d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9da80705c0 0x7f9da8158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:55464/0 (socket says 172.21.15.135:55464) 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.488+0000 7f9da5d9b640 1 -- 172.21.15.135:0/1569754932 learned_addr learned my addr 172.21.15.135:0/1569754932 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.489+0000 7f9da6d9d640 1 --2- 172.21.15.135:0/1569754932 >> v2:172.21.15.135:3300/0 conn(0x7f9da8073be0 0x7f9da8074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.489+0000 7f9da659c640 1 --2- 172.21.15.135:0/1569754932 >> v2:172.21.15.139:3300/0 conn(0x7f9da806d590 0x7f9da815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.489+0000 7f9da659c640 1 -- 172.21.15.135:0/1569754932 >> v2:172.21.15.135:3300/0 conn(0x7f9da8073be0 msgr2=0x7f9da8074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.489+0000 7f9da659c640 1 --2- 172.21.15.135:0/1569754932 >> v2:172.21.15.135:3300/0 conn(0x7f9da8073be0 0x7f9da8074030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.489+0000 7f9da659c640 1 -- 172.21.15.135:0/1569754932 >> v1:172.21.15.139:6789/0 conn(0x7f9da80705c0 legacy=0x7f9da8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:28.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.489+0000 7f9da659c640 1 -- 172.21.15.135:0/1569754932 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da8070b40 con 0x7f9da806d590 2024-04-26T22:03:28.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.491+0000 7f9da659c640 1 --2- 172.21.15.135:0/1569754932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da815bc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9da8070370 tx=0x7f9d9c02a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb08abb68062d34b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:28.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.492+0000 7f9da559a640 1 -- 172.21.15.135:0/1569754932 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c037070 con 0x7f9da806d590 2024-04-26T22:03:28.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.494+0000 7f9da559a640 1 -- 172.21.15.135:0/1569754932 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c02ad40 con 0x7f9da806d590 2024-04-26T22:03:28.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.495+0000 7f9dace7b640 1 -- 172.21.15.135:0/1569754932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 msgr2=0x7f9da815bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:28.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.495+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1569754932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da815bc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9da8070370 tx=0x7f9d9c02a0b0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.496+0000 7f9dace7b640 1 -- 172.21.15.135:0/1569754932 shutdown_connections 2024-04-26T22:03:28.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.496+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1569754932 >> v2:172.21.15.135:3300/0 conn(0x7f9da8073be0 0x7f9da8074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.496+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1569754932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da815bc60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.496+0000 7f9dace7b640 1 -- 172.21.15.135:0/1569754932 >> 172.21.15.135:0/1569754932 conn(0x7f9da80c5ce0 msgr2=0x7f9da80c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.499+0000 7f9dace7b640 1 -- 172.21.15.135:0/1569754932 shutdown_connections 2024-04-26T22:03:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.500+0000 7f9dace7b640 1 -- 172.21.15.135:0/1569754932 wait complete. 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.502+0000 7f9dace7b640 1 Processor -- start 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.504+0000 7f9dace7b640 1 -- start start 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9dace7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9dace7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9da80705c0 0x7f9da80ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9dace7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 0x7f9da80adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9dace7b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8074570 con 0x7f9da80705c0 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9dace7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9da815ed50 con 0x7f9da806d590 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9dace7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9da815f550 con 0x7f9da8073be0 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9da6d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 0x7f9da80adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9da6d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 0x7f9da80adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:57856/0 (socket says 172.21.15.135:57856) 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9da6d9d640 1 -- 172.21.15.135:0/1824166959 learned_addr learned my addr 172.21.15.135:0/1824166959 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:28.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9da659c640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:28.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.506+0000 7f9da6d9d640 1 -- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 msgr2=0x7f9da814e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:28.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9da6d9d640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da814e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9da6d9d640 1 -- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9da80705c0 msgr2=0x7f9da80ac3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:28.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9da6d9d640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9da80705c0 0x7f9da80ac3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9da6d9d640 1 -- 172.21.15.135:0/1824166959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da80c7020 con 0x7f9da8073be0 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9da6d9d640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 0x7f9da80adf90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800e830 tx=0x7f9d9800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98017070 con 0x7f9da8073be0 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da81f4040 con 0x7f9da8073be0 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.507+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da80ac910 con 0x7f9da8073be0 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.508+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9800c940 con 0x7f9da8073be0 2024-04-26T22:03:28.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.509+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800cd10 con 0x7f9da8073be0 2024-04-26T22:03:28.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.509+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c005290 con 0x7f9da8073be0 2024-04-26T22:03:28.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.511+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d9801a070 con 0x7f9da8073be0 2024-04-26T22:03:28.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.511+0000 7f9d96ffd640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9d8806c8f0 0x7f9d8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:28.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.512+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f9d9808bef0 con 0x7f9da8073be0 2024-04-26T22:03:28.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.512+0000 7f9da659c640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9d8806c8f0 0x7f9d8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:28.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.513+0000 7f9da659c640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9d8806c8f0 0x7f9d8806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c034000 tx=0x7f9d9c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:28.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:28.521+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d980597a0 con 0x7f9da8073be0 2024-04-26T22:03:28.707 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:03:28.711 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:03:29.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.038+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d8c005580 con 0x7f9da8073be0 2024-04-26T22:03:29.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.040+0000 7f9d96ffd640 1 -- 172.21.15.135:0/1824166959 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5667 (secure 0 0 0) 0x7f9d9805daa0 con 0x7f9da8073be0 2024-04-26T22:03:29.041 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:29.041 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:27.791651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:03:29.041 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 29 2024-04-26T22:03:29.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.048+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9d8806c8f0 msgr2=0x7f9d8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:29.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.048+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9d8806c8f0 0x7f9d8806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c034000 tx=0x7f9d9c02e000 comp rx=0 tx=0).stop 2024-04-26T22:03:29.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.048+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 msgr2=0x7f9da80adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:29.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.048+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 0x7f9da80adf90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800e830 tx=0x7f9d9800ecf0 comp rx=0 tx=0).stop 2024-04-26T22:03:29.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.051+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 shutdown_connections 2024-04-26T22:03:29.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.051+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9d8806c8f0 0x7f9d8806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:29.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.051+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9da8073be0 0x7f9da80adf90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:29.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.051+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9da80705c0 0x7f9da80ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:29.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.051+0000 7f9dace7b640 1 --2- 172.21.15.135:0/1824166959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9da806d590 0x7f9da814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:29.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.051+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 >> 172.21.15.135:0/1824166959 conn(0x7f9da80c5ce0 msgr2=0x7f9da80680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:29.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.053+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 shutdown_connections 2024-04-26T22:03:29.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:29.053+0000 7f9dace7b640 1 -- 172.21.15.135:0/1824166959 wait complete. 2024-04-26T22:03:29.076 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5268": { "addr": "172.21.15.135:6835/2910412510", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 2910412510, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 2910412510, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5268, "incarnation": 24, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:03:15.024868+0000", "qdb_cluster": [ 5268 ], "qdb_leader": 5268, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5268 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5086": { "addr": "172.21.15.135:6837/3652200818", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 3652200818, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 3652200818, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 29, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:03:27.791651+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:03:31.078 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:31.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.259+0000 7fb0f1a6f640 1 Processor -- start 2024-04-26T22:03:31.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.259+0000 7fb0f1a6f640 1 -- start start 2024-04-26T22:03:31.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.260+0000 7fb0f1a6f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb0ec0c8ff0 0x7fb0ec073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.260+0000 7fb0f1a6f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ec0c7140 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.261+0000 7fb0f1a6f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ec0c7420 con 0x7fb0ec153d00 2024-04-26T22:03:31.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.261+0000 7fb0f1a6f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ec0ca5c0 con 0x7fb0ec0c8ff0 2024-04-26T22:03:31.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.261+0000 7fb0f126e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb0ec0cadc0 0x7fb0ec0cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:34466/0 (socket says 172.21.15.135:34466) 2024-04-26T22:03:31.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.261+0000 7fb0f126e640 1 -- 172.21.15.135:0/1856572643 learned_addr learned my addr 172.21.15.135:0/1856572643 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:31.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.262+0000 7fb0f0a6d640 1 --2- 172.21.15.135:0/1856572643 >> v2:172.21.15.135:3300/0 conn(0x7fb0ec0c8ff0 0x7fb0ec073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154413481 0 0) 0x7fb0ec0c7140 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0dc003520 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437690854 0 0) 0x7fb0ec0c7420 con 0x7fb0ec153d00 2024-04-26T22:03:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0ec0c7140 con 0x7fb0ec153d00 2024-04-26T22:03:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0f0a6d640 1 -- 172.21.15.135:0/1856572643 >> v1:172.21.15.139:6789/0 conn(0x7fb0ec153d00 legacy=0x7fb0ec154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0f0a6d640 1 -- 172.21.15.135:0/1856572643 >> v1:172.21.15.135:6789/0 conn(0x7fb0ec0cadc0 legacy=0x7fb0ec0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.263+0000 7fb0f0a6d640 1 -- 172.21.15.135:0/1856572643 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec0ca8a0 con 0x7fb0ec0c8ff0 2024-04-26T22:03:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.264+0000 7fb0f0a6d640 1 --2- 172.21.15.135:0/1856572643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0c8ff0 0x7fb0ec073340 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80097d0 tx=0x7fb0d802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a606335daa166904 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.265+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8037070 con 0x7fb0ec0c8ff0 2024-04-26T22:03:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.265+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d8032440 con 0x7fb0ec0c8ff0 2024-04-26T22:03:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.265+0000 7fb0eb7fe640 1 -- 172.21.15.135:0/1856572643 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d802ae60 con 0x7fb0ec0c8ff0 2024-04-26T22:03:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.267+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/1856572643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0c8ff0 msgr2=0x7fb0ec073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.267+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/1856572643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0c8ff0 0x7fb0ec073340 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80097d0 tx=0x7fb0d802a150 comp rx=0 tx=0).stop 2024-04-26T22:03:31.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.268+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/1856572643 shutdown_connections 2024-04-26T22:03:31.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.268+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/1856572643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0c8ff0 0x7fb0ec073340 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.268+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/1856572643 >> 172.21.15.135:0/1856572643 conn(0x7fb0ec0adea0 msgr2=0x7fb0ec0ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.268+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/1856572643 shutdown_connections 2024-04-26T22:03:31.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.268+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/1856572643 wait complete. 2024-04-26T22:03:31.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.270+0000 7fb0f1a6f640 1 Processor -- start 2024-04-26T22:03:31.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.270+0000 7fb0f1a6f640 1 -- start start 2024-04-26T22:03:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.272+0000 7fb0f1a6f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0ec0c8ff0 0x7fb0ec0c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.272+0000 7fb0f1a6f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 0x7fb0ec147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.273+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 0x7fb0ec147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.273+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 0x7fb0ec147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47196/0 (socket says 172.21.15.135:47196) 2024-04-26T22:03:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.273+0000 7fb0ebfff640 1 -- 172.21.15.135:0/723162607 learned_addr learned my addr 172.21.15.135:0/723162607 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.273+0000 7fb0f0a6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0ec0c8ff0 0x7fb0ec0c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.273+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec153d00 0x7fb0ec0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.274+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec073a00 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.274+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec073880 con 0x7fb0ec153d00 2024-04-26T22:03:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.274+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ec0cb940 con 0x7fb0ec0c8ff0 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.274+0000 7fb0ebfff640 1 -- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0ec0c8ff0 msgr2=0x7fb0ec0c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.275+0000 7fb0ebfff640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0ec0c8ff0 0x7fb0ec0c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.275+0000 7fb0ebfff640 1 -- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec153d00 msgr2=0x7fb0ec0c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.275+0000 7fb0ebfff640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec153d00 0x7fb0ec0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.275+0000 7fb0ebfff640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec0c3980 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.275+0000 7fb0f0a6d640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0ec0c8ff0 0x7fb0ec0c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.275+0000 7fb0ebfff640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 0x7fb0ec147950 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c7b0 tx=0x7fb0e000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.276+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00050d0 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.276+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e0005270 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.276+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000a640 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.276+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec0af000 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.276+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec0c1750 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.278+0000 7fb0d6ffd640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c4005290 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.280+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0e000a800 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.280+0000 7fb0e97fa640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0d006c9c0 0x7fb0d006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:31.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.281+0000 7fb0f0a6d640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0d006c9c0 0x7fb0d006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:31.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.281+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7fb0e008bbb0 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.281+0000 7fb0f0a6d640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0d006c9c0 0x7fb0d006ee80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8034000 tx=0x7fb0d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:31.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.288+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0e00102e0 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.774+0000 7fb0d6ffd640 1 -- 172.21.15.135:0/723162607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0c4005580 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.775+0000 7fb0e97fa640 1 -- 172.21.15.135:0/723162607 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5667 (secure 0 0 0) 0x7fb0e0059b30 con 0x7fb0ec0cadc0 2024-04-26T22:03:31.775 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:31.775 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:27.791651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:03:31.776 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 29 2024-04-26T22:03:31.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.780+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0d006c9c0 msgr2=0x7fb0d006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.780+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0d006c9c0 0x7fb0d006ee80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8034000 tx=0x7fb0d802e000 comp rx=0 tx=0).stop 2024-04-26T22:03:31.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.780+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 msgr2=0x7fb0ec147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:31.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.780+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 0x7fb0ec147950 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c7b0 tx=0x7fb0e000cc70 comp rx=0 tx=0).stop 2024-04-26T22:03:31.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 shutdown_connections 2024-04-26T22:03:31.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0d006c9c0 0x7fb0d006ee80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec153d00 0x7fb0ec0c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0ec0cadc0 0x7fb0ec147950 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 --2- 172.21.15.135:0/723162607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0ec0c8ff0 0x7fb0ec0c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:31.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 >> 172.21.15.135:0/723162607 conn(0x7fb0ec0adea0 msgr2=0x7fb0ec0ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:31.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.782+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 shutdown_connections 2024-04-26T22:03:31.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:31.783+0000 7fb0f1a6f640 1 -- 172.21.15.135:0/723162607 wait complete. 2024-04-26T22:03:31.805 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:03:33.805 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:34.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.008+0000 7f7cd68f9640 1 Processor -- start 2024-04-26T22:03:34.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.009+0000 7f7cd68f9640 1 -- start start 2024-04-26T22:03:34.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.010+0000 7f7ccffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7cd00ae3d0 0x7f7cd00c5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:34476/0 (socket says 172.21.15.135:34476) 2024-04-26T22:03:34.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.010+0000 7f7ccffff640 1 -- 172.21.15.135:0/1047036094 learned_addr learned my addr 172.21.15.135:0/1047036094 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:34.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.010+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00a80a0 0x7f7cd00c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:34.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.011+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/1047036094 >> v2:172.21.15.139:3300/0 conn(0x7f7cd00ae7b0 0x7f7cd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:34.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.011+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd0070750 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.011+0000 7f7cd4e96640 1 --2- 172.21.15.135:0/1047036094 >> v2:172.21.15.139:3300/0 conn(0x7f7cd00ae7b0 0x7f7cd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:34.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.011+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd01565b0 con 0x7f7cd00ae7b0 2024-04-26T22:03:34.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.011+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd0156800 con 0x7f7cd00a80a0 2024-04-26T22:03:34.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7ccf7fe640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00a80a0 0x7f7cd00c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:34.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7cceffd640 1 -- 172.21.15.135:0/1047036094 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1843526248 0 0) 0x7f7cd0070750 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7cceffd640 1 -- 172.21.15.135:0/1047036094 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cb8003520 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7cd4e96640 1 -- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00a80a0 msgr2=0x7f7cd00c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:34.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7cd4e96640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00a80a0 0x7f7cd00c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7cd4e96640 1 -- 172.21.15.135:0/1047036094 >> v1:172.21.15.135:6789/0 conn(0x7f7cd00ae3d0 legacy=0x7f7cd00c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:34.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.013+0000 7f7cd4e96640 1 -- 172.21.15.135:0/1047036094 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd0156ae0 con 0x7f7cd00ae7b0 2024-04-26T22:03:34.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.014+0000 7f7cd4e96640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae7b0 0x7f7cd006ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009c80 tx=0x7f7cc802c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed55b2e3757a5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:34.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.015+0000 7f7cceffd640 1 -- 172.21.15.135:0/1047036094 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8032760 con 0x7f7cd00ae7b0 2024-04-26T22:03:34.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.015+0000 7f7cceffd640 1 -- 172.21.15.135:0/1047036094 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8038070 con 0x7f7cd00ae7b0 2024-04-26T22:03:34.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.015+0000 7f7cceffd640 1 -- 172.21.15.135:0/1047036094 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8032b20 con 0x7f7cd00ae7b0 2024-04-26T22:03:34.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.016+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae7b0 msgr2=0x7f7cd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:34.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.016+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae7b0 0x7f7cd006ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009c80 tx=0x7f7cc802c860 comp rx=0 tx=0).stop 2024-04-26T22:03:34.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.017+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 shutdown_connections 2024-04-26T22:03:34.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.017+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae7b0 0x7f7cd006ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.017+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/1047036094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00a80a0 0x7f7cd00c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.017+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 >> 172.21.15.135:0/1047036094 conn(0x7f7cd00c89f0 msgr2=0x7f7cd00c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:34.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.017+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 shutdown_connections 2024-04-26T22:03:34.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.017+0000 7f7cd68f9640 1 -- 172.21.15.135:0/1047036094 wait complete. 2024-04-26T22:03:34.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.019+0000 7f7cd68f9640 1 Processor -- start 2024-04-26T22:03:34.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.019+0000 7f7cd68f9640 1 -- start start 2024-04-26T22:03:34.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.020+0000 7f7cd68f9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00a80a0 0x7f7cd01eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7cd68f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae3d0 0x7f7cd00a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7cd68f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00ae7b0 0x7f7cd00a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00a80a0 0x7f7cd01eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00a80a0 0x7f7cd01eb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47202/0 (socket says 172.21.15.135:47202) 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccffff640 1 -- 172.21.15.135:0/3437360854 learned_addr learned my addr 172.21.15.135:0/3437360854 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00c5b90 con 0x7f7cd00a80a0 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd01570a0 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00c5d10 con 0x7f7cd00ae7b0 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccf7fe640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae3d0 0x7f7cd00a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:34.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7cd4e96640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00ae7b0 0x7f7cd00a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:34.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccf7fe640 1 -- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00ae7b0 msgr2=0x7f7cd00a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:34.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccf7fe640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00ae7b0 0x7f7cd00a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccf7fe640 1 -- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00a80a0 msgr2=0x7f7cd01eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:34.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccf7fe640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00a80a0 0x7f7cd01eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.021+0000 7f7ccf7fe640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd00a92d0 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.023+0000 7f7ccf7fe640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae3d0 0x7f7cd00a8850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400e880 tx=0x7f7cc400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:34.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.023+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4017070 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.023+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc400c940 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.023+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400cd30 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.023+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd01e9ec0 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.023+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd0071dd0 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.025+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd00765e0 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.026+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cc401a070 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.027+0000 7f7cccff9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7cbc06c9f0 0x7f7cbc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:34.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.027+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f7cc408be00 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.028+0000 7f7ccffff640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7cbc06c9f0 0x7f7cbc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:34.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.031+0000 7f7ccffff640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7cbc06c9f0 0x7f7cbc06eeb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005880 tx=0x7f7cc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:34.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.031+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7cc4059730 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.518+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cd00507a0 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.519+0000 7f7cccff9640 1 -- 172.21.15.135:0/3437360854 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5667 (secure 0 0 0) 0x7f7cc405da30 con 0x7f7cd00ae3d0 2024-04-26T22:03:34.519 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:34.519 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:27.791651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:03:34.519 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 29 2024-04-26T22:03:34.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.524+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7cbc06c9f0 msgr2=0x7f7cbc06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:34.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.524+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7cbc06c9f0 0x7f7cbc06eeb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005880 tx=0x7f7cc0009040 comp rx=0 tx=0).stop 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.524+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae3d0 msgr2=0x7f7cd00a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.524+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae3d0 0x7f7cd00a8850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400e880 tx=0x7f7cc400ed40 comp rx=0 tx=0).stop 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.524+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 shutdown_connections 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.525+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7cbc06c9f0 0x7f7cbc06eeb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.525+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd00ae7b0 0x7f7cd00a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.525+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd00ae3d0 0x7f7cd00a8850 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.525+0000 7f7cd68f9640 1 --2- 172.21.15.135:0/3437360854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd00a80a0 0x7f7cd01eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:34.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.525+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 >> 172.21.15.135:0/3437360854 conn(0x7f7cd00c89f0 msgr2=0x7f7cd00ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:34.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.526+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 shutdown_connections 2024-04-26T22:03:34.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:34.529+0000 7f7cd68f9640 1 -- 172.21.15.135:0/3437360854 wait complete. 2024-04-26T22:03:34.550 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:03:36.551 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:36.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.740+0000 7f9a2d631640 1 Processor -- start 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.740+0000 7f9a2d631640 1 -- start start 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a2d631640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9a280c47d0 0x7f9a280adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a2d631640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a2806e5e0 0x7f9a280c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a2d631640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a280ae050 con 0x7f9a2806fd20 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a2d631640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9a280ad680 con 0x7f9a280c47d0 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a2d631640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a280c4bb0 con 0x7f9a2806e5e0 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a277fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9a2806fd20 0x7f9a28157400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:34490/0 (socket says 172.21.15.135:34490) 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.742+0000 7f9a277fe640 1 -- 172.21.15.135:0/1380890998 learned_addr learned my addr 172.21.15.135:0/1380890998 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:36.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.743+0000 7f9a27fff640 1 --2- 172.21.15.135:0/1380890998 >> v2:172.21.15.139:3300/0 conn(0x7f9a280c47d0 0x7f9a280adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.743+0000 7f9a2ce30640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a2806e5e0 0x7f9a280c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.743+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271838007 0 0) 0x7f9a280ae050 con 0x7f9a2806fd20 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.743+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a10003520 con 0x7f9a2806fd20 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2668707874 0 0) 0x7f9a10003520 con 0x7f9a2806fd20 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a2806e5e0 msgr2=0x7f9a280c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a2806e5e0 0x7f9a280c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:36.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 >> v2:172.21.15.139:3300/0 conn(0x7f9a280c47d0 msgr2=0x7f9a280adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 --2- 172.21.15.135:0/1380890998 >> v2:172.21.15.139:3300/0 conn(0x7f9a280c47d0 0x7f9a280adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a280ad050 con 0x7f9a2806fd20 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f9a20001670 con 0x7f9a2806fd20 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 >> v1:172.21.15.135:6789/0 conn(0x7f9a2806fd20 legacy=0x7f9a28157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a100039b0 0x7f9a10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.744+0000 7f9a26ffd640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a10004800 0x7f9a10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.745+0000 7f9a26ffd640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a100051c0 0x7f9a100141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.745+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a280ad680 con 0x7f9a10004800 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.745+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a280c4bb0 con 0x7f9a100039b0 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.745+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a10005a40 con 0x7f9a100051c0 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.745+0000 7f9a27fff640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a100039b0 0x7f9a10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.746+0000 7f9a2ce30640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a2806e5e0 0x7f9a280c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:36.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.746+0000 7f9a277fe640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a10004800 0x7f9a10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.747+0000 7f9a27fff640 1 -- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a100051c0 msgr2=0x7f9a100141d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:36.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.747+0000 7f9a27fff640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a100051c0 0x7f9a100141d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:36.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.747+0000 7f9a27fff640 1 -- 172.21.15.135:0/1380890998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a10004800 msgr2=0x7f9a10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:36.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.747+0000 7f9a27fff640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a10004800 0x7f9a10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:36.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.747+0000 7f9a27fff640 1 -- 172.21.15.135:0/1380890998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a280ad050 con 0x7f9a100039b0 2024-04-26T22:03:36.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.750+0000 7f9a27fff640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a100039b0 0x7f9a10003dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00ba30 tx=0x7f9a10005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77107955d9181aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:36.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.750+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c030070 con 0x7f9a100039b0 2024-04-26T22:03:36.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.750+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c02adb0 con 0x7f9a100039b0 2024-04-26T22:03:36.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.750+0000 7f9a26ffd640 1 -- 172.21.15.135:0/1380890998 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00b600 con 0x7f9a100039b0 2024-04-26T22:03:36.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.751+0000 7f9a2d631640 1 -- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a100039b0 msgr2=0x7f9a10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:36.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.751+0000 7f9a2d631640 1 --2- 172.21.15.135:0/1380890998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a100039b0 0x7f9a10003dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00ba30 tx=0x7f9a10005830 comp rx=0 tx=0).stop 2024-04-26T22:03:36.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.751+0000 7f9a2ce30640 1 -- 172.21.15.135:0/1380890998 reap_dead start 2024-04-26T22:03:36.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.754+0000 7f9a2d631640 1 -- 172.21.15.135:0/1380890998 shutdown_connections 2024-04-26T22:03:36.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.754+0000 7f9a2d631640 1 -- 172.21.15.135:0/1380890998 >> 172.21.15.135:0/1380890998 conn(0x7f9a280c5090 msgr2=0x7f9a280c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:36.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.755+0000 7f9a2d631640 1 -- 172.21.15.135:0/1380890998 shutdown_connections 2024-04-26T22:03:36.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.756+0000 7f9a2d631640 1 -- 172.21.15.135:0/1380890998 wait complete. 2024-04-26T22:03:36.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.757+0000 7f9a2d631640 1 Processor -- start 2024-04-26T22:03:36.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.758+0000 7f9a2d631640 1 -- start start 2024-04-26T22:03:36.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.759+0000 7f9a2d631640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 0x7f9a2814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.759+0000 7f9a2d631640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815a750 0x7f9a281eb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a2d631640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280741a0 0x7f9a28071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a27fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 0x7f9a2814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a27fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 0x7f9a2814d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47228/0 (socket says 172.21.15.135:47228) 2024-04-26T22:03:36.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a2d631640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a280ad310 con 0x7f9a100039b0 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a2d631640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a280ab950 con 0x7f9a2815a750 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a2d631640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2806ea90 con 0x7f9a280741a0 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a27fff640 1 -- 172.21.15.135:0/3678751104 learned_addr learned my addr 172.21.15.135:0/3678751104 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.760+0000 7f9a2ce30640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280741a0 0x7f9a28071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.761+0000 7f9a277fe640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815a750 0x7f9a281eb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.761+0000 7f9a2ce30640 1 -- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815a750 msgr2=0x7f9a281eb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:36.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.761+0000 7f9a2ce30640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815a750 0x7f9a281eb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:36.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.761+0000 7f9a2ce30640 1 -- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 msgr2=0x7f9a2814d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:36.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.761+0000 7f9a2ce30640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 0x7f9a2814d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:36.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.761+0000 7f9a2ce30640 1 -- 172.21.15.135:0/3678751104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2814d270 con 0x7f9a280741a0 2024-04-26T22:03:36.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.762+0000 7f9a2ce30640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280741a0 0x7f9a28071c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a2806d370 tx=0x7f9a1800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:36.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.762+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18018030 con 0x7f9a280741a0 2024-04-26T22:03:36.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.762+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1800cca0 con 0x7f9a280741a0 2024-04-26T22:03:36.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.762+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a281ebfa0 con 0x7f9a280741a0 2024-04-26T22:03:36.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.763+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18003bd0 con 0x7f9a280741a0 2024-04-26T22:03:36.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.763+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a281ebbc0 con 0x7f9a280741a0 2024-04-26T22:03:36.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.765+0000 7f9a27fff640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 0x7f9a2814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:36.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.765+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a1801b070 con 0x7f9a280741a0 2024-04-26T22:03:36.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.766+0000 7f9a0e7fc640 1 -- 172.21.15.135:0/3678751104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a08005290 con 0x7f9a280741a0 2024-04-26T22:03:36.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.766+0000 7f9a24ff9640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a1406c8f0 0x7f9a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:36.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.767+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f9a1808c560 con 0x7f9a280741a0 2024-04-26T22:03:36.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.767+0000 7f9a27fff640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a1406c8f0 0x7f9a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:36.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.768+0000 7f9a27fff640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a1406c8f0 0x7f9a1406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00ba30 tx=0x7f9a1c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:36.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:36.777+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a18010040 con 0x7f9a280741a0 2024-04-26T22:03:37.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.268+0000 7f9a0e7fc640 1 -- 172.21.15.135:0/3678751104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a08005580 con 0x7f9a280741a0 2024-04-26T22:03:37.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.269+0000 7f9a24ff9640 1 -- 172.21.15.135:0/3678751104 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5668 (secure 0 0 0) 0x7f9a1805a400 con 0x7f9a280741a0 2024-04-26T22:03:37.269 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:37.269 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:35.349572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:03:37.270 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 31 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.274+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a1406c8f0 msgr2=0x7f9a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.274+0000 7f9a2d631640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a1406c8f0 0x7f9a1406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00ba30 tx=0x7f9a1c030000 comp rx=0 tx=0).stop 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.274+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280741a0 msgr2=0x7f9a28071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.274+0000 7f9a2d631640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280741a0 0x7f9a28071c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a2806d370 tx=0x7f9a1800c490 comp rx=0 tx=0).stop 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.275+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 shutdown_connections 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.275+0000 7f9a2d631640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a1406c8f0 0x7f9a1406edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.275+0000 7f9a2d631640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280741a0 0x7f9a28071c00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.275+0000 7f9a2d631640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815a750 0x7f9a281eb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.275+0000 7f9a2d631640 1 --2- 172.21.15.135:0/3678751104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a100039b0 0x7f9a2814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.275+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 >> 172.21.15.135:0/3678751104 conn(0x7f9a280c5090 msgr2=0x7f9a28154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:37.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.277+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 shutdown_connections 2024-04-26T22:03:37.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:37.278+0000 7f9a2d631640 1 -- 172.21.15.135:0/3678751104 wait complete. 2024-04-26T22:03:37.298 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:03:39.299 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:39.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.511+0000 7f8ebb59e640 1 Processor -- start 2024-04-26T22:03:39.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.511+0000 7f8ebb59e640 1 -- start start 2024-04-26T22:03:39.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.513+0000 7f8ebb59e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8eb404ba70 0x7f8eb40df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.513+0000 7f8ebb59e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb4046ff0 con 0x7f8eb4047230 2024-04-26T22:03:39.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.513+0000 7f8ebb59e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb4046d10 con 0x7f8eb403f280 2024-04-26T22:03:39.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.513+0000 7f8ebb59e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40df6d0 con 0x7f8eb404ba70 2024-04-26T22:03:39.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.513+0000 7f8ebad9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8eb4047230 0x7f8eb40e9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:50572/0 (socket says 172.21.15.135:50572) 2024-04-26T22:03:39.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.513+0000 7f8ebad9d640 1 -- 172.21.15.135:0/3884260954 learned_addr learned my addr 172.21.15.135:0/3884260954 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:39.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 391543810 0 0) 0x7f8eb4046ff0 con 0x7f8eb4047230 2024-04-26T22:03:39.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea4003520 con 0x7f8eb4047230 2024-04-26T22:03:39.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2972425676 0 0) 0x7f8ea4003520 con 0x7f8eb4047230 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 >> v2:172.21.15.135:3300/0 conn(0x7f8eb404ba70 msgr2=0x7f8eb40df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 --2- 172.21.15.135:0/3884260954 >> v2:172.21.15.135:3300/0 conn(0x7f8eb404ba70 0x7f8eb40df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 >> v1:172.21.15.139:6789/0 conn(0x7f8eb403f280 legacy=0x7f8eb40dea80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb404b010 con 0x7f8eb4047230 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f8ebc050be0 con 0x7f8eb4047230 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 >> v1:172.21.15.135:6789/0 conn(0x7f8eb4047230 legacy=0x7f8eb40e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea40039b0 0x7f8ea4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ea4004800 0x7f8ea4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8ea400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40df6d0 con 0x7f8ea4004800 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea400c140 con 0x7f8ea40039b0 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.515+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea400c340 con 0x7f8ea4005210 2024-04-26T22:03:39.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.516+0000 7f8ebad9d640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8ea400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.518+0000 7f8ebad9d640 1 -- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea40039b0 msgr2=0x7f8ea4003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:39.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.518+0000 7f8ebad9d640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ea40039b0 0x7f8ea4003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.518+0000 7f8ebad9d640 1 -- 172.21.15.135:0/3884260954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ea4004800 msgr2=0x7f8ea4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.518+0000 7f8ebad9d640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ea4004800 0x7f8ea4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.518+0000 7f8ebad9d640 1 -- 172.21.15.135:0/3884260954 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb404b010 con 0x7f8ea4005210 2024-04-26T22:03:39.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.519+0000 7f8ebad9d640 1 -- 172.21.15.135:0/3884260954 reap_dead start 2024-04-26T22:03:39.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.520+0000 7f8ebad9d640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8ea400b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc051fc0 tx=0x7f8ea4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f31514b20f9747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.520+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc050f10 con 0x7f8ea4005210 2024-04-26T22:03:39.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.520+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc0aa070 con 0x7f8ea4005210 2024-04-26T22:03:39.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.520+0000 7f8eb959a640 1 -- 172.21.15.135:0/3884260954 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0ae6e0 con 0x7f8ea4005210 2024-04-26T22:03:39.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.521+0000 7f8ebb59e640 1 -- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 msgr2=0x7f8ea400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.521+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8ea400b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc051fc0 tx=0x7f8ea4005830 comp rx=0 tx=0).stop 2024-04-26T22:03:39.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.522+0000 7f8ebb59e640 1 -- 172.21.15.135:0/3884260954 shutdown_connections 2024-04-26T22:03:39.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.522+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/3884260954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8ea400b9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.522+0000 7f8ebb59e640 1 -- 172.21.15.135:0/3884260954 >> 172.21.15.135:0/3884260954 conn(0x7f8eb4048ac0 msgr2=0x7f8eb4048ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.526+0000 7f8ebb59e640 1 -- 172.21.15.135:0/3884260954 shutdown_connections 2024-04-26T22:03:39.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.527+0000 7f8ebb59e640 1 -- 172.21.15.135:0/3884260954 wait complete. 2024-04-26T22:03:39.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.529+0000 7f8ebb59e640 1 Processor -- start 2024-04-26T22:03:39.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.534+0000 7f8ebb59e640 1 -- start start 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.535+0000 7f8ebb59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8eb402dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8ebb59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb402bd60 0x7f8eb402c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8ebb59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb40d2320 0x7f8eb40d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8ebb59e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40469c0 con 0x7f8eb402bd60 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8ebb59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb404b5d0 con 0x7f8eb40d2320 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8ebb59e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb403f660 con 0x7f8ea4005210 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8eba59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8eb402dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.540+0000 7f8eba59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8eb402dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40186/0 (socket says 172.21.15.135:40186) 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.541+0000 7f8eba59c640 1 -- 172.21.15.135:0/269609049 learned_addr learned my addr 172.21.15.135:0/269609049 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.541+0000 7f8eb9d9b640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb402bd60 0x7f8eb402c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.541+0000 7f8eb9d9b640 1 -- 172.21.15.135:0/269609049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 msgr2=0x7f8eb402dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.542+0000 7f8eb9d9b640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8eb402dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.542+0000 7f8eb9d9b640 1 -- 172.21.15.135:0/269609049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb40d2320 msgr2=0x7f8eb40d27a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.542+0000 7f8eb9d9b640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb40d2320 0x7f8eb40d27a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.542+0000 7f8eb9d9b640 1 -- 172.21.15.135:0/269609049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb402c720 con 0x7f8eb402bd60 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.542+0000 7f8eb9d9b640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb402bd60 0x7f8eb402c1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006fd0 tx=0x7f8eac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.543+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac015020 con 0x7f8eb402bd60 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.543+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac011850 con 0x7f8eb402bd60 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.543+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac011c40 con 0x7f8eb402bd60 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.543+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb40e1960 con 0x7f8eb402bd60 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.543+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb4174a40 con 0x7f8eb402bd60 2024-04-26T22:03:39.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.545+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea0005290 con 0x7f8eb402bd60 2024-04-26T22:03:39.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.549+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eac018020 con 0x7f8eb402bd60 2024-04-26T22:03:39.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.550+0000 7f8eaaffd640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8e9c06c9c0 0x7f8e9c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.550+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f8eac08bec0 con 0x7f8eb402bd60 2024-04-26T22:03:39.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.550+0000 7f8eba59c640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8e9c06c9c0 0x7f8e9c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.551+0000 7f8eba59c640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8e9c06c9c0 0x7f8e9c06ee80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006fd0 tx=0x7f8eb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:39.560+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8eac01f020 con 0x7f8eb402bd60 2024-04-26T22:03:40.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.076+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ea0005580 con 0x7f8eb402bd60 2024-04-26T22:03:40.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.078+0000 7f8eaaffd640 1 -- 172.21.15.135:0/269609049 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5668 (secure 0 0 0) 0x7f8eb402c720 con 0x7f8eb402bd60 2024-04-26T22:03:40.078 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:40.080 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:35.349572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:03:40.080 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 31 2024-04-26T22:03:40.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8e9c06c9c0 msgr2=0x7f8e9c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8e9c06c9c0 0x7f8e9c06ee80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006fd0 tx=0x7f8eb0008040 comp rx=0 tx=0).stop 2024-04-26T22:03:40.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb402bd60 msgr2=0x7f8eb402c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb402bd60 0x7f8eb402c1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006fd0 tx=0x7f8eac00b8e0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 shutdown_connections 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8e9c06c9c0 0x7f8e9c06ee80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb40d2320 0x7f8eb40d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb402bd60 0x7f8eb402c1e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 --2- 172.21.15.135:0/269609049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ea4005210 0x7f8eb402dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.085+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 >> 172.21.15.135:0/269609049 conn(0x7f8eb4048ac0 msgr2=0x7f8eb40de910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.087+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 shutdown_connections 2024-04-26T22:03:40.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:40.088+0000 7f8ebb59e640 1 -- 172.21.15.135:0/269609049 wait complete. 2024-04-26T22:03:40.112 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:03:42.113 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:42.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.305+0000 7f64eda93640 1 Processor -- start 2024-04-26T22:03:42.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.305+0000 7f64eda93640 1 -- start start 2024-04-26T22:03:42.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.306+0000 7f64eda93640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64e8154160 0x7f64e8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.306+0000 7f64eda93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e80ca5a0 0x7f64e815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.306+0000 7f64eda93640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e80c8df0 con 0x7f64e80c6e00 2024-04-26T22:03:42.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.306+0000 7f64eda93640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f64e815a6e0 con 0x7f64e8154160 2024-04-26T22:03:42.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.306+0000 7f64eda93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80c9100 con 0x7f64e80ca5a0 2024-04-26T22:03:42.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64e8154160 0x7f64e8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64e8154160 0x7f64e8156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54322/0 (socket says 172.21.15.135:54322) 2024-04-26T22:03:42.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 -- 172.21.15.135:0/3654266112 learned_addr learned my addr 172.21.15.135:0/3654266112 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:42.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e6ffd640 1 --2- 172.21.15.135:0/3654266112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e80ca5a0 0x7f64e815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 -- 172.21.15.135:0/3654266112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e80ca5a0 msgr2=0x7f64e815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:42.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 --2- 172.21.15.135:0/3654266112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e80ca5a0 0x7f64e815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 -- 172.21.15.135:0/3654266112 >> v1:172.21.15.135:6789/0 conn(0x7f64e80c6e00 legacy=0x7f64e80c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.307+0000 7f64e77fe640 1 -- 172.21.15.135:0/3654266112 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e8154540 con 0x7f64e8154160 2024-04-26T22:03:42.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.309+0000 7f64e77fe640 1 --2- 172.21.15.135:0/3654266112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e8154160 0x7f64e8156bb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009e50 tx=0x7f64e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbdf7f07ace566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.309+0000 7f64e67fc640 1 -- 172.21.15.135:0/3654266112 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0036070 con 0x7f64e8154160 2024-04-26T22:03:42.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.310+0000 7f64e67fc640 1 -- 172.21.15.135:0/3654266112 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e0031600 con 0x7f64e8154160 2024-04-26T22:03:42.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.311+0000 7f64eda93640 1 -- 172.21.15.135:0/3654266112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e8154160 msgr2=0x7f64e8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.311+0000 7f64eda93640 1 --2- 172.21.15.135:0/3654266112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e8154160 0x7f64e8156bb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009e50 tx=0x7f64e002a710 comp rx=0 tx=0).stop 2024-04-26T22:03:42.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.312+0000 7f64eda93640 1 -- 172.21.15.135:0/3654266112 shutdown_connections 2024-04-26T22:03:42.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.312+0000 7f64eda93640 1 --2- 172.21.15.135:0/3654266112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e80ca5a0 0x7f64e815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.312+0000 7f64eda93640 1 --2- 172.21.15.135:0/3654266112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e8154160 0x7f64e8156bb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.312+0000 7f64eda93640 1 -- 172.21.15.135:0/3654266112 >> 172.21.15.135:0/3654266112 conn(0x7f64e80ae010 msgr2=0x7f64e80ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.315+0000 7f64eda93640 1 -- 172.21.15.135:0/3654266112 shutdown_connections 2024-04-26T22:03:42.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.316+0000 7f64eda93640 1 -- 172.21.15.135:0/3654266112 wait complete. 2024-04-26T22:03:42.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.319+0000 7f64eda93640 1 Processor -- start 2024-04-26T22:03:42.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.320+0000 7f64eda93640 1 -- start start 2024-04-26T22:03:42.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64eda93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 0x7f64e80cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64eda93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e80ca5a0 0x7f64e81451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64eda93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e8154160 0x7f64e80c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64eda93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64e815aed0 con 0x7f64e80c6e00 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64eda93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64e815abf0 con 0x7f64e80ca5a0 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64eda93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80c9ca0 con 0x7f64e8154160 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e8154160 0x7f64e80c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 0x7f64e80cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 0x7f64e80cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54326/0 (socket says 172.21.15.135:54326) 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64e77fe640 1 -- 172.21.15.135:0/3529407683 learned_addr learned my addr 172.21.15.135:0/3529407683 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:42.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.321+0000 7f64e6ffd640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e80ca5a0 0x7f64e81451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.322+0000 7f64e77fe640 1 -- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e8154160 msgr2=0x7f64e80c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.322+0000 7f64e77fe640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e8154160 0x7f64e80c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.322+0000 7f64e77fe640 1 -- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e80ca5a0 msgr2=0x7f64e81451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.322+0000 7f64e77fe640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e80ca5a0 0x7f64e81451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.322+0000 7f64e77fe640 1 -- 172.21.15.135:0/3529407683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80c1f70 con 0x7f64e80c6e00 2024-04-26T22:03:42.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.322+0000 7f64e77fe640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 0x7f64e80cb170 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009e50 tx=0x7f64e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.324+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0041020 con 0x7f64e80c6e00 2024-04-26T22:03:42.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.324+0000 7f64eda93640 1 -- 172.21.15.135:0/3529407683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e8146250 con 0x7f64e80c6e00 2024-04-26T22:03:42.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.325+0000 7f64eda93640 1 -- 172.21.15.135:0/3529407683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e8146810 con 0x7f64e80c6e00 2024-04-26T22:03:42.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.325+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e003c470 con 0x7f64e80c6e00 2024-04-26T22:03:42.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.325+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0036040 con 0x7f64e80c6e00 2024-04-26T22:03:42.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.328+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64e0044020 con 0x7f64e80c6e00 2024-04-26T22:03:42.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.328+0000 7f64cffff640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f64d406c8f0 0x7f64d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.329+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f64e00ad6e0 con 0x7f64e80c6e00 2024-04-26T22:03:42.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.329+0000 7f64e6ffd640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f64d406c8f0 0x7f64d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.330+0000 7f64e6ffd640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f64d406c8f0 0x7f64d406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64e80738e0 tx=0x7f64dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.330+0000 7f64eda93640 1 -- 172.21.15.135:0/3529407683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c8005290 con 0x7f64e80c6e00 2024-04-26T22:03:42.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.339+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64e007b550 con 0x7f64e80c6e00 2024-04-26T22:03:42.844 INFO:tasks.ceph.mds.c.smithi135.stderr:2024-04-26T22:03:42.845+0000 7f3027760640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:03:42.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.853+0000 7f64eda93640 1 -- 172.21.15.135:0/3529407683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64c8005580 con 0x7f64e80c6e00 2024-04-26T22:03:42.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.855+0000 7f64cffff640 1 -- 172.21.15.135:0/3529407683 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5675 (secure 0 0 0) 0x7f64e007ae50 con 0x7f64e80c6e00 2024-04-26T22:03:42.855 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:42.855 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:03:42.855 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 32 2024-04-26T22:03:42.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.859+0000 7f64cdffb640 1 -- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f64d406c8f0 msgr2=0x7f64d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.859+0000 7f64cdffb640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f64d406c8f0 0x7f64d406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64e80738e0 tx=0x7f64dc008040 comp rx=0 tx=0).stop 2024-04-26T22:03:42.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 -- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 msgr2=0x7f64e80cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 0x7f64e80cb170 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009e50 tx=0x7f64e0004a40 comp rx=0 tx=0).stop 2024-04-26T22:03:42.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 -- 172.21.15.135:0/3529407683 shutdown_connections 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f64d406c8f0 0x7f64d406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e8154160 0x7f64e80c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e80ca5a0 0x7f64e81451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.860+0000 7f64cdffb640 1 --2- 172.21.15.135:0/3529407683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e80c6e00 0x7f64e80cb170 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.861+0000 7f64cdffb640 1 -- 172.21.15.135:0/3529407683 >> 172.21.15.135:0/3529407683 conn(0x7f64e80ae010 msgr2=0x7f64e8150460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.861+0000 7f64cdffb640 1 -- 172.21.15.135:0/3529407683 shutdown_connections 2024-04-26T22:03:42.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:42.861+0000 7f64cdffb640 1 -- 172.21.15.135:0/3529407683 wait complete. 2024-04-26T22:03:42.883 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=4241 2024-04-26T22:03:42.883 INFO:tasks.mds_thrash.fs.[b]:waiting for 3 secs before reviving mds.b 2024-04-26T22:03:45.884 INFO:tasks.mds_thrash.fs.[b]:reviving mds.b 2024-04-26T22:03:45.884 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T22:03:45.885 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T22:03:45.887 INFO:tasks.ceph.mds.b:Started 2024-04-26T22:03:45.887 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:46.008 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-04-26T22:03:46.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.098+0000 7f44b086d640 1 Processor -- start 2024-04-26T22:03:46.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.099+0000 7f44b086d640 1 -- start start 2024-04-26T22:03:46.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.100+0000 7f44b086d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44ac0abfc0 0x7f44ac15e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:46.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.100+0000 7f44b086d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f44ac15e170 0x7f44ac06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:46.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.101+0000 7f44b086d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ac15edd0 con 0x7f44ac0c3750 2024-04-26T22:03:46.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.101+0000 7f44b086d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac15eba0 con 0x7f44ac15e170 2024-04-26T22:03:46.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.101+0000 7f44b086d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac06e3d0 con 0x7f44ac0abfc0 2024-04-26T22:03:46.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.101+0000 7f44a3fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f44ac0c3750 0x7f44ac15da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:50582/0 (socket says 172.21.15.135:50582) 2024-04-26T22:03:46.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.101+0000 7f44a3fff640 1 -- 172.21.15.135:0/789091023 learned_addr learned my addr 172.21.15.135:0/789091023 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:03:46.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.102+0000 7f44aa59c640 1 --2- 172.21.15.135:0/789091023 >> v2:172.21.15.139:3300/0 conn(0x7f44ac0abfc0 0x7f44ac15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:46.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.103+0000 7f44aad9d640 1 --2- 172.21.15.135:0/789091023 >> v2:172.21.15.135:3300/0 conn(0x7f44ac15e170 0x7f44ac06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:46.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.103+0000 7f44aa59c640 1 -- 172.21.15.135:0/789091023 >> v2:172.21.15.135:3300/0 conn(0x7f44ac15e170 msgr2=0x7f44ac06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:46.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.103+0000 7f44aa59c640 1 --2- 172.21.15.135:0/789091023 >> v2:172.21.15.135:3300/0 conn(0x7f44ac15e170 0x7f44ac06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.103+0000 7f44aa59c640 1 -- 172.21.15.135:0/789091023 >> v1:172.21.15.135:6789/0 conn(0x7f44ac0c3750 legacy=0x7f44ac15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:46.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.103+0000 7f44aa59c640 1 -- 172.21.15.135:0/789091023 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac15f0b0 con 0x7f44ac0abfc0 2024-04-26T22:03:46.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.104+0000 7f44aad9d640 1 --2- 172.21.15.135:0/789091023 >> v2:172.21.15.135:3300/0 conn(0x7f44ac15e170 0x7f44ac06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:46.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.105+0000 7f44aa59c640 1 --2- 172.21.15.135:0/789091023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac0abfc0 0x7f44ac15e660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06fa00 tx=0x7f449402a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afd23a8366c9a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:46.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.105+0000 7f44a9d9b640 1 -- 172.21.15.135:0/789091023 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494037070 con 0x7f44ac0abfc0 2024-04-26T22:03:46.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.105+0000 7f44a9d9b640 1 -- 172.21.15.135:0/789091023 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449402ad60 con 0x7f44ac0abfc0 2024-04-26T22:03:46.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.106+0000 7f44b086d640 1 -- 172.21.15.135:0/789091023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac0abfc0 msgr2=0x7f44ac15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:46.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.106+0000 7f44b086d640 1 --2- 172.21.15.135:0/789091023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac0abfc0 0x7f44ac15e660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06fa00 tx=0x7f449402a0d0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.108+0000 7f44b086d640 1 -- 172.21.15.135:0/789091023 shutdown_connections 2024-04-26T22:03:46.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.108+0000 7f44b086d640 1 --2- 172.21.15.135:0/789091023 >> v2:172.21.15.135:3300/0 conn(0x7f44ac15e170 0x7f44ac06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.108+0000 7f44b086d640 1 --2- 172.21.15.135:0/789091023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac0abfc0 0x7f44ac15e660 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.108+0000 7f44b086d640 1 -- 172.21.15.135:0/789091023 >> 172.21.15.135:0/789091023 conn(0x7f44ac06c310 msgr2=0x7f44ac06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:46.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.108+0000 7f44b086d640 1 -- 172.21.15.135:0/789091023 shutdown_connections 2024-04-26T22:03:46.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.110+0000 7f44b086d640 1 -- 172.21.15.135:0/789091023 wait complete. 2024-04-26T22:03:46.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.112+0000 7f44b086d640 1 Processor -- start 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.112+0000 7f44b086d640 1 -- start start 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.113+0000 7f44b086d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 0x7f44ac1fc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.113+0000 7f44b086d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ac0c3750 0x7f44ac142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.113+0000 7f44b086d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac15e170 0x7f44ac143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.113+0000 7f44b086d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac15c560 con 0x7f44ac0abfc0 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.113+0000 7f44b086d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac06e620 con 0x7f44ac15e170 2024-04-26T22:03:46.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.113+0000 7f44b086d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0ac3a0 con 0x7f44ac0c3750 2024-04-26T22:03:46.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.114+0000 7f44aa59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 0x7f44ac1fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:46.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.114+0000 7f44aa59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 0x7f44ac1fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54334/0 (socket says 172.21.15.135:54334) 2024-04-26T22:03:46.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.114+0000 7f44aa59c640 1 -- 172.21.15.135:0/2451334219 learned_addr learned my addr 172.21.15.135:0/2451334219 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:46.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aa59c640 1 -- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ac0c3750 msgr2=0x7f44ac142fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:46.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aa59c640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ac0c3750 0x7f44ac142fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aa59c640 1 -- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac15e170 msgr2=0x7f44ac143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:46.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aad9d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac15e170 0x7f44ac143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:46.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aa59c640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac15e170 0x7f44ac143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aa59c640 1 -- 172.21.15.135:0/2451334219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac0c0050 con 0x7f44ac0abfc0 2024-04-26T22:03:46.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.115+0000 7f44aa59c640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 0x7f44ac1fc930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4494009890 tx=0x7f44940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:46.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.116+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494041030 con 0x7f44ac0abfc0 2024-04-26T22:03:46.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.116+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac1fd170 con 0x7f44ac0abfc0 2024-04-26T22:03:46.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.116+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac0c2430 con 0x7f44ac0abfc0 2024-04-26T22:03:46.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.118+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449403d420 con 0x7f44ac0abfc0 2024-04-26T22:03:46.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.118+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494037040 con 0x7f44ac0abfc0 2024-04-26T22:03:46.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.120+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4494044070 con 0x7f44ac0abfc0 2024-04-26T22:03:46.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.121+0000 7f44a2ffd640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f448c06c8f0 0x7f448c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:46.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.123+0000 7f44a3fff640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f448c06c8f0 0x7f448c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:46.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.124+0000 7f44a3fff640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f448c06c8f0 0x7f448c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f449c007fd0 tx=0x7f449c0079a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:46.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.125+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f44940ac3c0 con 0x7f44ac0abfc0 2024-04-26T22:03:46.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.126+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4490005290 con 0x7f44ac0abfc0 2024-04-26T22:03:46.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.139+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4494044360 con 0x7f44ac0abfc0 2024-04-26T22:03:46.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.620+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4490005580 con 0x7f44ac0abfc0 2024-04-26T22:03:46.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.621+0000 7f44a2ffd640 1 -- 172.21.15.135:0/2451334219 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7f449407a2c0 con 0x7f44ac0abfc0 2024-04-26T22:03:46.624 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:03:46.624 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:03:46.624 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:03:46.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.631+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f448c06c8f0 msgr2=0x7f448c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:46.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.631+0000 7f44b086d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f448c06c8f0 0x7f448c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f449c007fd0 tx=0x7f449c0079a0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.631+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 msgr2=0x7f44ac1fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.631+0000 7f44b086d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 0x7f44ac1fc930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4494009890 tx=0x7f44940046c0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 shutdown_connections 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f448c06c8f0 0x7f448c06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ac15e170 0x7f44ac143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ac0c3750 0x7f44ac142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 --2- 172.21.15.135:0/2451334219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44ac0abfc0 0x7f44ac1fc930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 >> 172.21.15.135:0/2451334219 conn(0x7f44ac06c310 msgr2=0x7f44ac0c2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 shutdown_connections 2024-04-26T22:03:46.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:46.632+0000 7f44b086d640 1 -- 172.21.15.135:0/2451334219 wait complete. 2024-04-26T22:03:46.659 INFO:tasks.mds_thrash.fs.[b]:mds.b reported in up:standby state 2024-04-26T22:03:46.659 INFO:tasks.mds_thrash.fs.[b]:waiting for 13 secs before thrashing 2024-04-26T22:03:58.056 INFO:tasks.workunit.client.1.smithi139.stdout:fs setup took 95 secs 2024-04-26T22:03:58.794 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:03:58.797 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:03:59.660 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:59.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.866+0000 7f1231324640 1 Processor -- start 2024-04-26T22:03:59.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.866+0000 7f1231324640 1 -- start start 2024-04-26T22:03:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.867+0000 7f1231324640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 0x7f122c0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f1231324640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f122c0c7dc0 0x7f122c0c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f1231324640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c1549b0 con 0x7f122c151fb0 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f1231324640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f122c158ce0 con 0x7f122c0c7dc0 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f1231324640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f122c154c70 con 0x7f122c155750 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f122c0c7dc0 0x7f122c0c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f122c0c7dc0 0x7f122c0c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54976/0 (socket says 172.21.15.135:54976) 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 -- 172.21.15.135:0/834505078 learned_addr learned my addr 172.21.15.135:0/834505078 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 -- 172.21.15.135:0/834505078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 msgr2=0x7f122c0c3be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 --2- 172.21.15.135:0/834505078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 0x7f122c0c3be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 -- 172.21.15.135:0/834505078 >> v1:172.21.15.139:6789/0 conn(0x7f122c151fb0 legacy=0x7f122c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.868+0000 7f122b7fe640 1 -- 172.21.15.135:0/834505078 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c154f50 con 0x7f122c0c7dc0 2024-04-26T22:03:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.870+0000 7f122b7fe640 1 --2- 172.21.15.135:0/834505078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c0c7dc0 0x7f122c0c4120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f121c009870 tx=0x7f121c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4881ff197b53acd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.871+0000 7f1229ffb640 1 -- 172.21.15.135:0/834505078 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c037070 con 0x7f122c0c7dc0 2024-04-26T22:03:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.871+0000 7f1229ffb640 1 -- 172.21.15.135:0/834505078 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c032440 con 0x7f122c0c7dc0 2024-04-26T22:03:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.871+0000 7f1229ffb640 1 -- 172.21.15.135:0/834505078 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c032810 con 0x7f122c0c7dc0 2024-04-26T22:03:59.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.874+0000 7f1231324640 1 -- 172.21.15.135:0/834505078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c0c7dc0 msgr2=0x7f122c0c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:59.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.874+0000 7f1231324640 1 --2- 172.21.15.135:0/834505078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c0c7dc0 0x7f122c0c4120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f121c009870 tx=0x7f121c02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:03:59.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.874+0000 7f1231324640 1 -- 172.21.15.135:0/834505078 shutdown_connections 2024-04-26T22:03:59.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.874+0000 7f1231324640 1 --2- 172.21.15.135:0/834505078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c0c7dc0 0x7f122c0c4120 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:59.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.874+0000 7f1231324640 1 --2- 172.21.15.135:0/834505078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 0x7f122c0c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:59.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.874+0000 7f1231324640 1 -- 172.21.15.135:0/834505078 >> 172.21.15.135:0/834505078 conn(0x7f122c0c5ce0 msgr2=0x7f122c0c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:59.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.881+0000 7f1231324640 1 -- 172.21.15.135:0/834505078 shutdown_connections 2024-04-26T22:03:59.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.881+0000 7f1231324640 1 -- 172.21.15.135:0/834505078 wait complete. 2024-04-26T22:03:59.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.883+0000 7f1231324640 1 Processor -- start 2024-04-26T22:03:59.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.883+0000 7f1231324640 1 -- start start 2024-04-26T22:03:59.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.884+0000 7f1231324640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0c7dc0 0x7f122c074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:59.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.884+0000 7f1231324640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 0x7f122c1f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:59.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.884+0000 7f1231324640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 0x7f122c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:59.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.884+0000 7f1231324640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f122c1589b0 con 0x7f122c151fb0 2024-04-26T22:03:59.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.884+0000 7f1231324640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f122c152390 con 0x7f122c0c7dc0 2024-04-26T22:03:59.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.884+0000 7f1231324640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f122c06da30 con 0x7f122c155750 2024-04-26T22:03:59.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.885+0000 7f122a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 0x7f122c1f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:59.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.885+0000 7f122a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 0x7f122c1f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54978/0 (socket says 172.21.15.135:54978) 2024-04-26T22:03:59.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.885+0000 7f122a7fc640 1 -- 172.21.15.135:0/1245489003 learned_addr learned my addr 172.21.15.135:0/1245489003 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:03:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.886+0000 7f122a7fc640 1 -- 172.21.15.135:0/1245489003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 msgr2=0x7f122c1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.886+0000 7f122a7fc640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 0x7f122c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.886+0000 7f122a7fc640 1 -- 172.21.15.135:0/1245489003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0c7dc0 msgr2=0x7f122c074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:59.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.886+0000 7f122a7fc640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0c7dc0 0x7f122c074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.886+0000 7f122a7fc640 1 -- 172.21.15.135:0/1245489003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c1f0f70 con 0x7f122c151fb0 2024-04-26T22:03:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.886+0000 7f122a7fc640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 0x7f122c1f02e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f122000b3e0 tx=0x7f122000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.887+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220015020 con 0x7f122c151fb0 2024-04-26T22:03:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.887+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c157840 con 0x7f122c151fb0 2024-04-26T22:03:59.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.887+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c1f0c90 con 0x7f122c151fb0 2024-04-26T22:03:59.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.889+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c0696c0 con 0x7f122c151fb0 2024-04-26T22:03:59.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.900+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1220011850 con 0x7f122c151fb0 2024-04-26T22:03:59.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.900+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220011c40 con 0x7f122c151fb0 2024-04-26T22:03:59.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.900+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1220018020 con 0x7f122c151fb0 2024-04-26T22:03:59.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.901+0000 7f12137fe640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f121406cb30 0x7f121406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:59.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.901+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f122008bd10 con 0x7f122c151fb0 2024-04-26T22:03:59.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.901+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f122001f030 con 0x7f122c151fb0 2024-04-26T22:03:59.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.904+0000 7f122affd640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f121406cb30 0x7f121406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:59.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:03:59.905+0000 7f122affd640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f121406cb30 0x7f121406eff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1224001cb0 tx=0x7f1224007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:00.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.385+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f122c0507a0 con 0x7f122c151fb0 2024-04-26T22:04:00.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.386+0000 7f12137fe640 1 -- 172.21.15.135:0/1245489003 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7f1220059c90 con 0x7f122c151fb0 2024-04-26T22:04:00.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:00.392 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:04:00.392 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:04:00.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f121406cb30 msgr2=0x7f121406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:00.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f121406cb30 0x7f121406eff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1224001cb0 tx=0x7f1224007d10 comp rx=0 tx=0).stop 2024-04-26T22:04:00.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 msgr2=0x7f122c1f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:00.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 0x7f122c1f02e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f122000b3e0 tx=0x7f122000b8a0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 shutdown_connections 2024-04-26T22:04:00.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f121406cb30 0x7f121406eff0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f122c155750 0x7f122c1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f122c151fb0 0x7f122c1f02e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 --2- 172.21.15.135:0/1245489003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0c7dc0 0x7f122c074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.393+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 >> 172.21.15.135:0/1245489003 conn(0x7f122c0c5ce0 msgr2=0x7f122c0c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:00.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.395+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 shutdown_connections 2024-04-26T22:04:00.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.395+0000 7f1231324640 1 -- 172.21.15.135:0/1245489003 wait complete. 2024-04-26T22:04:00.421 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-04-26T22:04:00.421 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:04:00.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:04:00.443 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:04:00.443 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:00.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.692+0000 7ff266778640 1 Processor -- start 2024-04-26T22:04:00.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.692+0000 7ff266778640 1 -- start start 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff266778640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff260070120 0x7ff2600aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff25ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff26006d590 0x7ff260158d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:60650/0 (socket says 172.21.15.135:60650) 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff25f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff260070120 0x7ff2600aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff264d15640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff2600aef70 0x7ff2600c3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39106/0 (socket says 172.21.15.135:39106) 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff25f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff260070120 0x7ff2600aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54990/0 (socket says 172.21.15.135:54990) 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff25ffff640 1 -- 172.21.15.135:0/1550854180 learned_addr learned my addr 172.21.15.135:0/1550854180 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff260154e10 con 0x7ff2600aef70 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2601550b0 con 0x7ff26006d590 2024-04-26T22:04:00.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.694+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff2601559e0 con 0x7ff260070120 2024-04-26T22:04:00.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.695+0000 7ff25effd640 1 -- 172.21.15.135:0/1550854180 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622954221 0 0) 0x7ff260154e10 con 0x7ff2600aef70 2024-04-26T22:04:00.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.695+0000 7ff25effd640 1 -- 172.21.15.135:0/1550854180 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff248003520 con 0x7ff2600aef70 2024-04-26T22:04:00.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.695+0000 7ff25effd640 1 -- 172.21.15.135:0/1550854180 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464864789 0 0) 0x7ff2601550b0 con 0x7ff26006d590 2024-04-26T22:04:00.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.695+0000 7ff25effd640 1 -- 172.21.15.135:0/1550854180 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff260154e10 con 0x7ff26006d590 2024-04-26T22:04:00.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.696+0000 7ff25f7fe640 1 -- 172.21.15.135:0/1550854180 >> v1:172.21.15.139:6789/0 conn(0x7ff26006d590 legacy=0x7ff260158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:00.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.696+0000 7ff25f7fe640 1 -- 172.21.15.135:0/1550854180 >> v1:172.21.15.135:6789/0 conn(0x7ff2600aef70 legacy=0x7ff2600c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:00.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.696+0000 7ff25f7fe640 1 -- 172.21.15.135:0/1550854180 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff260155390 con 0x7ff260070120 2024-04-26T22:04:00.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.697+0000 7ff25f7fe640 1 --2- 172.21.15.135:0/1550854180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 0x7ff2600aea30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff254009720 tx=0x7ff25402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f6ff73de1e9c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:00.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.698+0000 7ff25effd640 1 -- 172.21.15.135:0/1550854180 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254037070 con 0x7ff260070120 2024-04-26T22:04:00.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.698+0000 7ff25effd640 1 -- 172.21.15.135:0/1550854180 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25402ab60 con 0x7ff260070120 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.699+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 msgr2=0x7ff2600aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.699+0000 7ff266778640 1 --2- 172.21.15.135:0/1550854180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 0x7ff2600aea30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff254009720 tx=0x7ff25402a0d0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.699+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 shutdown_connections 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.699+0000 7ff266778640 1 --2- 172.21.15.135:0/1550854180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 0x7ff2600aea30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.699+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 >> 172.21.15.135:0/1550854180 conn(0x7ff2600c5ce0 msgr2=0x7ff2600c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.699+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 shutdown_connections 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.700+0000 7ff266778640 1 -- 172.21.15.135:0/1550854180 wait complete. 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.701+0000 7ff266778640 1 Processor -- start 2024-04-26T22:04:00.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.702+0000 7ff266778640 1 -- start start 2024-04-26T22:04:00.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff266778640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 0x7ff260074130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:00.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff266778640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 0x7ff26014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:00.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff266778640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2600aef70 0x7ff260150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:00.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff266778640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff26006d970 con 0x7ff260070120 2024-04-26T22:04:00.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff266778640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2600af3e0 con 0x7ff26006d590 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff266778640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff26006daf0 con 0x7ff2600aef70 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 0x7ff260074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 0x7ff260074130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56362/0 (socket says 172.21.15.135:56362) 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 -- 172.21.15.135:0/3818096102 learned_addr learned my addr 172.21.15.135:0/3818096102 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff264d15640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2600aef70 0x7ff260150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 -- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2600aef70 msgr2=0x7ff260150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2600aef70 0x7ff260150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 -- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 msgr2=0x7ff26014c520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 0x7ff26014c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.703+0000 7ff25ffff640 1 -- 172.21.15.135:0/3818096102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2601fa2d0 con 0x7ff26006d590 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.704+0000 7ff25ffff640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 0x7ff260074130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2500089a0 tx=0x7ff25000c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:00.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.704+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25000cd40 con 0x7ff26006d590 2024-04-26T22:04:00.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.704+0000 7ff266778640 1 -- 172.21.15.135:0/3818096102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff260156610 con 0x7ff26006d590 2024-04-26T22:04:00.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.704+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2500070d0 con 0x7ff26006d590 2024-04-26T22:04:00.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.704+0000 7ff266778640 1 -- 172.21.15.135:0/3818096102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2600c3130 con 0x7ff26006d590 2024-04-26T22:04:00.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.704+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007480 con 0x7ff26006d590 2024-04-26T22:04:00.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.707+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff25000a110 con 0x7ff26006d590 2024-04-26T22:04:00.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.707+0000 7ff25cff9640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff24c06c8f0 0x7ff24c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:00.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.707+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7ff25008c3c0 con 0x7ff26006d590 2024-04-26T22:04:00.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.709+0000 7ff25f7fe640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff24c06c8f0 0x7ff24c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:00.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.709+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2600c5550 con 0x7ff26006d590 2024-04-26T22:04:00.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.710+0000 7ff25f7fe640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff24c06c8f0 0x7ff24c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff254034000 tx=0x7ff25402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:00.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:00.718+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2601fa2d0 con 0x7ff26006d590 2024-04-26T22:04:01.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.236+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2600507a0 con 0x7ff26006d590 2024-04-26T22:04:01.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.237+0000 7ff25cff9640 1 -- 172.21.15.135:0/3818096102 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7ff25005a240 con 0x7ff26006d590 2024-04-26T22:04:01.240 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:01.240 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:04:01.240 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.242+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff24c06c8f0 msgr2=0x7ff24c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.242+0000 7ff2467fc640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff24c06c8f0 0x7ff24c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff254034000 tx=0x7ff25402e000 comp rx=0 tx=0).stop 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.242+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 msgr2=0x7ff260074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.242+0000 7ff2467fc640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 0x7ff260074130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2500089a0 tx=0x7ff25000c420 comp rx=0 tx=0).stop 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.243+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 shutdown_connections 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.243+0000 7ff2467fc640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff24c06c8f0 0x7ff24c06edb0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff254034000 tx=0x7ff25402e000 comp rx=0 tx=0).stop 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.243+0000 7ff2467fc640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2600aef70 0x7ff260150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.243+0000 7ff2467fc640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff260070120 0x7ff26014c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.243+0000 7ff2467fc640 1 --2- 172.21.15.135:0/3818096102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff26006d590 0x7ff260074130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.243+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 >> 172.21.15.135:0/3818096102 conn(0x7ff2600c5ce0 msgr2=0x7ff260153e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:01.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.247+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 shutdown_connections 2024-04-26T22:04:01.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:01.247+0000 7ff2467fc640 1 -- 172.21.15.135:0/3818096102 wait complete. 2024-04-26T22:04:01.269 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:04:03.270 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:03.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.459+0000 7f9f4938e640 1 Processor -- start 2024-04-26T22:04:03.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.459+0000 7f9f4938e640 1 -- start start 2024-04-26T22:04:03.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f4938e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f3c0d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f43fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f3c0d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:03.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f43fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f3c0d8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40228/0 (socket says 172.21.15.135:40228) 2024-04-26T22:04:03.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f43fff640 1 -- 172.21.15.135:0/3614206597 learned_addr learned my addr 172.21.15.135:0/3614206597 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:03.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c0d9180 con 0x7f9f3c02d0a0 2024-04-26T22:04:03.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c04bc40 con 0x7f9f3c04cd00 2024-04-26T22:04:03.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.461+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c04bf20 con 0x7f9f3c0d5900 2024-04-26T22:04:03.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.462+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228584280 0 0) 0x7f9f3c04bc40 con 0x7f9f3c04cd00 2024-04-26T22:04:03.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.462+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f34003520 con 0x7f9f3c04cd00 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1905410400 0 0) 0x7f9f34003520 con 0x7f9f3c04cd00 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 msgr2=0x7f9f3c0d8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f3c0d8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 >> v1:172.21.15.135:6789/0 conn(0x7f9f3c02d0a0 legacy=0x7f9f3c048360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c04c500 con 0x7f9f3c04cd00 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f9f38001690 con 0x7f9f3c04cd00 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 >> v1:172.21.15.139:6789/0 conn(0x7f9f3c04cd00 legacy=0x7f9f3c04d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:03.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f340039b0 0x7f9f34003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f34004d40 0x7f9f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c04bf20 con 0x7f9f34004d40 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3400ff60 con 0x7f9f340039b0 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f34010130 con 0x7f9f3c0d5900 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f43fff640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f340039b0 0x7f9f34003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.463+0000 7f9f437fe640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.464+0000 7f9f43fff640 1 -- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 msgr2=0x7f9f34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.464+0000 7f9f43fff640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c0d5900 0x7f9f34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.464+0000 7f9f43fff640 1 -- 172.21.15.135:0/3614206597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f34004d40 msgr2=0x7f9f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.464+0000 7f9f43fff640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f34004d40 0x7f9f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.464+0000 7f9f43fff640 1 -- 172.21.15.135:0/3614206597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c04c500 con 0x7f9f340039b0 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.465+0000 7f9f43fff640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f340039b0 0x7f9f34003db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f4406bd80 tx=0x7f9f34005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1510627bf922431a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.465+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440b5070 con 0x7f9f340039b0 2024-04-26T22:04:03.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.466+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f440b0410 con 0x7f9f340039b0 2024-04-26T22:04:03.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.466+0000 7f9f42ffd640 1 -- 172.21.15.135:0/3614206597 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440515b0 con 0x7f9f340039b0 2024-04-26T22:04:03.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f340039b0 msgr2=0x7f9f34003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:03.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f4938e640 1 --2- 172.21.15.135:0/3614206597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f340039b0 0x7f9f34003db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f4406bd80 tx=0x7f9f34005830 comp rx=0 tx=0).stop 2024-04-26T22:04:03.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f48b8d640 1 -- 172.21.15.135:0/3614206597 reap_dead start 2024-04-26T22:04:03.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 shutdown_connections 2024-04-26T22:04:03.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 >> 172.21.15.135:0/3614206597 conn(0x7f9f3c04d9e0 msgr2=0x7f9f3c04ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:03.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 shutdown_connections 2024-04-26T22:04:03.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.467+0000 7f9f4938e640 1 -- 172.21.15.135:0/3614206597 wait complete. 2024-04-26T22:04:03.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.469+0000 7f9f4938e640 1 Processor -- start 2024-04-26T22:04:03.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.470+0000 7f9f4938e640 1 -- start start 2024-04-26T22:04:03.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f4938e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f340039b0 0x7f9f3c1780a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f4938e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f3c174e10 0x7f9f3c0d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f4938e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 0x7f9f3c046290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f4938e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c048a70 con 0x7f9f340039b0 2024-04-26T22:04:03.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f4938e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c00c3c0 con 0x7f9f3c174e10 2024-04-26T22:04:03.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f4938e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c048bf0 con 0x7f9f3c045e00 2024-04-26T22:04:03.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f48b8d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 0x7f9f3c046290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:03.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f48b8d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 0x7f9f3c046290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40244/0 (socket says 172.21.15.135:40244) 2024-04-26T22:04:03.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.471+0000 7f9f48b8d640 1 -- 172.21.15.135:0/2492706752 learned_addr learned my addr 172.21.15.135:0/2492706752 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.472+0000 7f9f48b8d640 1 -- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f3c174e10 msgr2=0x7f9f3c0d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.472+0000 7f9f48b8d640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f3c174e10 0x7f9f3c0d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.472+0000 7f9f48b8d640 1 -- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f340039b0 msgr2=0x7f9f3c1780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.472+0000 7f9f48b8d640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f340039b0 0x7f9f3c1780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.472+0000 7f9f48b8d640 1 -- 172.21.15.135:0/2492706752 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c041560 con 0x7f9f3c045e00 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.473+0000 7f9f48b8d640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 0x7f9f3c046290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c04b500 tx=0x7f9f3800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.473+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38017070 con 0x7f9f3c045e00 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.473+0000 7f9f4938e640 1 -- 172.21.15.135:0/2492706752 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c174330 con 0x7f9f3c045e00 2024-04-26T22:04:03.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.473+0000 7f9f4938e640 1 -- 172.21.15.135:0/2492706752 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c0d19e0 con 0x7f9f3c045e00 2024-04-26T22:04:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.473+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f38007270 con 0x7f9f3c045e00 2024-04-26T22:04:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.473+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f380057c0 con 0x7f9f3c045e00 2024-04-26T22:04:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.475+0000 7f9f4938e640 1 -- 172.21.15.135:0/2492706752 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24005290 con 0x7f9f3c045e00 2024-04-26T22:04:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.479+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f3801a070 con 0x7f9f3c045e00 2024-04-26T22:04:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.479+0000 7f9f40ff9640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f2806c8f0 0x7f9f2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:03.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.480+0000 7f9f43fff640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f2806c8f0 0x7f9f2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:03.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.481+0000 7f9f43fff640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f2806c8f0 0x7f9f2806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9f440af000 tx=0x7f9f440ad000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:03.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.487+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f9f3808b390 con 0x7f9f3c045e00 2024-04-26T22:04:03.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:03.498+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f38059290 con 0x7f9f3c045e00 2024-04-26T22:04:04.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.039+0000 7f9f4938e640 1 -- 172.21.15.135:0/2492706752 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f24005580 con 0x7f9f3c045e00 2024-04-26T22:04:04.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.043+0000 7f9f40ff9640 1 -- 172.21.15.135:0/2492706752 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7f9f38058b90 con 0x7f9f3c045e00 2024-04-26T22:04:04.043 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:04.043 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:04:04.043 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:04:04.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 -- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f2806c8f0 msgr2=0x7f9f2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f2806c8f0 0x7f9f2806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9f440af000 tx=0x7f9f440ad000 comp rx=0 tx=0).stop 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 -- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 msgr2=0x7f9f3c046290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 0x7f9f3c046290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c04b500 tx=0x7f9f3800cb90 comp rx=0 tx=0).stop 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 -- 172.21.15.135:0/2492706752 shutdown_connections 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9f2806c8f0 0x7f9f2806edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f3c045e00 0x7f9f3c046290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f3c174e10 0x7f9f3c0d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 --2- 172.21.15.135:0/2492706752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f340039b0 0x7f9f3c1780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:04.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.048+0000 7f9f2e7fc640 1 -- 172.21.15.135:0/2492706752 >> 172.21.15.135:0/2492706752 conn(0x7f9f3c04d9e0 msgr2=0x7f9f3c02d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:04.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.049+0000 7f9f2e7fc640 1 -- 172.21.15.135:0/2492706752 shutdown_connections 2024-04-26T22:04:04.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:04.050+0000 7f9f2e7fc640 1 -- 172.21.15.135:0/2492706752 wait complete. 2024-04-26T22:04:04.071 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:04:06.071 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:06.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.294+0000 7f5ca661d640 1 Processor -- start 2024-04-26T22:04:06.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.294+0000 7f5ca661d640 1 -- start start 2024-04-26T22:04:06.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.296+0000 7f5ca661d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c9804a980 0x7f5c98006b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.296+0000 7f5ca661d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c980dbea0 0x7f5c980d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.296+0000 7f5ca661d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c980072c0 con 0x7f5c980d5580 2024-04-26T22:04:06.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.296+0000 7f5ca661d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5c98030ad0 con 0x7f5c9804a980 2024-04-26T22:04:06.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.296+0000 7f5ca661d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c98030db0 con 0x7f5c980dbea0 2024-04-26T22:04:06.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.297+0000 7f5c9f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5c980d5580 0x7f5c980db790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39128/0 (socket says 172.21.15.135:39128) 2024-04-26T22:04:06.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.297+0000 7f5c9f7fe640 1 -- 172.21.15.135:0/1696576893 learned_addr learned my addr 172.21.15.135:0/1696576893 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:06.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.297+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107794206 0 0) 0x7f5c980072c0 con 0x7f5c980d5580 2024-04-26T22:04:06.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.297+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c90003520 con 0x7f5c980d5580 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 530541447 0 0) 0x7f5c90003520 con 0x7f5c980d5580 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c980dbea0 msgr2=0x7f5c980d59a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c980dbea0 0x7f5c980d59a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 >> v2:172.21.15.139:3300/0 conn(0x7f5c9804a980 msgr2=0x7f5c98006b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 --2- 172.21.15.135:0/1696576893 >> v2:172.21.15.139:3300/0 conn(0x7f5c9804a980 0x7f5c98006b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c98031090 con 0x7f5c980d5580 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f5c8c001670 con 0x7f5c980d5580 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 >> v1:172.21.15.135:6789/0 conn(0x7f5c980d5580 legacy=0x7f5c980db790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.298+0000 7f5c9effd640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c900039b0 0x7f5c90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9effd640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c90004800 0x7f5c90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9effd640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900052e0 0x7f5c9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c98030ad0 con 0x7f5c900039b0 2024-04-26T22:04:06.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9f7fe640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c90004800 0x7f5c90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:06.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c98030db0 con 0x7f5c90004800 2024-04-26T22:04:06.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c900039b0 0x7f5c90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:06.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.299+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9000c1d0 con 0x7f5c900052e0 2024-04-26T22:04:06.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.301+0000 7f5ca4bba640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900052e0 0x7f5c9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:06.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.302+0000 7f5c9ffff640 1 -- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900052e0 msgr2=0x7f5c9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:06.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.302+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900052e0 0x7f5c9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.302+0000 7f5c9ffff640 1 -- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c90004800 msgr2=0x7f5c90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:06.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.302+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c90004800 0x7f5c90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.302+0000 7f5c9ffff640 1 -- 172.21.15.135:0/1696576893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c90003520 con 0x7f5c900039b0 2024-04-26T22:04:06.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.303+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c900039b0 0x7f5c90003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0075190 tx=0x7f5c90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4bc7b0011984bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:06.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.305+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00b4070 con 0x7f5c900039b0 2024-04-26T22:04:06.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.305+0000 7f5c9effd640 1 -- 172.21.15.135:0/1696576893 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca00a7d40 con 0x7f5c900039b0 2024-04-26T22:04:06.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.306+0000 7f5ca661d640 1 -- 172.21.15.135:0/1696576893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c900039b0 msgr2=0x7f5c90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:06.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.306+0000 7f5ca661d640 1 --2- 172.21.15.135:0/1696576893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c900039b0 0x7f5c90003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0075190 tx=0x7f5c90005830 comp rx=0 tx=0).stop 2024-04-26T22:04:06.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.306+0000 7f5ca4bba640 1 -- 172.21.15.135:0/1696576893 reap_dead start 2024-04-26T22:04:06.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.308+0000 7f5ca661d640 1 -- 172.21.15.135:0/1696576893 shutdown_connections 2024-04-26T22:04:06.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.308+0000 7f5ca661d640 1 -- 172.21.15.135:0/1696576893 >> 172.21.15.135:0/1696576893 conn(0x7f5c98005e90 msgr2=0x7f5c980062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:06.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.309+0000 7f5ca661d640 1 -- 172.21.15.135:0/1696576893 shutdown_connections 2024-04-26T22:04:06.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.309+0000 7f5ca661d640 1 -- 172.21.15.135:0/1696576893 wait complete. 2024-04-26T22:04:06.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.310+0000 7f5ca661d640 1 Processor -- start 2024-04-26T22:04:06.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.312+0000 7f5ca661d640 1 -- start start 2024-04-26T22:04:06.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.313+0000 7f5ca661d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900039b0 0x7f5c980453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.313+0000 7f5ca661d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 0x7f5c98179bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.313+0000 7f5ca661d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9817a110 0x7f5c9817ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.313+0000 7f5ca661d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980315b0 con 0x7f5c980d7680 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.313+0000 7f5ca661d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9802e080 con 0x7f5c9817a110 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.313+0000 7f5ca661d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c980cfbd0 con 0x7f5c900039b0 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 0x7f5c98179bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 0x7f5c98179bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55042/0 (socket says 172.21.15.135:55042) 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 -- 172.21.15.135:0/2175957231 learned_addr learned my addr 172.21.15.135:0/2175957231 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900039b0 0x7f5c980453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:06.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 -- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900039b0 msgr2=0x7f5c980453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:06.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900039b0 0x7f5c980453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 -- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9817a110 msgr2=0x7f5c9817ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:06.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.314+0000 7f5c9f7fe640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9817a110 0x7f5c9817ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.315+0000 7f5c9f7fe640 1 -- 172.21.15.135:0/2175957231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c980d9590 con 0x7f5c980d7680 2024-04-26T22:04:06.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.315+0000 7f5c9f7fe640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 0x7f5c98179bd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007340 tx=0x7f5c8c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:06.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.316+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c01a070 con 0x7f5c980d7680 2024-04-26T22:04:06.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.316+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9817c960 con 0x7f5c980d7680 2024-04-26T22:04:06.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.316+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c980d0720 con 0x7f5c980d7680 2024-04-26T22:04:06.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.316+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00f940 con 0x7f5c980d7680 2024-04-26T22:04:06.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.317+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00fd10 con 0x7f5c980d7680 2024-04-26T22:04:06.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.317+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c80005290 con 0x7f5c980d7680 2024-04-26T22:04:06.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.325+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c8c01d070 con 0x7f5c980d7680 2024-04-26T22:04:06.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.326+0000 7f5c9cff9640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c8406c9c0 0x7f5c8406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:06.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.330+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7f5c8c08bb10 con 0x7f5c980d7680 2024-04-26T22:04:06.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.331+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c8406c9c0 0x7f5c8406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:06.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.331+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c8c08f280 con 0x7f5c980d7680 2024-04-26T22:04:06.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.331+0000 7f5c9ffff640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c8406c9c0 0x7f5c8406ee80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c120 tx=0x7f5ca00ab000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:06.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.849+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c80005580 con 0x7f5c980d7680 2024-04-26T22:04:06.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.853+0000 7f5c9cff9640 1 -- 172.21.15.135:0/2175957231 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7f5c8c059a90 con 0x7f5c980d7680 2024-04-26T22:04:06.854 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:06.854 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:04:06.854 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:04:06.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.856+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c8406c9c0 msgr2=0x7f5c8406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:06.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.856+0000 7f5ca661d640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c8406c9c0 0x7f5c8406ee80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c120 tx=0x7f5ca00ab000 comp rx=0 tx=0).stop 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.856+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 msgr2=0x7f5c98179bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.856+0000 7f5ca661d640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 0x7f5c98179bd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007340 tx=0x7f5c8c007800 comp rx=0 tx=0).stop 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.856+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 shutdown_connections 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.856+0000 7f5ca661d640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c8406c9c0 0x7f5c8406ee80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.857+0000 7f5ca661d640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9817a110 0x7f5c9817ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.857+0000 7f5ca661d640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c980d7680 0x7f5c98179bd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.857+0000 7f5ca661d640 1 --2- 172.21.15.135:0/2175957231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c900039b0 0x7f5c980453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.857+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 >> 172.21.15.135:0/2175957231 conn(0x7f5c98005e90 msgr2=0x7f5c980055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.857+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 shutdown_connections 2024-04-26T22:04:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:06.857+0000 7f5ca661d640 1 -- 172.21.15.135:0/2175957231 wait complete. 2024-04-26T22:04:06.883 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:04:08.884 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:09.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.084+0000 7fd6bb59e640 1 Processor -- start 2024-04-26T22:04:09.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.084+0000 7fd6bb59e640 1 -- start start 2024-04-26T22:04:09.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6bb59e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd6b40071b0 0x7fd6b4007590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:09.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:09.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6bb59e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b404a560 con 0x7fd6b400e9c0 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6bb59e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b400e530 con 0x7fd6b40071b0 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b40e2c70 con 0x7fd6b404db50 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6ba59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd6b40071b0 0x7fd6b4007590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6ba59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd6b40071b0 0x7fd6b4007590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39958/0 (socket says 172.21.15.135:39958) 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6ba59c640 1 -- 172.21.15.135:0/14347930 learned_addr learned my addr 172.21.15.135:0/14347930 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.086+0000 7fd6b9d9b640 1 --2- 172.21.15.135:0/14347930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.087+0000 7fd6ba59c640 1 -- 172.21.15.135:0/14347930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 msgr2=0x7fd6b40e2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:09.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.087+0000 7fd6ba59c640 1 --2- 172.21.15.135:0/14347930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.087+0000 7fd6ba59c640 1 -- 172.21.15.135:0/14347930 >> v1:172.21.15.135:6789/0 conn(0x7fd6b400e9c0 legacy=0x7fd6b40de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:09.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.087+0000 7fd6ba59c640 1 -- 172.21.15.135:0/14347930 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b400e200 con 0x7fd6b40071b0 2024-04-26T22:04:09.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.088+0000 7fd6ba59c640 1 --2- 172.21.15.135:0/14347930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b40071b0 0x7fd6b4007590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc065e50 tx=0x7fd6bc0a7270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deccd2dbb71efc9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:09.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.089+0000 7fd6b959a640 1 -- 172.21.15.135:0/14347930 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b4070 con 0x7fd6b40071b0 2024-04-26T22:04:09.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.089+0000 7fd6b959a640 1 -- 172.21.15.135:0/14347930 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0af640 con 0x7fd6b40071b0 2024-04-26T22:04:09.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.089+0000 7fd6b959a640 1 -- 172.21.15.135:0/14347930 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0afab0 con 0x7fd6b40071b0 2024-04-26T22:04:09.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.092+0000 7fd6bb59e640 1 -- 172.21.15.135:0/14347930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b40071b0 msgr2=0x7fd6b4007590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:09.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.092+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/14347930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b40071b0 0x7fd6b4007590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc065e50 tx=0x7fd6bc0a7270 comp rx=0 tx=0).stop 2024-04-26T22:04:09.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.092+0000 7fd6bb59e640 1 -- 172.21.15.135:0/14347930 shutdown_connections 2024-04-26T22:04:09.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.092+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/14347930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.092+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/14347930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b40071b0 0x7fd6b4007590 secure :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc065e50 tx=0x7fd6bc0a7270 comp rx=0 tx=0).stop 2024-04-26T22:04:09.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.092+0000 7fd6bb59e640 1 -- 172.21.15.135:0/14347930 >> 172.21.15.135:0/14347930 conn(0x7fd6b404c660 msgr2=0x7fd6b404ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:09.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.095+0000 7fd6bb59e640 1 -- 172.21.15.135:0/14347930 shutdown_connections 2024-04-26T22:04:09.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.095+0000 7fd6bb59e640 1 -- 172.21.15.135:0/14347930 wait complete. 2024-04-26T22:04:09.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.097+0000 7fd6bb59e640 1 Processor -- start 2024-04-26T22:04:09.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.097+0000 7fd6bb59e640 1 -- start start 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.098+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 0x7fd6b40dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.098+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.098+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b4049e10 0x7fd6b4046cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.099+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b404adc0 con 0x7fd6b400e9c0 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.099+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b404aae0 con 0x7fd6b4049e10 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.099+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b404af40 con 0x7fd6b404db50 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.099+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 0x7fd6b40dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.099+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 0x7fd6b40dfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39966/0 (socket says 172.21.15.135:39966) 2024-04-26T22:04:09.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.099+0000 7fd6ba59c640 1 -- 172.21.15.135:0/502034546 learned_addr learned my addr 172.21.15.135:0/502034546 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:09.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.100+0000 7fd6ba59c640 1 -- 172.21.15.135:0/502034546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 msgr2=0x7fd6b40e00c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:09.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.100+0000 7fd6ba59c640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.100+0000 7fd6ba59c640 1 -- 172.21.15.135:0/502034546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b4049e10 msgr2=0x7fd6b4046cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:09.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.100+0000 7fd6ba59c640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b4049e10 0x7fd6b4046cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.100+0000 7fd6ba59c640 1 -- 172.21.15.135:0/502034546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b4175500 con 0x7fd6b400e9c0 2024-04-26T22:04:09.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.100+0000 7fd6ba59c640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 0x7fd6b40dfb80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc051fc0 tx=0x7fd6bc0661d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:09.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.101+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0bf020 con 0x7fd6b400e9c0 2024-04-26T22:04:09.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.101+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc06e640 con 0x7fd6b400e9c0 2024-04-26T22:04:09.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.101+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b4040 con 0x7fd6b400e9c0 2024-04-26T22:04:09.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.102+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b4175000 con 0x7fd6b400e9c0 2024-04-26T22:04:09.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.102+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b40db740 con 0x7fd6b400e9c0 2024-04-26T22:04:09.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.105+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a0005290 con 0x7fd6b400e9c0 2024-04-26T22:04:09.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.117+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6bc0c2020 con 0x7fd6b400e9c0 2024-04-26T22:04:09.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.118+0000 7fd6aaffd640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd69c06c9c0 0x7fd69c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:09.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.120+0000 7fd6b9d9b640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd69c06c9c0 0x7fd69c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:09.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.120+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7fd6bc1292c0 con 0x7fd6b400e9c0 2024-04-26T22:04:09.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.120+0000 7fd6b9d9b640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd69c06c9c0 0x7fd69c06ee80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6b41723c0 tx=0x7fd6ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:09.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.121+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6b4175500 con 0x7fd6b400e9c0 2024-04-26T22:04:09.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.594+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a0005580 con 0x7fd6b400e9c0 2024-04-26T22:04:09.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.595+0000 7fd6aaffd640 1 -- 172.21.15.135:0/502034546 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7fd6bc0f7240 con 0x7fd6b400e9c0 2024-04-26T22:04:09.596 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:09.597 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:04:09.597 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:04:09.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.599+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd69c06c9c0 msgr2=0x7fd69c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:09.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.599+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd69c06c9c0 0x7fd69c06ee80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6b41723c0 tx=0x7fd6ac009190 comp rx=0 tx=0).stop 2024-04-26T22:04:09.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.600+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 msgr2=0x7fd6b40dfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:09.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.600+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 0x7fd6b40dfb80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc051fc0 tx=0x7fd6bc0661d0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.601+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 shutdown_connections 2024-04-26T22:04:09.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.601+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd69c06c9c0 0x7fd69c06ee80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.601+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b4049e10 0x7fd6b4046cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.601+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6b404db50 0x7fd6b40e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.601+0000 7fd6bb59e640 1 --2- 172.21.15.135:0/502034546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b400e9c0 0x7fd6b40dfb80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:09.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.601+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 >> 172.21.15.135:0/502034546 conn(0x7fd6b404c660 msgr2=0x7fd6b40e17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:09.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.602+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 shutdown_connections 2024-04-26T22:04:09.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:09.602+0000 7fd6bb59e640 1 -- 172.21.15.135:0/502034546 wait complete. 2024-04-26T22:04:09.628 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:04:11.628 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:11.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.830+0000 7fdc7195f640 1 Processor -- start 2024-04-26T22:04:11.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.830+0000 7fdc7195f640 1 -- start start 2024-04-26T22:04:11.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.831+0000 7fdc7195f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdc640d3c40 0x7fdc6404ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.831+0000 7fdc7195f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc6404b1b0 0x7fdc640d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.831+0000 7fdc7195f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc640468d0 con 0x7fdc640d6fa0 2024-04-26T22:04:11.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.831+0000 7fdc7195f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdc64047660 con 0x7fdc640d3c40 2024-04-26T22:04:11.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.831+0000 7fdc7195f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc640d65d0 con 0x7fdc6404b1b0 2024-04-26T22:04:11.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.832+0000 7fdc6b7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fdc640d6fa0 0x7fdc640d99a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:45890/0 (socket says 172.21.15.135:45890) 2024-04-26T22:04:11.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.832+0000 7fdc6b7fe640 1 -- 172.21.15.135:0/1700516995 learned_addr learned my addr 172.21.15.135:0/1700516995 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:11.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.832+0000 7fdc6affd640 1 --2- 172.21.15.135:0/1700516995 >> v2:172.21.15.135:3300/0 conn(0x7fdc640d3c40 0x7fdc6404ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.832+0000 7fdc6a7fc640 1 --2- 172.21.15.135:0/1700516995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc6404b1b0 0x7fdc640d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.833+0000 7fdc6affd640 1 -- 172.21.15.135:0/1700516995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc6404b1b0 msgr2=0x7fdc640d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:11.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.833+0000 7fdc6affd640 1 --2- 172.21.15.135:0/1700516995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc6404b1b0 0x7fdc640d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.833+0000 7fdc6affd640 1 -- 172.21.15.135:0/1700516995 >> v1:172.21.15.135:6789/0 conn(0x7fdc640d6fa0 legacy=0x7fdc640d99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.833+0000 7fdc6affd640 1 -- 172.21.15.135:0/1700516995 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64046bb0 con 0x7fdc640d3c40 2024-04-26T22:04:11.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.834+0000 7fdc6affd640 1 --2- 172.21.15.135:0/1700516995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc640d3c40 0x7fdc6404ac70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0097a0 tx=0x7fdc5c02c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4fe63fbdef2165 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.835+0000 7fdc69ffb640 1 -- 172.21.15.135:0/1700516995 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c039070 con 0x7fdc640d3c40 2024-04-26T22:04:11.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.835+0000 7fdc69ffb640 1 -- 172.21.15.135:0/1700516995 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc5c02cdf0 con 0x7fdc640d3c40 2024-04-26T22:04:11.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.835+0000 7fdc69ffb640 1 -- 172.21.15.135:0/1700516995 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c034660 con 0x7fdc640d3c40 2024-04-26T22:04:11.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.836+0000 7fdc7195f640 1 -- 172.21.15.135:0/1700516995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc640d3c40 msgr2=0x7fdc6404ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.836+0000 7fdc7195f640 1 --2- 172.21.15.135:0/1700516995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc640d3c40 0x7fdc6404ac70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0097a0 tx=0x7fdc5c02c160 comp rx=0 tx=0).stop 2024-04-26T22:04:11.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.837+0000 7fdc7195f640 1 -- 172.21.15.135:0/1700516995 shutdown_connections 2024-04-26T22:04:11.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.837+0000 7fdc7195f640 1 --2- 172.21.15.135:0/1700516995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc6404b1b0 0x7fdc640d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.837+0000 7fdc7195f640 1 --2- 172.21.15.135:0/1700516995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc640d3c40 0x7fdc6404ac70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.837+0000 7fdc7195f640 1 -- 172.21.15.135:0/1700516995 >> 172.21.15.135:0/1700516995 conn(0x7fdc64005e90 msgr2=0x7fdc640062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:11.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.837+0000 7fdc7195f640 1 -- 172.21.15.135:0/1700516995 shutdown_connections 2024-04-26T22:04:11.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.837+0000 7fdc7195f640 1 -- 172.21.15.135:0/1700516995 wait complete. 2024-04-26T22:04:11.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.839+0000 7fdc7195f640 1 Processor -- start 2024-04-26T22:04:11.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.841+0000 7fdc7195f640 1 -- start start 2024-04-26T22:04:11.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.842+0000 7fdc7195f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc6404b1b0 0x7fdc6402bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.842+0000 7fdc7195f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 0x7fdc641702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.842+0000 7fdc7195f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 0x7fdc64170830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.842+0000 7fdc7195f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64046360 con 0x7fdc6404b1b0 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.842+0000 7fdc7195f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc640461e0 con 0x7fdc640d3c40 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.842+0000 7fdc7195f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc64046e90 con 0x7fdc640d6fa0 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.843+0000 7fdc6a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 0x7fdc641702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.843+0000 7fdc6b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 0x7fdc64170830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.843+0000 7fdc6b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 0x7fdc64170830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46704/0 (socket says 172.21.15.135:46704) 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.843+0000 7fdc6a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 0x7fdc641702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:51720/0 (socket says 172.21.15.135:51720) 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.843+0000 7fdc6affd640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc6404b1b0 0x7fdc6402bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.843+0000 7fdc6b7fe640 1 -- 172.21.15.135:0/626781 learned_addr learned my addr 172.21.15.135:0/626781 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:11.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.844+0000 7fdc6b7fe640 1 -- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 msgr2=0x7fdc641702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.844+0000 7fdc6b7fe640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 0x7fdc641702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.844+0000 7fdc6b7fe640 1 -- 172.21.15.135:0/626781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc6404b1b0 msgr2=0x7fdc6402bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:11.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.844+0000 7fdc6b7fe640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc6404b1b0 0x7fdc6402bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:11.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.844+0000 7fdc6b7fe640 1 -- 172.21.15.135:0/626781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64044c50 con 0x7fdc640d6fa0 2024-04-26T22:04:11.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.845+0000 7fdc6a7fc640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 0x7fdc641702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:11.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.845+0000 7fdc6affd640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc6404b1b0 0x7fdc6402bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:11.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.847+0000 7fdc6b7fe640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 0x7fdc64170830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c06b030 tx=0x7fdc6c069f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:11.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.848+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c06f840 con 0x7fdc640d6fa0 2024-04-26T22:04:11.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.848+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc64173d20 con 0x7fdc640d6fa0 2024-04-26T22:04:11.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.848+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc640d0fb0 con 0x7fdc640d6fa0 2024-04-26T22:04:11.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.849+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c06f9e0 con 0x7fdc640d6fa0 2024-04-26T22:04:11.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.849+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c06fd90 con 0x7fdc640d6fa0 2024-04-26T22:04:11.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.851+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc6c071420 con 0x7fdc640d6fa0 2024-04-26T22:04:11.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.852+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c005290 con 0x7fdc640d6fa0 2024-04-26T22:04:11.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.853+0000 7fdc53fff640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdc5806c940 0x7fdc5806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:11.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.853+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 7010+0+0 (secure 0 0 0) 0x7fdc6c108130 con 0x7fdc640d6fa0 2024-04-26T22:04:11.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.854+0000 7fdc6affd640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdc5806c940 0x7fdc5806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:11.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.858+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc64044c50 con 0x7fdc640d6fa0 2024-04-26T22:04:11.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:11.858+0000 7fdc6affd640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdc5806c940 0x7fdc5806ee00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c036000 tx=0x7fdc5c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:12.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.377+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc4c005580 con 0x7fdc640d6fa0 2024-04-26T22:04:12.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.379+0000 7fdc53fff640 1 -- 172.21.15.135:0/626781 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6500 (secure 0 0 0) 0x7fdc6c0d5a60 con 0x7fdc640d6fa0 2024-04-26T22:04:12.385 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:12.385 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:03:41.405296+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":31,"addr":"172.21.15.135:6837/3652200818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3652200818},{"type":"v1","addr":"172.21.15.135:6837","nonce":3652200818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086]},"id":2}]} 2024-04-26T22:04:12.386 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 33 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdc5806c940 msgr2=0x7fdc5806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdc5806c940 0x7fdc5806ee00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c036000 tx=0x7fdc5c030000 comp rx=0 tx=0).stop 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 msgr2=0x7fdc64170830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 0x7fdc64170830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c06b030 tx=0x7fdc6c069f90 comp rx=0 tx=0).stop 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 shutdown_connections 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdc5806c940 0x7fdc5806ee00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc640d6fa0 0x7fdc64170830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc640d3c40 0x7fdc641702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 --2- 172.21.15.135:0/626781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc6404b1b0 0x7fdc6402bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:12.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.388+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 >> 172.21.15.135:0/626781 conn(0x7fdc64005e90 msgr2=0x7fdc64030aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:12.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.389+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 shutdown_connections 2024-04-26T22:04:12.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:12.389+0000 7fdc7195f640 1 -- 172.21.15.135:0/626781 wait complete. 2024-04-26T22:04:12.410 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:04:14.412 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:14.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.598+0000 7f08d881d640 1 Processor -- start 2024-04-26T22:04:14.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.601+0000 7f08d881d640 1 -- start start 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.602+0000 7f08d881d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f08d40abfc0 0x7f08d415e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.602+0000 7f08d881d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f08d40c3750 0x7f08d415da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.602+0000 7f08d881d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d415edd0 con 0x7f08d415dfa0 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.602+0000 7f08d881d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f08d415eba0 con 0x7f08d40c3750 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.602+0000 7f08d881d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f08d406e3d0 con 0x7f08d40abfc0 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.603+0000 7f08d259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f08d40abfc0 0x7f08d415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.603+0000 7f08d259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f08d40abfc0 0x7f08d415e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:51730/0 (socket says 172.21.15.135:51730) 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.603+0000 7f08d259c640 1 -- 172.21.15.135:0/2874676331 learned_addr learned my addr 172.21.15.135:0/2874676331 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.604+0000 7f08d259c640 1 -- 172.21.15.135:0/2874676331 >> v2:172.21.15.135:3300/0 conn(0x7f08d40c3750 msgr2=0x7f08d415da60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:14.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.604+0000 7f08d259c640 1 --2- 172.21.15.135:0/2874676331 >> v2:172.21.15.135:3300/0 conn(0x7f08d40c3750 0x7f08d415da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.604+0000 7f08d259c640 1 -- 172.21.15.135:0/2874676331 >> v1:172.21.15.139:6789/0 conn(0x7f08d415dfa0 legacy=0x7f08d406eb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:14.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.604+0000 7f08d259c640 1 -- 172.21.15.135:0/2874676331 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d415f0b0 con 0x7f08d40abfc0 2024-04-26T22:04:14.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.608+0000 7f08d259c640 1 --2- 172.21.15.135:0/2874676331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d415e660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08d406fa00 tx=0x7f08c802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c7c4492532cf57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.618+0000 7f08d159a640 1 -- 172.21.15.135:0/2874676331 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8037070 con 0x7f08d40abfc0 2024-04-26T22:04:14.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.620+0000 7f08d159a640 1 -- 172.21.15.135:0/2874676331 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c8032440 con 0x7f08d40abfc0 2024-04-26T22:04:14.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.621+0000 7f08d881d640 1 -- 172.21.15.135:0/2874676331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 msgr2=0x7f08d415e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.621+0000 7f08d881d640 1 --2- 172.21.15.135:0/2874676331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d415e660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08d406fa00 tx=0x7f08c802a250 comp rx=0 tx=0).stop 2024-04-26T22:04:14.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.621+0000 7f08d881d640 1 -- 172.21.15.135:0/2874676331 shutdown_connections 2024-04-26T22:04:14.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.621+0000 7f08d881d640 1 --2- 172.21.15.135:0/2874676331 >> v2:172.21.15.135:3300/0 conn(0x7f08d40c3750 0x7f08d415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.621+0000 7f08d881d640 1 --2- 172.21.15.135:0/2874676331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d415e660 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.621+0000 7f08d881d640 1 -- 172.21.15.135:0/2874676331 >> 172.21.15.135:0/2874676331 conn(0x7f08d406c310 msgr2=0x7f08d406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.639+0000 7f08d881d640 1 -- 172.21.15.135:0/2874676331 shutdown_connections 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.646+0000 7f08d881d640 1 -- 172.21.15.135:0/2874676331 wait complete. 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.648+0000 7f08d881d640 1 Processor -- start 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.648+0000 7f08d881d640 1 -- start start 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.649+0000 7f08d881d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d4152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.649+0000 7f08d881d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08d40c3750 0x7f08d40c1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.649+0000 7f08d881d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 0x7f08d41f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.649+0000 7f08d881d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08d415e3b0 con 0x7f08d415dfa0 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.649+0000 7f08d881d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08d406e620 con 0x7f08d40abfc0 2024-04-26T22:04:14.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.649+0000 7f08d881d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f08d415c560 con 0x7f08d40c3750 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.650+0000 7f08d2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 0x7f08d41f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.650+0000 7f08d2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 0x7f08d41f0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39994/0 (socket says 172.21.15.135:39994) 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.650+0000 7f08d2d9d640 1 -- 172.21.15.135:0/3136573686 learned_addr learned my addr 172.21.15.135:0/3136573686 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.650+0000 7f08d259c640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d4152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.651+0000 7f08d2d9d640 1 -- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08d40c3750 msgr2=0x7f08d40c1290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.651+0000 7f08d2d9d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08d40c3750 0x7f08d40c1290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.651+0000 7f08d2d9d640 1 -- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 msgr2=0x7f08d4152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.651+0000 7f08d2d9d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d4152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.651+0000 7f08d2d9d640 1 -- 172.21.15.135:0/3136573686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d40743a0 con 0x7f08d415dfa0 2024-04-26T22:04:14.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.651+0000 7f08d2d9d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 0x7f08d41f0220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00bd60 tx=0x7f08cc009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.652+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc019070 con 0x7f08d415dfa0 2024-04-26T22:04:14.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.652+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d4074680 con 0x7f08d415dfa0 2024-04-26T22:04:14.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.652+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d4155b70 con 0x7f08d415dfa0 2024-04-26T22:04:14.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.653+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc00cc00 con 0x7f08d415dfa0 2024-04-26T22:04:14.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.653+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc014440 con 0x7f08d415dfa0 2024-04-26T22:04:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.659+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08cc01c070 con 0x7f08d415dfa0 2024-04-26T22:04:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.660+0000 7f08c2ffd640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f08b406c9c0 0x7f08b406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.660+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f08cc08be00 con 0x7f08d415dfa0 2024-04-26T22:04:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.661+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b8005290 con 0x7f08d415dfa0 2024-04-26T22:04:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.661+0000 7f08d259c640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f08b406c9c0 0x7f08b406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.662+0000 7f08d259c640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f08b406c9c0 0x7f08b406ee80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08c8034000 tx=0x7f08c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:14.670+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08cc059d30 con 0x7f08d415dfa0 2024-04-26T22:04:15.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.171+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08b8005580 con 0x7f08d415dfa0 2024-04-26T22:04:15.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.173+0000 7f08c2ffd640 1 -- 172.21.15.135:0/3136573686 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5664 (secure 0 0 0) 0x7f08cc059630 con 0x7f08d415dfa0 2024-04-26T22:04:15.174 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:15.174 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:12.801355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:04:15.174 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 34 2024-04-26T22:04:15.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.178+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f08b406c9c0 msgr2=0x7f08b406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.178+0000 7f08d881d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f08b406c9c0 0x7f08b406ee80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08c8034000 tx=0x7f08c802e000 comp rx=0 tx=0).stop 2024-04-26T22:04:15.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.178+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 msgr2=0x7f08d41f0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.178+0000 7f08d881d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 0x7f08d41f0220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00bd60 tx=0x7f08cc009e70 comp rx=0 tx=0).stop 2024-04-26T22:04:15.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.180+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 shutdown_connections 2024-04-26T22:04:15.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.180+0000 7f08d881d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f08b406c9c0 0x7f08b406ee80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.180+0000 7f08d881d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08d415dfa0 0x7f08d41f0220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.180+0000 7f08d881d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08d40c3750 0x7f08d40c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.180+0000 7f08d881d640 1 --2- 172.21.15.135:0/3136573686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d40abfc0 0x7f08d4152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.180+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 >> 172.21.15.135:0/3136573686 conn(0x7f08d406c310 msgr2=0x7f08d415d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:15.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.181+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 shutdown_connections 2024-04-26T22:04:15.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.181+0000 7f08d881d640 1 -- 172.21.15.135:0/3136573686 wait complete. 2024-04-26T22:04:15.200 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-04-26T22:04:15.200 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:04:15.200 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:15.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.404+0000 7f46f09ec640 1 Processor -- start 2024-04-26T22:04:15.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.405+0000 7f46f09ec640 1 -- start start 2024-04-26T22:04:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.406+0000 7f46f09ec640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46ec150300 0x7f46ec152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.406+0000 7f46f09ec640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f46ec0ab380 0x7f46ec0c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.406+0000 7f46f09ec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0c4c90 0x7f46ec0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.406+0000 7f46f09ec640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f46ec06e3d0 con 0x7f46ec150300 2024-04-26T22:04:15.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.406+0000 7f46f09ec640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f46ec06e6b0 con 0x7f46ec0ab380 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.406+0000 7f46f09ec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec0c3b80 con 0x7f46ec0c4c90 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ea59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46ec150300 0x7f46ec152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ea59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46ec150300 0x7f46ec152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39998/0 (socket says 172.21.15.135:39998) 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ea59c640 1 -- 172.21.15.135:0/2963750748 learned_addr learned my addr 172.21.15.135:0/2963750748 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ead9d640 1 --2- 172.21.15.135:0/2963750748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0c4c90 0x7f46ec0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ead9d640 1 -- 172.21.15.135:0/2963750748 >> v2:172.21.15.139:3300/0 conn(0x7f46ec0ab380 msgr2=0x7f46ec0c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ead9d640 1 --2- 172.21.15.135:0/2963750748 >> v2:172.21.15.139:3300/0 conn(0x7f46ec0ab380 0x7f46ec0c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ead9d640 1 -- 172.21.15.135:0/2963750748 >> v2:172.21.15.135:3300/0 conn(0x7f46ec150300 msgr2=0x7f46ec152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.407+0000 7f46ead9d640 1 --2- 172.21.15.135:0/2963750748 >> v2:172.21.15.135:3300/0 conn(0x7f46ec150300 0x7f46ec152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.408+0000 7f46ead9d640 1 -- 172.21.15.135:0/2963750748 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec0c3e00 con 0x7f46ec0c4c90 2024-04-26T22:04:15.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.410+0000 7f46ead9d640 1 --2- 172.21.15.135:0/2963750748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0c4c90 0x7f46ec0c5120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0718e0 tx=0x7f46e402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2222c299ab47a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.415+0000 7f46e9d9b640 1 -- 172.21.15.135:0/2963750748 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4007740 con 0x7f46ec0c4c90 2024-04-26T22:04:15.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.415+0000 7f46e9d9b640 1 -- 172.21.15.135:0/2963750748 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e40078e0 con 0x7f46ec0c4c90 2024-04-26T22:04:15.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.415+0000 7f46e9d9b640 1 -- 172.21.15.135:0/2963750748 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4007cf0 con 0x7f46ec0c4c90 2024-04-26T22:04:15.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.416+0000 7f46f09ec640 1 -- 172.21.15.135:0/2963750748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0c4c90 msgr2=0x7f46ec0c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.416+0000 7f46f09ec640 1 --2- 172.21.15.135:0/2963750748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0c4c90 0x7f46ec0c5120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0718e0 tx=0x7f46e402a5f0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.417+0000 7f46f09ec640 1 -- 172.21.15.135:0/2963750748 shutdown_connections 2024-04-26T22:04:15.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.417+0000 7f46f09ec640 1 --2- 172.21.15.135:0/2963750748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0c4c90 0x7f46ec0c5120 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.417+0000 7f46f09ec640 1 --2- 172.21.15.135:0/2963750748 >> v2:172.21.15.139:3300/0 conn(0x7f46ec0ab380 0x7f46ec0c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.417+0000 7f46f09ec640 1 --2- 172.21.15.135:0/2963750748 >> v2:172.21.15.135:3300/0 conn(0x7f46ec150300 0x7f46ec152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.417+0000 7f46f09ec640 1 -- 172.21.15.135:0/2963750748 >> 172.21.15.135:0/2963750748 conn(0x7f46ec06c310 msgr2=0x7f46ec06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:15.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.418+0000 7f46f09ec640 1 -- 172.21.15.135:0/2963750748 shutdown_connections 2024-04-26T22:04:15.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.421+0000 7f46f09ec640 1 -- 172.21.15.135:0/2963750748 wait complete. 2024-04-26T22:04:15.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.422+0000 7f46f09ec640 1 Processor -- start 2024-04-26T22:04:15.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.422+0000 7f46f09ec640 1 -- start start 2024-04-26T22:04:15.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.423+0000 7f46f09ec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0ab380 0x7f46ec0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.423+0000 7f46f09ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ec0c4c90 0x7f46ec153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.423+0000 7f46f09ec640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 0x7f46ec141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.424+0000 7f46ead9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 0x7f46ec141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.424+0000 7f46ead9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 0x7f46ec141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40006/0 (socket says 172.21.15.135:40006) 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.424+0000 7f46ead9d640 1 -- 172.21.15.135:0/1773584077 learned_addr learned my addr 172.21.15.135:0/1773584077 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.424+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0c43c0 con 0x7f46ec150300 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.424+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec06f3b0 con 0x7f46ec0c4c90 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.424+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec1506e0 con 0x7f46ec0ab380 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.425+0000 7f46ead9d640 1 -- 172.21.15.135:0/1773584077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0ab380 msgr2=0x7f46ec0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.425+0000 7f46ead9d640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0ab380 0x7f46ec0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.425+0000 7f46ead9d640 1 -- 172.21.15.135:0/1773584077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ec0c4c90 msgr2=0x7f46ec153420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:15.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.425+0000 7f46ead9d640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ec0c4c90 0x7f46ec153420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.425+0000 7f46ead9d640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec145430 con 0x7f46ec150300 2024-04-26T22:04:15.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.426+0000 7f46ead9d640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 0x7f46ec141bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009e80 tx=0x7f46e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.427+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4040070 con 0x7f46ec150300 2024-04-26T22:04:15.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.427+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4031c30 con 0x7f46ec150300 2024-04-26T22:04:15.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.427+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4045460 con 0x7f46ec150300 2024-04-26T22:04:15.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.427+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec145710 con 0x7f46ec150300 2024-04-26T22:04:15.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.427+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec145cd0 con 0x7f46ec150300 2024-04-26T22:04:15.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.429+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d0005290 con 0x7f46ec150300 2024-04-26T22:04:15.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.432+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46e4039040 con 0x7f46ec150300 2024-04-26T22:04:15.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.433+0000 7f46e2ffd640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46cc06c9c0 0x7f46cc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.433+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f46e40ad8b0 con 0x7f46ec150300 2024-04-26T22:04:15.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.433+0000 7f46ea59c640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46cc06c9c0 0x7f46cc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.434+0000 7f46ea59c640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46cc06c9c0 0x7f46cc06ee80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46d40094a0 tx=0x7f46d40093f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.445+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46e407b7e0 con 0x7f46ec150300 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:12.801355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 34 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.971+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d0005580 con 0x7f46ec150300 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.972+0000 7f46e2ffd640 1 -- 172.21.15.135:0/1773584077 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5664 (secure 0 0 0) 0x7f46e407b0e0 con 0x7f46ec150300 2024-04-26T22:04:15.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.980+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46cc06c9c0 msgr2=0x7f46cc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.980+0000 7f46f09ec640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46cc06c9c0 0x7f46cc06ee80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46d40094a0 tx=0x7f46d40093f0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.980+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 msgr2=0x7f46ec141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.980+0000 7f46f09ec640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 0x7f46ec141bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009e80 tx=0x7f46e4004a40 comp rx=0 tx=0).stop 2024-04-26T22:04:15.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 shutdown_connections 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f46cc06c9c0 0x7f46cc06ee80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46ec150300 0x7f46ec141bf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ec0c4c90 0x7f46ec153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 --2- 172.21.15.135:0/1773584077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46ec0ab380 0x7f46ec0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 >> 172.21.15.135:0/1773584077 conn(0x7f46ec06c310 msgr2=0x7f46ec06ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 shutdown_connections 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:15.989+0000 7f46f09ec640 1 -- 172.21.15.135:0/1773584077 wait complete. 2024-04-26T22:04:16.011 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5268": { "addr": "172.21.15.135:6835/2910412510", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 2910412510, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 2910412510, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5268, "incarnation": 24, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:03:15.024868+0000", "qdb_cluster": [ 5268 ], "qdb_leader": 5268, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5268 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4973": { "addr": "172.21.15.139:6835/302589348", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 302589348, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 302589348, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4973, "incarnation": 34, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:04:12.801355+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4973 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:04:18.012 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:18.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.224+0000 7f8fa9b89640 1 Processor -- start 2024-04-26T22:04:18.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.224+0000 7f8fa9b89640 1 -- start start 2024-04-26T22:04:18.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.225+0000 7f8fa9b89640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8fa4151fb0 0x7f8fa406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.225+0000 7f8fa9b89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 0x7f8fa40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.225+0000 7f8fa9b89640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8fa40c7dc0 0x7f8fa40c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.225+0000 7f8fa9b89640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa41549b0 con 0x7f8fa40c7dc0 2024-04-26T22:04:18.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.225+0000 7f8fa9b89640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa4158ce0 con 0x7f8fa4151fb0 2024-04-26T22:04:18.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.225+0000 7f8fa9b89640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa4154c70 con 0x7f8fa4155750 2024-04-26T22:04:18.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.227+0000 7f8fa2ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 0x7f8fa40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:18.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.227+0000 7f8fa2ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 0x7f8fa40c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46738/0 (socket says 172.21.15.135:46738) 2024-04-26T22:04:18.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.227+0000 7f8fa2ffd640 1 -- 172.21.15.135:0/1351749359 learned_addr learned my addr 172.21.15.135:0/1351749359 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa2ffd640 1 -- 172.21.15.135:0/1351749359 >> v2:172.21.15.139:3300/0 conn(0x7f8fa4151fb0 msgr2=0x7f8fa406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa2ffd640 1 --2- 172.21.15.135:0/1351749359 >> v2:172.21.15.139:3300/0 conn(0x7f8fa4151fb0 0x7f8fa406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa2ffd640 1 -- 172.21.15.135:0/1351749359 >> v2:172.21.15.135:3300/0 conn(0x7f8fa40c7dc0 msgr2=0x7f8fa40c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa3fff640 1 --2- 172.21.15.135:0/1351749359 >> v2:172.21.15.135:3300/0 conn(0x7f8fa40c7dc0 0x7f8fa40c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa2ffd640 1 --2- 172.21.15.135:0/1351749359 >> v2:172.21.15.135:3300/0 conn(0x7f8fa40c7dc0 0x7f8fa40c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa2ffd640 1 -- 172.21.15.135:0/1351749359 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa4154f50 con 0x7f8fa4155750 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.228+0000 7f8fa3fff640 1 --2- 172.21.15.135:0/1351749359 >> v2:172.21.15.135:3300/0 conn(0x7f8fa40c7dc0 0x7f8fa40c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:18.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.229+0000 7f8fa2ffd640 1 --2- 172.21.15.135:0/1351749359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 0x7f8fa40c3be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40c6c20 tx=0x7f8f9802a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5abc8ff2b82c58ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.230+0000 7f8fa27fc640 1 -- 172.21.15.135:0/1351749359 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98037070 con 0x7f8fa4155750 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.230+0000 7f8fa27fc640 1 -- 172.21.15.135:0/1351749359 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f98032440 con 0x7f8fa4155750 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.231+0000 7f8fa9b89640 1 -- 172.21.15.135:0/1351749359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 msgr2=0x7f8fa40c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.231+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/1351749359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 0x7f8fa40c3be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40c6c20 tx=0x7f8f9802a160 comp rx=0 tx=0).stop 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.232+0000 7f8fa9b89640 1 -- 172.21.15.135:0/1351749359 shutdown_connections 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.232+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/1351749359 >> v2:172.21.15.135:3300/0 conn(0x7f8fa40c7dc0 0x7f8fa40c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.232+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/1351749359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4155750 0x7f8fa40c3be0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40c6c20 tx=0x7f8f9802a160 comp rx=0 tx=0).stop 2024-04-26T22:04:18.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.232+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/1351749359 >> v2:172.21.15.139:3300/0 conn(0x7f8fa4151fb0 0x7f8fa406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.232+0000 7f8fa9b89640 1 -- 172.21.15.135:0/1351749359 >> 172.21.15.135:0/1351749359 conn(0x7f8fa40c5ce0 msgr2=0x7f8fa40c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:18.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.235+0000 7f8fa9b89640 1 -- 172.21.15.135:0/1351749359 shutdown_connections 2024-04-26T22:04:18.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.235+0000 7f8fa9b89640 1 -- 172.21.15.135:0/1351749359 wait complete. 2024-04-26T22:04:18.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.237+0000 7f8fa9b89640 1 Processor -- start 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.237+0000 7f8fa9b89640 1 -- start start 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.238+0000 7f8fa9b89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa40c7dc0 0x7f8fa41501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.238+0000 7f8fa9b89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fa4151fb0 0x7f8fa4156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.238+0000 7f8fa9b89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 0x7f8fa4145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.238+0000 7f8fa9b89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41589b0 con 0x7f8fa4151fb0 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.238+0000 7f8fa9b89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4152390 con 0x7f8fa40c7dc0 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.238+0000 7f8fa9b89640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa406d860 con 0x7f8fa4157800 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.239+0000 7f8fa3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 0x7f8fa4145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.239+0000 7f8fa37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa40c7dc0 0x7f8fa41501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.239+0000 7f8fa3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 0x7f8fa4145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46746/0 (socket says 172.21.15.135:46746) 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.239+0000 7f8fa3fff640 1 -- 172.21.15.135:0/811902188 learned_addr learned my addr 172.21.15.135:0/811902188 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.239+0000 7f8fa2ffd640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fa4151fb0 0x7f8fa4156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa3fff640 1 -- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa40c7dc0 msgr2=0x7f8fa41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:18.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa3fff640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa40c7dc0 0x7f8fa41501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa3fff640 1 -- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fa4151fb0 msgr2=0x7f8fa4156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:18.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa3fff640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fa4151fb0 0x7f8fa4156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa3fff640 1 -- 172.21.15.135:0/811902188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa41506f0 con 0x7f8fa4157800 2024-04-26T22:04:18.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa2ffd640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fa4151fb0 0x7f8fa4156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:18.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.240+0000 7f8fa3fff640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 0x7f8fa4145020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00cef0 tx=0x7f8f9c00a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:18.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.241+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c02b070 con 0x7f8fa4157800 2024-04-26T22:04:18.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.241+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa41509c0 con 0x7f8fa4157800 2024-04-26T22:04:18.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.241+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa40adea0 con 0x7f8fa4157800 2024-04-26T22:04:18.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.243+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f9c01e870 con 0x7f8fa4157800 2024-04-26T22:04:18.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.243+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c01ec60 con 0x7f8fa4157800 2024-04-26T22:04:18.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.243+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa4150ee0 con 0x7f8fa4157800 2024-04-26T22:04:18.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.245+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f9c02e070 con 0x7f8fa4157800 2024-04-26T22:04:18.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.246+0000 7f8f8bfff640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f9006c8f0 0x7f8f9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:18.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.246+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f8f9c09d8d0 con 0x7f8fa4157800 2024-04-26T22:04:18.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.253+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f9c06b1b0 con 0x7f8fa4157800 2024-04-26T22:04:18.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.258+0000 7f8fa37fe640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f9006c8f0 0x7f8f9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.263+0000 7f8fa37fe640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f9006c8f0 0x7f8f9006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400d440 tx=0x7f8f9400f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:18.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.716+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fa40507a0 con 0x7f8fa4157800 2024-04-26T22:04:18.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.717+0000 7f8f8bfff640 1 -- 172.21.15.135:0/811902188 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5664 (secure 0 0 0) 0x7f8f9c06f4b0 con 0x7f8fa4157800 2024-04-26T22:04:18.719 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:18.719 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:12.801355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:04:18.719 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 34 2024-04-26T22:04:18.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f9006c8f0 msgr2=0x7f8f9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:18.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f9006c8f0 0x7f8f9006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400d440 tx=0x7f8f9400f040 comp rx=0 tx=0).stop 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 msgr2=0x7f8fa4145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 0x7f8fa4145020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00cef0 tx=0x7f8f9c00a710 comp rx=0 tx=0).stop 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 shutdown_connections 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8f9006c8f0 0x7f8f9006edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fa4157800 0x7f8fa4145020 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fa4151fb0 0x7f8fa4156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 --2- 172.21.15.135:0/811902188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fa40c7dc0 0x7f8fa41501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.729+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 >> 172.21.15.135:0/811902188 conn(0x7f8fa40c5ce0 msgr2=0x7f8fa40c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.730+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 shutdown_connections 2024-04-26T22:04:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:18.730+0000 7f8fa9b89640 1 -- 172.21.15.135:0/811902188 wait complete. 2024-04-26T22:04:18.749 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:04:20.750 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:20.897 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:20.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.977+0000 7f4bf2866640 1 Processor -- start 2024-04-26T22:04:20.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.978+0000 7f4bf2866640 1 -- start start 2024-04-26T22:04:20.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.979+0000 7f4bf2866640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d6ee0 0x7f4be40e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.979+0000 7f4bf2866640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be40468d0 con 0x7f4be40d3c40 2024-04-26T22:04:20.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.979+0000 7f4bf2866640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be4046bb0 con 0x7f4be40d6a60 2024-04-26T22:04:20.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.979+0000 7f4bf2866640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4be4047660 con 0x7f4be40d6ee0 2024-04-26T22:04:20.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.980+0000 7f4bf1864640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4be40d3c40 0x7f4be404ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38280/0 (socket says 172.21.15.135:38280) 2024-04-26T22:04:20.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.980+0000 7f4bf1864640 1 -- 172.21.15.135:0/567450633 learned_addr learned my addr 172.21.15.135:0/567450633 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:20.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.980+0000 7f4bf2065640 1 --2- 172.21.15.135:0/567450633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d6ee0 0x7f4be40e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.981+0000 7f4bf2065640 1 -- 172.21.15.135:0/567450633 >> v1:172.21.15.139:6789/0 conn(0x7f4be40d6a60 legacy=0x7f4be40decc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:20.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.981+0000 7f4bf2065640 1 -- 172.21.15.135:0/567450633 >> v1:172.21.15.135:6789/0 conn(0x7f4be40d3c40 legacy=0x7f4be404ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.981+0000 7f4bf2065640 1 -- 172.21.15.135:0/567450633 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be40096c0 con 0x7f4be40d6ee0 2024-04-26T22:04:20.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.983+0000 7f4bf2065640 1 --2- 172.21.15.135:0/567450633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d6ee0 0x7f4be40e23f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0097d0 tx=0x7f4bdc029f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f24327fc659a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.985+0000 7f4bf0862640 1 -- 172.21.15.135:0/567450633 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc037070 con 0x7f4be40d6ee0 2024-04-26T22:04:20.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.985+0000 7f4bf0862640 1 -- 172.21.15.135:0/567450633 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bdc02ac10 con 0x7f4be40d6ee0 2024-04-26T22:04:20.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.986+0000 7f4bf2866640 1 -- 172.21.15.135:0/567450633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d6ee0 msgr2=0x7f4be40e23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.986+0000 7f4bf2866640 1 --2- 172.21.15.135:0/567450633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d6ee0 0x7f4be40e23f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0097d0 tx=0x7f4bdc029f80 comp rx=0 tx=0).stop 2024-04-26T22:04:20.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.987+0000 7f4bf2866640 1 -- 172.21.15.135:0/567450633 shutdown_connections 2024-04-26T22:04:20.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.987+0000 7f4bf2866640 1 --2- 172.21.15.135:0/567450633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d6ee0 0x7f4be40e23f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.987+0000 7f4bf2866640 1 -- 172.21.15.135:0/567450633 >> 172.21.15.135:0/567450633 conn(0x7f4be4005e90 msgr2=0x7f4be40062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.988+0000 7f4bf2866640 1 -- 172.21.15.135:0/567450633 shutdown_connections 2024-04-26T22:04:20.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.988+0000 7f4bf2866640 1 -- 172.21.15.135:0/567450633 wait complete. 2024-04-26T22:04:20.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.990+0000 7f4bf2866640 1 Processor -- start 2024-04-26T22:04:20.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.992+0000 7f4bf2866640 1 -- start start 2024-04-26T22:04:20.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.994+0000 7f4bf2866640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 0x7f4be40d9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.994+0000 7f4bf2866640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4be40d6a60 0x7f4be40406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.994+0000 7f4bf2866640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4be40d6ee0 0x7f4be40d2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf1864640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 0x7f4be40d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf1864640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 0x7f4be40d9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40736/0 (socket says 172.21.15.135:40736) 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf1864640 1 -- 172.21.15.135:0/1239726541 learned_addr learned my addr 172.21.15.135:0/1239726541 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf2065640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4be40d6ee0 0x7f4be40d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4be40d7370 con 0x7f4be40d6ee0 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4be40d65d0 con 0x7f4be40d6a60 2024-04-26T22:04:20.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.996+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4be4046e90 con 0x7f4be40d3c40 2024-04-26T22:04:20.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.997+0000 7f4bf1063640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4be40d6a60 0x7f4be40406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf2065640 1 -- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 msgr2=0x7f4be40d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf2065640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 0x7f4be40d9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf2065640 1 -- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4be40d6a60 msgr2=0x7f4be40406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf2065640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4be40d6a60 0x7f4be40406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf2065640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be41741b0 con 0x7f4be40d6ee0 2024-04-26T22:04:20.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf1063640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4be40d6a60 0x7f4be40406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:20.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf1864640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 0x7f4be40d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:20.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.998+0000 7f4bf2065640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4be40d6ee0 0x7f4be40d2950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc031000 tx=0x7f4bdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.999+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc037040 con 0x7f4be40d6ee0 2024-04-26T22:04:20.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.999+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bdc004270 con 0x7f4be40d6ee0 2024-04-26T22:04:20.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:20.999+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc030670 con 0x7f4be40d6ee0 2024-04-26T22:04:21.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.000+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be40da2c0 con 0x7f4be40d6ee0 2024-04-26T22:04:21.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.000+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be40e7f80 con 0x7f4be40d6ee0 2024-04-26T22:04:21.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.001+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be4005390 con 0x7f4be40d6ee0 2024-04-26T22:04:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.016+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bdc02abb0 con 0x7f4be40d6ee0 2024-04-26T22:04:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.017+0000 7f4be27fc640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4bd006c9c0 0x7f4bd006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.017+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f4bdc0ace40 con 0x7f4be40d6ee0 2024-04-26T22:04:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.017+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4bdc0b1470 con 0x7f4be40d6ee0 2024-04-26T22:04:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.018+0000 7f4bf1864640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4bd006c9c0 0x7f4bd006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.019+0000 7f4bf1864640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4bd006c9c0 0x7f4bd006ee80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06b030 tx=0x7f4bec06b630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.098+0000 7f9ef8916640 1 Processor -- start 2024-04-26T22:04:21.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.098+0000 7f9ef8916640 1 -- start start 2024-04-26T22:04:21.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.099+0000 7f9ef8916640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec047240 0x7f9eec0db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.100+0000 7f9ef8916640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eec0db540 con 0x7f9eec04c2e0 2024-04-26T22:04:21.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.100+0000 7f9ef8916640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eec04a250 con 0x7f9eec04bec0 2024-04-26T22:04:21.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.100+0000 7f9ef8916640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec04ba30 con 0x7f9eec047240 2024-04-26T22:04:21.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.100+0000 7f9ef3fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9eec04c2e0 0x7f9eec006ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38282/0 (socket says 172.21.15.135:38282) 2024-04-26T22:04:21.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.100+0000 7f9ef3fff640 1 -- 172.21.15.135:0/3530031488 learned_addr learned my addr 172.21.15.135:0/3530031488 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:21.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003965699 0 0) 0x7f9ee8001110 con 0x7f9eec04c2e0 2024-04-26T22:04:21.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ee4003520 con 0x7f9eec04c2e0 2024-04-26T22:04:21.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511244209 0 0) 0x7f9eec04a250 con 0x7f9eec04bec0 2024-04-26T22:04:21.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ee8001110 con 0x7f9eec04bec0 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4255904615 0 0) 0x7f9ee4003520 con 0x7f9eec04c2e0 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec047240 msgr2=0x7f9eec0db000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec047240 0x7f9eec0db000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 >> v1:172.21.15.139:6789/0 conn(0x7f9eec04bec0 legacy=0x7f9eec04a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec0476d0 con 0x7f9eec04c2e0 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f9ee80017c0 con 0x7f9eec04c2e0 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 >> v1:172.21.15.135:6789/0 conn(0x7f9eec04c2e0 legacy=0x7f9eec006ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ee4003b70 0x7f9ee4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee4004800 0x7f9ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee40099b0 0x7f9ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec04ba30 con 0x7f9ee4003b70 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.101+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4009da0 con 0x7f9ee40099b0 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee4009fa0 con 0x7f9ee4004800 2024-04-26T22:04:21.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef2ffd640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee4004800 0x7f9ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef2ffd640 1 -- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee40099b0 msgr2=0x7f9ee40051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:21.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef2ffd640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee40099b0 0x7f9ee40051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef2ffd640 1 -- 172.21.15.135:0/3530031488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ee4003b70 msgr2=0x7f9ee4003f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:21.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef2ffd640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ee4003b70 0x7f9ee4003f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.102+0000 7f9ef2ffd640 1 -- 172.21.15.135:0/3530031488 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec0476d0 con 0x7f9ee4004800 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.103+0000 7f9ef2ffd640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee4004800 0x7f9ee4004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ef406acf0 tx=0x7f9ee4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde15784b2ef2b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.103+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40515b0 con 0x7f9ee4004800 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.103+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef40ab070 con 0x7f9ee4004800 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.103+0000 7f9ef27fc640 1 -- 172.21.15.135:0/3530031488 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40af650 con 0x7f9ee4004800 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.104+0000 7f9ef8916640 1 -- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee4004800 msgr2=0x7f9ee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.104+0000 7f9ef8916640 1 --2- 172.21.15.135:0/3530031488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee4004800 0x7f9ee4004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9ef406acf0 tx=0x7f9ee4005830 comp rx=0 tx=0).stop 2024-04-26T22:04:21.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.104+0000 7f9ef3fff640 1 -- 172.21.15.135:0/3530031488 reap_dead start 2024-04-26T22:04:21.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.105+0000 7f9ef8916640 1 -- 172.21.15.135:0/3530031488 shutdown_connections 2024-04-26T22:04:21.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.105+0000 7f9ef8916640 1 -- 172.21.15.135:0/3530031488 >> 172.21.15.135:0/3530031488 conn(0x7f9eec049340 msgr2=0x7f9eec049750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:21.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.106+0000 7f9ef8916640 1 -- 172.21.15.135:0/3530031488 shutdown_connections 2024-04-26T22:04:21.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.107+0000 7f9ef8916640 1 -- 172.21.15.135:0/3530031488 wait complete. 2024-04-26T22:04:21.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.109+0000 7f9ef8916640 1 Processor -- start 2024-04-26T22:04:21.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.110+0000 7f9ef8916640 1 -- start start 2024-04-26T22:04:21.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.111+0000 7f9ef8916640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec044b90 0x7f9eec16fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.112+0000 7f9ef8916640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eec043f50 0x7f9eec007e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.112+0000 7f9ef8916640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 0x7f9eec031750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.112+0000 7f9ef8916640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec047c90 con 0x7f9eec0443b0 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.112+0000 7f9ef8916640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec04b6c0 con 0x7f9eec043f50 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.112+0000 7f9ef8916640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec04b0f0 con 0x7f9eec044b90 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 0x7f9eec031750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 0x7f9eec031750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42494/0 (socket says 172.21.15.135:42494) 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 -- 172.21.15.135:0/1726598320 learned_addr learned my addr 172.21.15.135:0/1726598320 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef37fe640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec044b90 0x7f9eec16fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef2ffd640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eec043f50 0x7f9eec007e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 -- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec044b90 msgr2=0x7f9eec16fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec044b90 0x7f9eec16fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 -- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eec043f50 msgr2=0x7f9eec007e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eec043f50 0x7f9eec007e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.113+0000 7f9ef3fff640 1 -- 172.21.15.135:0/1726598320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec008b70 con 0x7f9eec0443b0 2024-04-26T22:04:21.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.114+0000 7f9ef3fff640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 0x7f9eec031750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9eec04c750 tx=0x7f9ee800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.114+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8004d70 con 0x7f9eec0443b0 2024-04-26T22:04:21.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.114+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee8007110 con 0x7f9eec0443b0 2024-04-26T22:04:21.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.115+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8007500 con 0x7f9eec0443b0 2024-04-26T22:04:21.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.115+0000 7f9ef8916640 1 -- 172.21.15.135:0/1726598320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec0093d0 con 0x7f9eec0443b0 2024-04-26T22:04:21.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.115+0000 7f9ef8916640 1 -- 172.21.15.135:0/1726598320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec0090f0 con 0x7f9eec0443b0 2024-04-26T22:04:21.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.118+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ee8011350 con 0x7f9eec0443b0 2024-04-26T22:04:21.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.119+0000 7f9edffff640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ed806c970 0x7f9ed806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.119+0000 7f9ef37fe640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ed806c970 0x7f9ed806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.120+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f9ee80076a0 con 0x7f9eec0443b0 2024-04-26T22:04:21.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.121+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc005290 con 0x7f9eec0443b0 2024-04-26T22:04:21.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.124+0000 7f9ef37fe640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ed806c970 0x7f9ed806ee30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0006fd0 tx=0x7f9ee0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.131+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9eec0db540 con 0x7f9eec0443b0 2024-04-26T22:04:21.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.532+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4be4005390 con 0x7f4be40d6ee0 2024-04-26T22:04:21.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.534+0000 7f4be27fc640 1 -- 172.21.15.135:0/1239726541 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5665 (secure 0 0 0) 0x7f4bdc07ae00 con 0x7f4be40d6ee0 2024-04-26T22:04:21.537 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:21.537 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:18.852464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:04:21.537 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 36 2024-04-26T22:04:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4bd006c9c0 msgr2=0x7f4bd006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4bd006c9c0 0x7f4bd006ee80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06b030 tx=0x7f4bec06b630 comp rx=0 tx=0).stop 2024-04-26T22:04:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4be40d6ee0 msgr2=0x7f4be40d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4be40d6ee0 0x7f4be40d2950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc031000 tx=0x7f4bdc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 shutdown_connections 2024-04-26T22:04:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4bd006c9c0 0x7f4bd006ee80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4be40d6ee0 0x7f4be40d2950 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4be40d6a60 0x7f4be40406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 --2- 172.21.15.135:0/1239726541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4be40d3c40 0x7f4be40d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.539+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 >> 172.21.15.135:0/1239726541 conn(0x7f4be4005e90 msgr2=0x7f4be40069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:21.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.540+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 shutdown_connections 2024-04-26T22:04:21.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.541+0000 7f4bf2866640 1 -- 172.21.15.135:0/1239726541 wait complete. 2024-04-26T22:04:21.568 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:04:21.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.647+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ecc005850 con 0x7f9eec0443b0 2024-04-26T22:04:21.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.649+0000 7f9edffff640 1 -- 172.21.15.135:0/1726598320 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5665 (secure 0 0 0) 0x7f9ee8058460 con 0x7f9eec0443b0 2024-04-26T22:04:21.649 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:21.649 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:18.852464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:04:21.649 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 36 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ed806c970 msgr2=0x7f9ed806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ed806c970 0x7f9ed806ee30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0006fd0 tx=0x7f9ee0009060 comp rx=0 tx=0).stop 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 msgr2=0x7f9eec031750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 0x7f9eec031750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9eec04c750 tx=0x7f9ee800c960 comp rx=0 tx=0).stop 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 shutdown_connections 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ed806c970 0x7f9ed806ee30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec0443b0 0x7f9eec031750 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eec043f50 0x7f9eec007e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 --2- 172.21.15.135:0/1726598320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eec044b90 0x7f9eec16fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.652+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 >> 172.21.15.135:0/1726598320 conn(0x7f9eec049340 msgr2=0x7f9eec0ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:21.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.656+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 shutdown_connections 2024-04-26T22:04:21.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.657+0000 7f9eddffb640 1 -- 172.21.15.135:0/1726598320 wait complete. 2024-04-26T22:04:21.677 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-04-26T22:04:21.677 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:04:21.678 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:04:21.697 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:04:21.698 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:21.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.933+0000 7f47ca19d640 1 Processor -- start 2024-04-26T22:04:21.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.933+0000 7f47ca19d640 1 -- start start 2024-04-26T22:04:21.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.935+0000 7f47ca19d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47c40c3650 0x7f47c406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.935+0000 7f47ca19d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c40ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.935+0000 7f47ca19d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c415c0d0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.935+0000 7f47ca19d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f47c40ab4b0 con 0x7f47c40c3650 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.935+0000 7f47ca19d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47c415bdf0 con 0x7f47c40abf50 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.936+0000 7f47c3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c40ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.936+0000 7f47c37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47c40c3650 0x7f47c406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.936+0000 7f47c3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c40ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40770/0 (socket says 172.21.15.135:40770) 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.936+0000 7f47c3fff640 1 -- 172.21.15.135:0/2596100347 learned_addr learned my addr 172.21.15.135:0/2596100347 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.937+0000 7f47c3fff640 1 -- 172.21.15.135:0/2596100347 >> v2:172.21.15.139:3300/0 conn(0x7f47c40c3650 msgr2=0x7f47c406e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.937+0000 7f47c3fff640 1 --2- 172.21.15.135:0/2596100347 >> v2:172.21.15.139:3300/0 conn(0x7f47c40c3650 0x7f47c406e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.937+0000 7f47c3fff640 1 -- 172.21.15.135:0/2596100347 >> v1:172.21.15.135:6789/0 conn(0x7f47c40aa8c0 legacy=0x7f47c40c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.937+0000 7f47c3fff640 1 -- 172.21.15.135:0/2596100347 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c40ab750 con 0x7f47c40abf50 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.941+0000 7f47c3fff640 1 --2- 172.21.15.135:0/2596100347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c40ac3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009e50 tx=0x7f47bc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fa66e49d0955ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.944+0000 7f47c27fc640 1 -- 172.21.15.135:0/2596100347 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc036070 con 0x7f47c40abf50 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.944+0000 7f47c27fc640 1 -- 172.21.15.135:0/2596100347 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc031600 con 0x7f47c40abf50 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.944+0000 7f47c27fc640 1 -- 172.21.15.135:0/2596100347 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0319d0 con 0x7f47c40abf50 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.945+0000 7f47ca19d640 1 -- 172.21.15.135:0/2596100347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 msgr2=0x7f47c40ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.945+0000 7f47ca19d640 1 --2- 172.21.15.135:0/2596100347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c40ac3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009e50 tx=0x7f47bc02a5f0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.945+0000 7f47ca19d640 1 -- 172.21.15.135:0/2596100347 shutdown_connections 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.945+0000 7f47ca19d640 1 --2- 172.21.15.135:0/2596100347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c40ac3e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.945+0000 7f47ca19d640 1 --2- 172.21.15.135:0/2596100347 >> v2:172.21.15.139:3300/0 conn(0x7f47c40c3650 0x7f47c406e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.945+0000 7f47ca19d640 1 -- 172.21.15.135:0/2596100347 >> 172.21.15.135:0/2596100347 conn(0x7f47c40c82d0 msgr2=0x7f47c40c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:21.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.947+0000 7f47ca19d640 1 -- 172.21.15.135:0/2596100347 shutdown_connections 2024-04-26T22:04:21.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.947+0000 7f47ca19d640 1 -- 172.21.15.135:0/2596100347 wait complete. 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.949+0000 7f47ca19d640 1 Processor -- start 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.949+0000 7f47ca19d640 1 -- start start 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.950+0000 7f47ca19d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 0x7f47c4147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.950+0000 7f47ca19d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c41483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.950+0000 7f47ca19d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c40c3650 0x7f47c413c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.950+0000 7f47ca19d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47c40c42c0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.950+0000 7f47ca19d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47c40c4140 con 0x7f47c40c3650 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.950+0000 7f47ca19d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47c415a6a0 con 0x7f47c40abf50 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.951+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 0x7f47c4147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.951+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 0x7f47c4147d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42502/0 (socket says 172.21.15.135:42502) 2024-04-26T22:04:21.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.951+0000 7f47c37fe640 1 -- 172.21.15.135:0/3890151030 learned_addr learned my addr 172.21.15.135:0/3890151030 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47c37fe640 1 -- 172.21.15.135:0/3890151030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 msgr2=0x7f47c41483d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47c37fe640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c41483d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47c37fe640 1 -- 172.21.15.135:0/3890151030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c40c3650 msgr2=0x7f47c413c5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47c37fe640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c40c3650 0x7f47c413c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47c37fe640 1 -- 172.21.15.135:0/3890151030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c413dbc0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47c37fe640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 0x7f47c4147d80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47b800ebe0 tx=0x7f47b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8017070 con 0x7f47c40aa8c0 2024-04-26T22:04:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.952+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b800ecd0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.954+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800cdf0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.954+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c413dea0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.954+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c413e460 con 0x7f47c40aa8c0 2024-04-26T22:04:21.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.957+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c406fa30 con 0x7f47c40aa8c0 2024-04-26T22:04:21.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.960+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b801a070 con 0x7f47c40aa8c0 2024-04-26T22:04:21.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.961+0000 7f47abfff640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b006c9c0 0x7f47b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:21.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.961+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f47b808c6e0 con 0x7f47c40aa8c0 2024-04-26T22:04:21.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.963+0000 7f47c2ffd640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b006c9c0 0x7f47b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:21.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.964+0000 7f47c2ffd640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b006c9c0 0x7f47b006ee80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000f60 tx=0x7f47b40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:21.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:21.966+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47b805a610 con 0x7f47c40aa8c0 2024-04-26T22:04:22.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.494+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47c40507a0 con 0x7f47c40aa8c0 2024-04-26T22:04:22.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.496+0000 7f47abfff640 1 -- 172.21.15.135:0/3890151030 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5665 (secure 0 0 0) 0x7f47b8059f10 con 0x7f47c40aa8c0 2024-04-26T22:04:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:22.500 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:18.852464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:04:22.500 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 36 2024-04-26T22:04:22.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.507+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b006c9c0 msgr2=0x7f47b006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:22.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.507+0000 7f47ca19d640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b006c9c0 0x7f47b006ee80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000f60 tx=0x7f47b40079b0 comp rx=0 tx=0).stop 2024-04-26T22:04:22.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.507+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 msgr2=0x7f47c4147d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:22.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.507+0000 7f47ca19d640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 0x7f47c4147d80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47b800ebe0 tx=0x7f47b800c520 comp rx=0 tx=0).stop 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.507+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 shutdown_connections 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.507+0000 7f47ca19d640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b006c9c0 0x7f47b006ee80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.508+0000 7f47ca19d640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c40c3650 0x7f47c413c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.508+0000 7f47ca19d640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c40abf50 0x7f47c41483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.508+0000 7f47ca19d640 1 --2- 172.21.15.135:0/3890151030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47c40aa8c0 0x7f47c4147d80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.508+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 >> 172.21.15.135:0/3890151030 conn(0x7f47c40c82d0 msgr2=0x7f47c40ad290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.508+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 shutdown_connections 2024-04-26T22:04:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:22.508+0000 7f47ca19d640 1 -- 172.21.15.135:0/3890151030 wait complete. 2024-04-26T22:04:22.531 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:04:23.568 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:23.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.757+0000 7f4f6e6da640 1 Processor -- start 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.758+0000 7f4f6e6da640 1 -- start start 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6e6da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6806e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6e6da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f680abf50 0x7f4f680ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6e6da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6815c0d0 con 0x7f4f680aa8c0 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6e6da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4f680ab4b0 con 0x7f4f680abf50 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6e6da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6815bdf0 con 0x7f4f680c3650 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6cc77640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f680abf50 0x7f4f680ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6cc77640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f680abf50 0x7f4f680ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42516/0 (socket says 172.21.15.135:42516) 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.759+0000 7f4f6cc77640 1 -- 172.21.15.135:0/1636317977 learned_addr learned my addr 172.21.15.135:0/1636317977 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.760+0000 7f4f6cc77640 1 -- 172.21.15.135:0/1636317977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 msgr2=0x7f4f6806e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.760+0000 7f4f6cc77640 1 --2- 172.21.15.135:0/1636317977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6806e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:23.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.760+0000 7f4f6cc77640 1 -- 172.21.15.135:0/1636317977 >> v1:172.21.15.135:6789/0 conn(0x7f4f680aa8c0 legacy=0x7f4f680c3a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:23.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.760+0000 7f4f6cc77640 1 -- 172.21.15.135:0/1636317977 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f680ab750 con 0x7f4f680abf50 2024-04-26T22:04:23.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.761+0000 7f4f6cc77640 1 --2- 172.21.15.135:0/1636317977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680abf50 0x7f4f680ac3e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009870 tx=0x7f4f5802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fe01486e152f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.762+0000 7f4f66ffd640 1 -- 172.21.15.135:0/1636317977 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58037070 con 0x7f4f680abf50 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.762+0000 7f4f66ffd640 1 -- 172.21.15.135:0/1636317977 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5802ae10 con 0x7f4f680abf50 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f6e6da640 1 -- 172.21.15.135:0/1636317977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680abf50 msgr2=0x7f4f680ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f66ffd640 1 -- 172.21.15.135:0/1636317977 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58037070 con 0x7f4f680abf50 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f6e6da640 1 --2- 172.21.15.135:0/1636317977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680abf50 0x7f4f680ac3e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009870 tx=0x7f4f5802a100 comp rx=0 tx=0).stop 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f6e6da640 1 -- 172.21.15.135:0/1636317977 shutdown_connections 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f6e6da640 1 --2- 172.21.15.135:0/1636317977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680abf50 0x7f4f680ac3e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f6e6da640 1 --2- 172.21.15.135:0/1636317977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6806e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.763+0000 7f4f6e6da640 1 -- 172.21.15.135:0/1636317977 >> 172.21.15.135:0/1636317977 conn(0x7f4f680c82d0 msgr2=0x7f4f680c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:23.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.765+0000 7f4f6e6da640 1 -- 172.21.15.135:0/1636317977 shutdown_connections 2024-04-26T22:04:23.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.765+0000 7f4f6e6da640 1 -- 172.21.15.135:0/1636317977 wait complete. 2024-04-26T22:04:23.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.767+0000 7f4f6e6da640 1 Processor -- start 2024-04-26T22:04:23.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.767+0000 7f4f6e6da640 1 -- start start 2024-04-26T22:04:23.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f6e6da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680aa8c0 0x7f4f680c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:23.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f6e6da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 0x7f4f680c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f6e6da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f6e6da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680c42c0 con 0x7f4f680aa8c0 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f6e6da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680c4140 con 0x7f4f680abf50 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f6e6da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6815a6a0 con 0x7f4f680c3650 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 0x7f4f680c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 0x7f4f680c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:46086/0 (socket says 172.21.15.135:46086) 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.768+0000 7f4f677fe640 1 -- 172.21.15.135:0/3679497040 learned_addr learned my addr 172.21.15.135:0/3679497040 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f6cc77640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f67fff640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680aa8c0 0x7f4f680c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f677fe640 1 -- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 msgr2=0x7f4f6815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:23.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f677fe640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:23.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f677fe640 1 -- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680aa8c0 msgr2=0x7f4f680c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:23.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f677fe640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680aa8c0 0x7f4f680c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:23.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f677fe640 1 -- 172.21.15.135:0/3679497040 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f68074620 con 0x7f4f680abf50 2024-04-26T22:04:23.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.769+0000 7f4f67fff640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680aa8c0 0x7f4f680c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:23.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.770+0000 7f4f677fe640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 0x7f4f680c64e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c400 tx=0x7f4f5c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:23.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.774+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c017070 con 0x7f4f680abf50 2024-04-26T22:04:23.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.774+0000 7f4f6e6da640 1 -- 172.21.15.135:0/3679497040 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f68074900 con 0x7f4f680abf50 2024-04-26T22:04:23.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.774+0000 7f4f6e6da640 1 -- 172.21.15.135:0/3679497040 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681582d0 con 0x7f4f680abf50 2024-04-26T22:04:23.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.774+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5c0035f0 con 0x7f4f680abf50 2024-04-26T22:04:23.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.776+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0039e0 con 0x7f4f680abf50 2024-04-26T22:04:23.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.777+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f5c01a070 con 0x7f4f680abf50 2024-04-26T22:04:23.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.777+0000 7f4f6e6da640 1 -- 172.21.15.135:0/3679497040 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f48005290 con 0x7f4f680abf50 2024-04-26T22:04:23.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.779+0000 7f4f64ff9640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f5406c970 0x7f4f5406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:23.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.779+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f4f5c08b310 con 0x7f4f680abf50 2024-04-26T22:04:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.780+0000 7f4f67fff640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f5406c970 0x7f4f5406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.781+0000 7f4f67fff640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f5406c970 0x7f4f5406ee30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f6815e550 tx=0x7f4f60009570 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:23.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:23.786+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f5c058cf0 con 0x7f4f680abf50 2024-04-26T22:04:24.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.274+0000 7f4f6e6da640 1 -- 172.21.15.135:0/3679497040 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f48005580 con 0x7f4f680abf50 2024-04-26T22:04:24.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.276+0000 7f4f64ff9640 1 -- 172.21.15.135:0/3679497040 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5672 (secure 0 0 0) 0x7f4f5c05cff0 con 0x7f4f680abf50 2024-04-26T22:04:24.276 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:24.276 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:24.277 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 37 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.281+0000 7f4f4e7fc640 1 -- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f5406c970 msgr2=0x7f4f5406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.281+0000 7f4f4e7fc640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f5406c970 0x7f4f5406ee30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f6815e550 tx=0x7f4f60009570 comp rx=0 tx=0).stop 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.281+0000 7f4f4e7fc640 1 -- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 msgr2=0x7f4f680c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.281+0000 7f4f4e7fc640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 0x7f4f680c64e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c400 tx=0x7f4f5c00c8c0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.282+0000 7f4f4e7fc640 1 -- 172.21.15.135:0/3679497040 shutdown_connections 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.282+0000 7f4f4e7fc640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f5406c970 0x7f4f5406ee30 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f6815e550 tx=0x7f4f60009570 comp rx=0 tx=0).stop 2024-04-26T22:04:24.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.282+0000 7f4f4e7fc640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f680c3650 0x7f4f6815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.282+0000 7f4f4e7fc640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f680abf50 0x7f4f680c64e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.282+0000 7f4f4e7fc640 1 --2- 172.21.15.135:0/3679497040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f680aa8c0 0x7f4f680c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.282+0000 7f4f4e7fc640 1 -- 172.21.15.135:0/3679497040 >> 172.21.15.135:0/3679497040 conn(0x7f4f680c82d0 msgr2=0x7f4f680ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.284+0000 7f4f4e7fc640 1 -- 172.21.15.135:0/3679497040 shutdown_connections 2024-04-26T22:04:24.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.284+0000 7f4f4e7fc640 1 -- 172.21.15.135:0/3679497040 wait complete. 2024-04-26T22:04:24.311 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=5086 2024-04-26T22:04:24.311 INFO:tasks.mds_thrash.fs.[b]:waiting for 7 secs before reviving mds.c 2024-04-26T22:04:24.394 INFO:tasks.workunit.client.1.smithi139.stdout:Syncing()...26 sec 2024-04-26T22:04:24.395 INFO:tasks.workunit.client.1.smithi139.stdout:Starting Actual Benchmark At: Fri Apr 26 22:04:24 2024 2024-04-26T22:04:24.395 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:04:24.532 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:24.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.713+0000 7f18b9474640 1 Processor -- start 2024-04-26T22:04:24.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.716+0000 7f18b9474640 1 -- start start 2024-04-26T22:04:24.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b9474640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c35d0 0x7f18b406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b9474640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f18b40ac390 0x7f18b41674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b9474640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b40c4210 con 0x7f18b40c4630 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b9474640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f18b40c4a50 con 0x7f18b40ac390 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b9474640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18b406f830 con 0x7f18b40c35d0 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f18b40ac390 0x7f18b41674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f18b40ac390 0x7f18b41674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42540/0 (socket says 172.21.15.135:42540) 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b37fe640 1 -- 172.21.15.135:0/2046695029 learned_addr learned my addr 172.21.15.135:0/2046695029 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.717+0000 7f18b2ffd640 1 --2- 172.21.15.135:0/2046695029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c35d0 0x7f18b406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.720+0000 7f18b37fe640 1 -- 172.21.15.135:0/2046695029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c35d0 msgr2=0x7f18b406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:24.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.720+0000 7f18b37fe640 1 --2- 172.21.15.135:0/2046695029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c35d0 0x7f18b406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.720+0000 7f18b37fe640 1 -- 172.21.15.135:0/2046695029 >> v1:172.21.15.135:6789/0 conn(0x7f18b40c4630 legacy=0x7f18b40718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.720+0000 7f18b37fe640 1 -- 172.21.15.135:0/2046695029 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b406ef00 con 0x7f18b40ac390 2024-04-26T22:04:24.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.722+0000 7f18b37fe640 1 --2- 172.21.15.135:0/2046695029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 0x7f18b41674d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f18a80083f0 tx=0x7f18a800abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6489ca3b1252556d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.722+0000 7f18b1ffb640 1 -- 172.21.15.135:0/2046695029 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8008df0 con 0x7f18b40ac390 2024-04-26T22:04:24.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.722+0000 7f18b1ffb640 1 -- 172.21.15.135:0/2046695029 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a802a8f0 con 0x7f18b40ac390 2024-04-26T22:04:24.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.723+0000 7f18b1ffb640 1 -- 172.21.15.135:0/2046695029 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a802acc0 con 0x7f18b40ac390 2024-04-26T22:04:24.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.724+0000 7f18b9474640 1 -- 172.21.15.135:0/2046695029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 msgr2=0x7f18b41674d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:24.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.724+0000 7f18b9474640 1 --2- 172.21.15.135:0/2046695029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 0x7f18b41674d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f18a80083f0 tx=0x7f18a800abe0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.726+0000 7f18b9474640 1 -- 172.21.15.135:0/2046695029 shutdown_connections 2024-04-26T22:04:24.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.726+0000 7f18b9474640 1 --2- 172.21.15.135:0/2046695029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 0x7f18b41674d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.726+0000 7f18b9474640 1 --2- 172.21.15.135:0/2046695029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c35d0 0x7f18b406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.726+0000 7f18b9474640 1 -- 172.21.15.135:0/2046695029 >> 172.21.15.135:0/2046695029 conn(0x7f18b40c82d0 msgr2=0x7f18b40c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:24.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.727+0000 7f18b9474640 1 -- 172.21.15.135:0/2046695029 shutdown_connections 2024-04-26T22:04:24.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.727+0000 7f18b9474640 1 -- 172.21.15.135:0/2046695029 wait complete. 2024-04-26T22:04:24.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.729+0000 7f18b9474640 1 Processor -- start 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.730+0000 7f18b9474640 1 -- start start 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b9474640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 0x7f18b40ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b9474640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 0x7f18b40bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b9474640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c4630 0x7f18b40c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b9474640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4071ff0 con 0x7f18b40ac390 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b9474640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18b406f4c0 con 0x7f18b40c35d0 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b9474640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18b40c39b0 con 0x7f18b40c4630 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 0x7f18b40bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 0x7f18b40bf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:46094/0 (socket says 172.21.15.135:46094) 2024-04-26T22:04:24.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.731+0000 7f18b27fc640 1 -- 172.21.15.135:0/491658108 learned_addr learned my addr 172.21.15.135:0/491658108 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:24.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.732+0000 7f18b27fc640 1 -- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c4630 msgr2=0x7f18b40c0ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:24.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.732+0000 7f18b27fc640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c4630 0x7f18b40c0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.732+0000 7f18b27fc640 1 -- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 msgr2=0x7f18b40ab560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:24.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.732+0000 7f18b27fc640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 0x7f18b40ab560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:24.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.732+0000 7f18b27fc640 1 -- 172.21.15.135:0/491658108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b40be340 con 0x7f18b40c35d0 2024-04-26T22:04:24.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.732+0000 7f18b27fc640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 0x7f18b40bf5e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c780 tx=0x7f18a400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.735+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4017070 con 0x7f18b40c35d0 2024-04-26T22:04:24.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.735+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a4005270 con 0x7f18b40c35d0 2024-04-26T22:04:24.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.735+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400a840 con 0x7f18b40c35d0 2024-04-26T22:04:24.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.735+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b40be620 con 0x7f18b40c35d0 2024-04-26T22:04:24.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.735+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b40bebe0 con 0x7f18b40c35d0 2024-04-26T22:04:24.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.736+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1894005290 con 0x7f18b40c35d0 2024-04-26T22:04:24.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.739+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18a401a070 con 0x7f18b40c35d0 2024-04-26T22:04:24.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.740+0000 7f189b7fe640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f18a006c9c0 0x7f18a006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:24.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.740+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f18a408b3b0 con 0x7f18b40c35d0 2024-04-26T22:04:24.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.743+0000 7f18b2ffd640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f18a006c9c0 0x7f18a006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:24.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.744+0000 7f18b2ffd640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f18a006c9c0 0x7f18a006ee80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f18b41f7820 tx=0x7f18ac00e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:24.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:24.747+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f18a4058c90 con 0x7f18b40c35d0 2024-04-26T22:04:25.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.230+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1894005580 con 0x7f18b40c35d0 2024-04-26T22:04:25.232 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:25.232 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:25.232 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 37 2024-04-26T22:04:25.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.232+0000 7f189b7fe640 1 -- 172.21.15.135:0/491658108 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5672 (secure 0 0 0) 0x7f18a405cf90 con 0x7f18b40c35d0 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f18a006c9c0 msgr2=0x7f18a006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f18a006c9c0 0x7f18a006ee80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f18b41f7820 tx=0x7f18ac00e040 comp rx=0 tx=0).stop 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 msgr2=0x7f18b40bf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 0x7f18b40bf5e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c780 tx=0x7f18a400cc40 comp rx=0 tx=0).stop 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 shutdown_connections 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f18a006c9c0 0x7f18a006ee80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:25.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b40c4630 0x7f18b40c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:25.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b40c35d0 0x7f18b40bf5e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:25.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 --2- 172.21.15.135:0/491658108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18b40ac390 0x7f18b40ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:25.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.237+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 >> 172.21.15.135:0/491658108 conn(0x7f18b40c82d0 msgr2=0x7f18b40c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:25.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.240+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 shutdown_connections 2024-04-26T22:04:25.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:25.240+0000 7f18b9474640 1 -- 172.21.15.135:0/491658108 wait complete. 2024-04-26T22:04:25.260 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:04:26.470 INFO:tasks.ceph.mds.b.smithi139.stderr:2024-04-26T22:04:26.470+0000 7f10c36ba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:04:27.261 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:27.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.450+0000 7f4e5f412640 1 Processor -- start 2024-04-26T22:04:27.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.450+0000 7f4e5f412640 1 -- start start 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.451+0000 7f4e5f412640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e580cc5f0 0x7f4e5815f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.451+0000 7f4e5f412640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4e580cb830 0x7f4e58158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5f412640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5f412640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4e580c8c40 con 0x7f4e580cc5f0 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5f412640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4e580cbd10 con 0x7f4e580cb830 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5f412640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4e580cac10 con 0x7f4e581587b0 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5e410640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e580cc5f0 0x7f4e5815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5e410640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e580cc5f0 0x7f4e5815f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42550/0 (socket says 172.21.15.135:42550) 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5e410640 1 -- 172.21.15.135:0/1849264880 learned_addr learned my addr 172.21.15.135:0/1849264880 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:27.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5ec11640 1 --2- 172.21.15.135:0/1849264880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5e410640 1 -- 172.21.15.135:0/1849264880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 msgr2=0x7f4e580c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5e410640 1 --2- 172.21.15.135:0/1849264880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.452+0000 7f4e5e410640 1 -- 172.21.15.135:0/1849264880 >> v2:172.21.15.139:3300/0 conn(0x7f4e580cb830 msgr2=0x7f4e58158270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.453+0000 7f4e5e410640 1 --2- 172.21.15.135:0/1849264880 >> v2:172.21.15.139:3300/0 conn(0x7f4e580cb830 0x7f4e58158270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.453+0000 7f4e5e410640 1 -- 172.21.15.135:0/1849264880 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e580caef0 con 0x7f4e580cc5f0 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.454+0000 7f4e5e410640 1 --2- 172.21.15.135:0/1849264880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 0x7f4e5815f010 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4e580cc280 tx=0x7f4e5002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69a9ec967f5cb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.454+0000 7f4e5d40e640 1 -- 172.21.15.135:0/1849264880 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50036070 con 0x7f4e580cc5f0 2024-04-26T22:04:27.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.454+0000 7f4e5d40e640 1 -- 172.21.15.135:0/1849264880 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5002ae50 con 0x7f4e580cc5f0 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.455+0000 7f4e5f412640 1 -- 172.21.15.135:0/1849264880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 msgr2=0x7f4e5815f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.455+0000 7f4e5f412640 1 --2- 172.21.15.135:0/1849264880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 0x7f4e5815f010 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4e580cc280 tx=0x7f4e5002a630 comp rx=0 tx=0).stop 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.456+0000 7f4e5f412640 1 -- 172.21.15.135:0/1849264880 shutdown_connections 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.456+0000 7f4e5f412640 1 --2- 172.21.15.135:0/1849264880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.456+0000 7f4e5f412640 1 --2- 172.21.15.135:0/1849264880 >> v2:172.21.15.139:3300/0 conn(0x7f4e580cb830 0x7f4e58158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.456+0000 7f4e5f412640 1 --2- 172.21.15.135:0/1849264880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 0x7f4e5815f010 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.456+0000 7f4e5f412640 1 -- 172.21.15.135:0/1849264880 >> 172.21.15.135:0/1849264880 conn(0x7f4e580ae180 msgr2=0x7f4e580ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:27.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.458+0000 7f4e5f412640 1 -- 172.21.15.135:0/1849264880 shutdown_connections 2024-04-26T22:04:27.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.458+0000 7f4e5f412640 1 -- 172.21.15.135:0/1849264880 wait complete. 2024-04-26T22:04:27.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.460+0000 7f4e5f412640 1 Processor -- start 2024-04-26T22:04:27.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.460+0000 7f4e5f412640 1 -- start start 2024-04-26T22:04:27.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.461+0000 7f4e5f412640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e580cb830 0x7f4e580bafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.461+0000 7f4e5f412640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 0x7f4e580c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.461+0000 7f4e5f412640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.461+0000 7f4e5f412640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e580bfc50 con 0x7f4e580cc5f0 2024-04-26T22:04:27.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.461+0000 7f4e5f412640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e580bfad0 con 0x7f4e580cb830 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.461+0000 7f4e5f412640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4e580cc9d0 con 0x7f4e581587b0 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5e410640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e580cb830 0x7f4e580bafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5e410640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e580cb830 0x7f4e580bafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:46122/0 (socket says 172.21.15.135:46122) 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5e410640 1 -- 172.21.15.135:0/3222314287 learned_addr learned my addr 172.21.15.135:0/3222314287 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5ec11640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5ec11640 1 -- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e580cb830 msgr2=0x7f4e580bafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5ec11640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e580cb830 0x7f4e580bafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5ec11640 1 -- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 msgr2=0x7f4e580c5430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5ec11640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 0x7f4e580c5430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.462+0000 7f4e5ec11640 1 -- 172.21.15.135:0/3222314287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e580bb4f0 con 0x7f4e581587b0 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.463+0000 7f4e5ec11640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c1af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400e5c0 tx=0x7f4e5400ea80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:27.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.463+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54017070 con 0x7f4e581587b0 2024-04-26T22:04:27.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.463+0000 7f4e5f412640 1 -- 172.21.15.135:0/3222314287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e580bfe30 con 0x7f4e581587b0 2024-04-26T22:04:27.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.463+0000 7f4e5f412640 1 -- 172.21.15.135:0/3222314287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e580c03f0 con 0x7f4e581587b0 2024-04-26T22:04:27.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.465+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e54013680 con 0x7f4e581587b0 2024-04-26T22:04:27.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.465+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54013a50 con 0x7f4e581587b0 2024-04-26T22:04:27.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.466+0000 7f4e5f412640 1 -- 172.21.15.135:0/3222314287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e40005290 con 0x7f4e581587b0 2024-04-26T22:04:27.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.468+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e5401a070 con 0x7f4e581587b0 2024-04-26T22:04:27.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.469+0000 7f4e4effd640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4e3c06c940 0x7f4e3c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:27.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.469+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f4e5408bfe0 con 0x7f4e581587b0 2024-04-26T22:04:27.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.470+0000 7f4e5e410640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4e3c06c940 0x7f4e3c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:27.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.471+0000 7f4e5e410640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4e3c06c940 0x7f4e3c06ee00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a670 tx=0x7f4e50009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:27.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.477+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e540599c0 con 0x7f4e581587b0 2024-04-26T22:04:27.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.973+0000 7f4e5f412640 1 -- 172.21.15.135:0/3222314287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e40005580 con 0x7f4e581587b0 2024-04-26T22:04:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.975+0000 7f4e4effd640 1 -- 172.21.15.135:0/3222314287 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5672 (secure 0 0 0) 0x7f4e5405dcc0 con 0x7f4e581587b0 2024-04-26T22:04:27.975 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:27.975 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 37 2024-04-26T22:04:27.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 -- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4e3c06c940 msgr2=0x7f4e3c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:27.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4e3c06c940 0x7f4e3c06ee00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a670 tx=0x7f4e50009000 comp rx=0 tx=0).stop 2024-04-26T22:04:27.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 -- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 msgr2=0x7f4e580c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c1af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400e5c0 tx=0x7f4e5400ea80 comp rx=0 tx=0).stop 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 -- 172.21.15.135:0/3222314287 shutdown_connections 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4e3c06c940 0x7f4e3c06ee00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e581587b0 0x7f4e580c1af0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e580cc5f0 0x7f4e580c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 --2- 172.21.15.135:0/3222314287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e580cb830 0x7f4e580bafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.979+0000 7f4e4cff9640 1 -- 172.21.15.135:0/3222314287 >> 172.21.15.135:0/3222314287 conn(0x7f4e580ae180 msgr2=0x7f4e580c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.980+0000 7f4e4cff9640 1 -- 172.21.15.135:0/3222314287 shutdown_connections 2024-04-26T22:04:27.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:27.980+0000 7f4e4cff9640 1 -- 172.21.15.135:0/3222314287 wait complete. 2024-04-26T22:04:28.002 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:04:29.416 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:04:29.418 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:04:30.002 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:30.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.192+0000 7f8c87d5f640 1 Processor -- start 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.192+0000 7f8c87d5f640 1 -- start start 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c87d5f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c80150200 0x7f8c800ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c87d5f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8006ff20 con 0x7f8c800ac680 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c87d5f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8006f4e0 con 0x7f8c800c2cd0 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c87d5f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8c800c4bd0 con 0x7f8c80150200 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c85afb640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c80150200 0x7f8c800ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c852fa640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8c800c2cd0 0x7f8c801533b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:47166/0 (socket says 172.21.15.135:47166) 2024-04-26T22:04:30.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.194+0000 7f8c852fa640 1 -- 172.21.15.135:0/525175024 learned_addr learned my addr 172.21.15.135:0/525175024 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:30.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.195+0000 7f8c85afb640 1 -- 172.21.15.135:0/525175024 >> v1:172.21.15.139:6789/0 conn(0x7f8c800c2cd0 legacy=0x7f8c801533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:30.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.195+0000 7f8c85afb640 1 -- 172.21.15.135:0/525175024 >> v1:172.21.15.135:6789/0 conn(0x7f8c800ac680 legacy=0x7f8c8015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:30.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.195+0000 7f8c85afb640 1 -- 172.21.15.135:0/525175024 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c800c48f0 con 0x7f8c80150200 2024-04-26T22:04:30.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.196+0000 7f8c85afb640 1 --2- 172.21.15.135:0/525175024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c80150200 0x7f8c800ac140 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00ba40 tx=0x7f8c7c02c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497d8247f7f3d2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:30.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.197+0000 7f8c84af9640 1 -- 172.21.15.135:0/525175024 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c038070 con 0x7f8c80150200 2024-04-26T22:04:30.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.197+0000 7f8c84af9640 1 -- 172.21.15.135:0/525175024 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c7c031710 con 0x7f8c80150200 2024-04-26T22:04:30.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.197+0000 7f8c84af9640 1 -- 172.21.15.135:0/525175024 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031ae0 con 0x7f8c80150200 2024-04-26T22:04:30.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.198+0000 7f8c87d5f640 1 -- 172.21.15.135:0/525175024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c80150200 msgr2=0x7f8c800ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:30.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.198+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/525175024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c80150200 0x7f8c800ac140 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00ba40 tx=0x7f8c7c02c7b0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.202+0000 7f8c87d5f640 1 -- 172.21.15.135:0/525175024 shutdown_connections 2024-04-26T22:04:30.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.202+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/525175024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c80150200 0x7f8c800ac140 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.202+0000 7f8c87d5f640 1 -- 172.21.15.135:0/525175024 >> 172.21.15.135:0/525175024 conn(0x7f8c800c82d0 msgr2=0x7f8c800c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:30.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.208+0000 7f8c87d5f640 1 -- 172.21.15.135:0/525175024 shutdown_connections 2024-04-26T22:04:30.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.209+0000 7f8c87d5f640 1 -- 172.21.15.135:0/525175024 wait complete. 2024-04-26T22:04:30.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.211+0000 7f8c87d5f640 1 Processor -- start 2024-04-26T22:04:30.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.211+0000 7f8c87d5f640 1 -- start start 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.212+0000 7f8c87d5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c800ac680 0x7f8c800721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.212+0000 7f8c87d5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c800c2cd0 0x7f8c801f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.212+0000 7f8c87d5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 0x7f8c800c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.212+0000 7f8c87d5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80150760 con 0x7f8c800c2cd0 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.212+0000 7f8c87d5f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801505e0 con 0x7f8c80150200 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.212+0000 7f8c87d5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8c80153ac0 con 0x7f8c800ac680 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.213+0000 7f8c862fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 0x7f8c800c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.213+0000 7f8c862fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 0x7f8c800c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:42908/0 (socket says 172.21.15.135:42908) 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.213+0000 7f8c862fc640 1 -- 172.21.15.135:0/3665955733 learned_addr learned my addr 172.21.15.135:0/3665955733 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:30.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.213+0000 7f8c85afb640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c800ac680 0x7f8c800721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:30.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.214+0000 7f8c862fc640 1 -- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c800ac680 msgr2=0x7f8c800721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:30.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.214+0000 7f8c862fc640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c800ac680 0x7f8c800721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.214+0000 7f8c862fc640 1 -- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c800c2cd0 msgr2=0x7f8c801f6a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:30.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.214+0000 7f8c862fc640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c800c2cd0 0x7f8c801f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.214+0000 7f8c862fc640 1 -- 172.21.15.135:0/3665955733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8014f710 con 0x7f8c80150200 2024-04-26T22:04:30.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.214+0000 7f8c862fc640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 0x7f8c800c5ec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c390 tx=0x7f8c7000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:30.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.215+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70017070 con 0x7f8c80150200 2024-04-26T22:04:30.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.215+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c80150a60 con 0x7f8c80150200 2024-04-26T22:04:30.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.215+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801fa1a0 con 0x7f8c80150200 2024-04-26T22:04:30.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.216+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c700035f0 con 0x7f8c80150200 2024-04-26T22:04:30.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.216+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700039e0 con 0x7f8c80150200 2024-04-26T22:04:30.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.218+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c68005290 con 0x7f8c80150200 2024-04-26T22:04:30.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.220+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c7001a070 con 0x7f8c80150200 2024-04-26T22:04:30.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.220+0000 7f8c767fc640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8c6406c920 0x7f8c6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:30.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.220+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f8c7008b430 con 0x7f8c80150200 2024-04-26T22:04:30.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.222+0000 7f8c85afb640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8c6406c920 0x7f8c6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:30.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.223+0000 7f8c85afb640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8c6406c920 0x7f8c6406ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c035000 tx=0x7f8c7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:30.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.227+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c8014f710 con 0x7f8c80150200 2024-04-26T22:04:30.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.670+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c68005580 con 0x7f8c80150200 2024-04-26T22:04:30.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.673+0000 7f8c767fc640 1 -- 172.21.15.135:0/3665955733 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5672 (secure 0 0 0) 0x7f8c70058c80 con 0x7f8c80150200 2024-04-26T22:04:30.678 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:30.678 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:30.679 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 37 2024-04-26T22:04:30.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.677+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8c6406c920 msgr2=0x7f8c6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:30.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.677+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8c6406c920 0x7f8c6406ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c035000 tx=0x7f8c7c008040 comp rx=0 tx=0).stop 2024-04-26T22:04:30.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.677+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 msgr2=0x7f8c800c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.677+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 0x7f8c800c5ec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c390 tx=0x7f8c7000c850 comp rx=0 tx=0).stop 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 shutdown_connections 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8c6406c920 0x7f8c6406ede0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c80150200 0x7f8c800c5ec0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c800c2cd0 0x7f8c801f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 --2- 172.21.15.135:0/3665955733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c800ac680 0x7f8c800721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 >> 172.21.15.135:0/3665955733 conn(0x7f8c800c82d0 msgr2=0x7f8c801533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 shutdown_connections 2024-04-26T22:04:30.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:30.678+0000 7f8c87d5f640 1 -- 172.21.15.135:0/3665955733 wait complete. 2024-04-26T22:04:30.697 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:04:31.312 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-04-26T22:04:31.312 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:04:31.312 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:04:31.317 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:04:31.317 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:31.455 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-26T22:04:31.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.498+0000 7f393e15f640 1 Processor -- start 2024-04-26T22:04:31.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.498+0000 7f393e15f640 1 -- start start 2024-04-26T22:04:31.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.499+0000 7f393e15f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 0x7f3930047570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.499+0000 7f393e15f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3930047ab0 0x7f3930047f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.499+0000 7f393e15f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393000bb00 con 0x7f39300dbff0 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.499+0000 7f393e15f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3930006f90 con 0x7f3930047ab0 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.499+0000 7f393e15f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3930007530 con 0x7f3930004b00 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.500+0000 7f393d15d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f39300dbff0 0x7f393003ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:47182/0 (socket says 172.21.15.135:47182) 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.500+0000 7f393d15d640 1 -- 172.21.15.135:0/4287833473 learned_addr learned my addr 172.21.15.135:0/4287833473 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.500+0000 7f393c95c640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 0x7f3930047570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203774796 0 0) 0x7f393000bb00 con 0x7f39300dbff0 2024-04-26T22:04:31.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3924003520 con 0x7f39300dbff0 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f393d95e640 1 --2- 172.21.15.135:0/4287833473 >> v2:172.21.15.135:3300/0 conn(0x7f3930047ab0 0x7f3930047f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3906695268 0 0) 0x7f3924003520 con 0x7f39300dbff0 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 msgr2=0x7f3930047570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 0x7f3930047570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 >> v2:172.21.15.135:3300/0 conn(0x7f3930047ab0 msgr2=0x7f3930047f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 --2- 172.21.15.135:0/4287833473 >> v2:172.21.15.135:3300/0 conn(0x7f3930047ab0 0x7f3930047f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.501+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393000bde0 con 0x7f39300dbff0 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f3938050be0 con 0x7f39300dbff0 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 >> v1:172.21.15.139:6789/0 conn(0x7f39300dbff0 legacy=0x7f393003ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:31.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240039b0 0x7f3924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3924004350 0x7f3924004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 0x7f3924004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3930007530 con 0x7f39240039b0 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f392400b9c0 con 0x7f3924004350 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f392400bb40 con 0x7f3930004b00 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d15d640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240039b0 0x7f3924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d95e640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 0x7f3924004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393c95c640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3924004350 0x7f3924004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d15d640 1 -- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 msgr2=0x7f3924004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d15d640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3930004b00 0x7f3924004fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d15d640 1 -- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3924004350 msgr2=0x7f3924004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d15d640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3924004350 0x7f3924004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393d15d640 1 -- 172.21.15.135:0/4287833473 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3924003520 con 0x7f39240039b0 2024-04-26T22:04:31.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.502+0000 7f393c95c640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3924004350 0x7f3924004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:31.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.503+0000 7f393d15d640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240039b0 0x7f3924003dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3938051fc0 tx=0x7f3924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3060dfd39003dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:31.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.504+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380b4070 con 0x7f39240039b0 2024-04-26T22:04:31.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.504+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39380af640 con 0x7f39240039b0 2024-04-26T22:04:31.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.504+0000 7f392ffff640 1 -- 172.21.15.135:0/4287833473 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938050f10 con 0x7f39240039b0 2024-04-26T22:04:31.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.505+0000 7f393e15f640 1 -- 172.21.15.135:0/4287833473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240039b0 msgr2=0x7f3924003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:31.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.505+0000 7f393e15f640 1 --2- 172.21.15.135:0/4287833473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240039b0 0x7f3924003dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3938051fc0 tx=0x7f3924005830 comp rx=0 tx=0).stop 2024-04-26T22:04:31.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.505+0000 7f393d95e640 1 -- 172.21.15.135:0/4287833473 reap_dead start 2024-04-26T22:04:31.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.505+0000 7f393e15f640 1 -- 172.21.15.135:0/4287833473 shutdown_connections 2024-04-26T22:04:31.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.505+0000 7f393e15f640 1 -- 172.21.15.135:0/4287833473 >> 172.21.15.135:0/4287833473 conn(0x7f39300062e0 msgr2=0x7f39300066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:31.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.507+0000 7f393e15f640 1 -- 172.21.15.135:0/4287833473 shutdown_connections 2024-04-26T22:04:31.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.507+0000 7f393e15f640 1 -- 172.21.15.135:0/4287833473 wait complete. 2024-04-26T22:04:31.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.509+0000 7f393e15f640 1 Processor -- start 2024-04-26T22:04:31.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.509+0000 7f393e15f640 1 -- start start 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.510+0000 7f393e15f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 0x7f393019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.510+0000 7f393e15f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3930031580 0x7f39300d5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.510+0000 7f393e15f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39300d5db0 0x7f39301a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.510+0000 7f393e15f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f393003f480 con 0x7f3930031580 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.510+0000 7f393e15f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f393003f300 con 0x7f39300d5db0 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.510+0000 7f393e15f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f393000b740 con 0x7f393019c020 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d15d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 0x7f393019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d15d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 0x7f393019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:49712/0 (socket says 172.21.15.135:49712) 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d15d640 1 -- 172.21.15.135:0/3903157707 learned_addr learned my addr 172.21.15.135:0/3903157707 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d95e640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39300d5db0 0x7f39301a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d15d640 1 -- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39300d5db0 msgr2=0x7f39301a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d15d640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39300d5db0 0x7f39301a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393d15d640 1 -- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3930031580 msgr2=0x7f39300d5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.511+0000 7f393c95c640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3930031580 0x7f39300d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.512+0000 7f393d15d640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3930031580 0x7f39300d5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.512+0000 7f393d15d640 1 -- 172.21.15.135:0/3903157707 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393002e8c0 con 0x7f393019c020 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.512+0000 7f393c95c640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3930031580 0x7f39300d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.512+0000 7f393d15d640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 0x7f393019c400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3938075230 tx=0x7f39380a9070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.513+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393808c070 con 0x7f393019c020 2024-04-26T22:04:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.513+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39301a2160 con 0x7f393019c020 2024-04-26T22:04:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.514+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f393019f770 con 0x7f393019c020 2024-04-26T22:04:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.514+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393806e800 con 0x7f393019c020 2024-04-26T22:04:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.514+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938050f10 con 0x7f393019c020 2024-04-26T22:04:31.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.517+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3918005290 con 0x7f393019c020 2024-04-26T22:04:31.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.520+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39380b7060 con 0x7f393019c020 2024-04-26T22:04:31.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.520+0000 7f392dffb640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f391c06c9f0 0x7f391c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:31.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.520+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f39380b5070 con 0x7f393019c020 2024-04-26T22:04:31.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.521+0000 7f393c95c640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f391c06c9f0 0x7f391c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:31.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.522+0000 7f393c95c640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f391c06c9f0 0x7f391c06eeb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f393002ca60 tx=0x7f3928008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:31.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.528+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39380e5820 con 0x7f393019c020 2024-04-26T22:04:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:31.998+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3918005580 con 0x7f393019c020 2024-04-26T22:04:32.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.000+0000 7f392dffb640 1 -- 172.21.15.135:0/3903157707 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6500 (secure 0 0 0) 0x7f39380e9b20 con 0x7f393019c020 2024-04-26T22:04:32.001 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:32.001 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:32.001 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 38 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.005+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f391c06c9f0 msgr2=0x7f391c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.005+0000 7f393e15f640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f391c06c9f0 0x7f391c06eeb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f393002ca60 tx=0x7f3928008040 comp rx=0 tx=0).stop 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.005+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 msgr2=0x7f393019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.005+0000 7f393e15f640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 0x7f393019c400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3938075230 tx=0x7f39380a9070 comp rx=0 tx=0).stop 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 shutdown_connections 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f391c06c9f0 0x7f391c06eeb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39300d5db0 0x7f39301a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3930031580 0x7f39300d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 --2- 172.21.15.135:0/3903157707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393019c020 0x7f393019c400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 >> 172.21.15.135:0/3903157707 conn(0x7f39300062e0 msgr2=0x7f39300d9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:32.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 shutdown_connections 2024-04-26T22:04:32.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.006+0000 7f393e15f640 1 -- 172.21.15.135:0/3903157707 wait complete. 2024-04-26T22:04:32.029 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-04-26T22:04:32.029 INFO:tasks.mds_thrash.fs.[b]:waiting for 62 secs before thrashing 2024-04-26T22:04:32.698 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:32.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.905+0000 7f9bfdebd640 1 Processor -- start 2024-04-26T22:04:32.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.908+0000 7f9bfdebd640 1 -- start start 2024-04-26T22:04:32.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.909+0000 7f9bfdebd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf004a9b0 0x7f9bf0006ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:32.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.909+0000 7f9bfdebd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9bf00dbed0 0x7f9bf00d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:32.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.909+0000 7f9bfdebd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf00072f0 con 0x7f9bf00d55b0 2024-04-26T22:04:32.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.909+0000 7f9bfdebd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf0030b00 con 0x7f9bf00dbed0 2024-04-26T22:04:32.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.909+0000 7f9bfdebd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf0030de0 con 0x7f9bf004a9b0 2024-04-26T22:04:32.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf6ffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9bf00d55b0 0x7f9bf00db7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:55678/0 (socket says 172.21.15.135:55678) 2024-04-26T22:04:32.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf6ffd640 1 -- 172.21.15.135:0/3699329087 learned_addr learned my addr 172.21.15.135:0/3699329087 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:32.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf7fff640 1 --2- 172.21.15.135:0/3699329087 >> v2:172.21.15.135:3300/0 conn(0x7f9bf00dbed0 0x7f9bf00d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:32.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf77fe640 1 --2- 172.21.15.135:0/3699329087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf004a9b0 0x7f9bf0006ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:32.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3699329087 >> v2:172.21.15.135:3300/0 conn(0x7f9bf00dbed0 msgr2=0x7f9bf00d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:32.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf77fe640 1 --2- 172.21.15.135:0/3699329087 >> v2:172.21.15.135:3300/0 conn(0x7f9bf00dbed0 0x7f9bf00d59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3699329087 >> v1:172.21.15.135:6789/0 conn(0x7f9bf00d55b0 legacy=0x7f9bf00db7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:32.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3699329087 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf00310c0 con 0x7f9bf004a9b0 2024-04-26T22:04:32.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.912+0000 7f9bf7fff640 1 --2- 172.21.15.135:0/3699329087 >> v2:172.21.15.135:3300/0 conn(0x7f9bf00dbed0 0x7f9bf00d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:32.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.910+0000 7f9bf67fc640 1 -- 172.21.15.135:0/3699329087 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787668787 0 0) 0x7f9bf00072f0 con 0x7f9bf00d55b0 2024-04-26T22:04:32.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.917+0000 7f9bf77fe640 1 --2- 172.21.15.135:0/3699329087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf004a9b0 0x7f9bf0006ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07a0aded6d9b19d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:32.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.918+0000 7f9bf67fc640 1 -- 172.21.15.135:0/3699329087 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4037070 con 0x7f9bf004a9b0 2024-04-26T22:04:32.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.918+0000 7f9bf67fc640 1 -- 172.21.15.135:0/3699329087 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4032440 con 0x7f9bf004a9b0 2024-04-26T22:04:32.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.919+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3699329087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf004a9b0 msgr2=0x7f9bf0006ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:32.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.919+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3699329087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf004a9b0 0x7f9bf0006ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be402a210 comp rx=0 tx=0).stop 2024-04-26T22:04:32.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.921+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3699329087 shutdown_connections 2024-04-26T22:04:32.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.921+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3699329087 >> v2:172.21.15.135:3300/0 conn(0x7f9bf00dbed0 0x7f9bf00d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.921+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3699329087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf004a9b0 0x7f9bf0006ba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.921+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3699329087 >> 172.21.15.135:0/3699329087 conn(0x7f9bf0005ec0 msgr2=0x7f9bf00062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:32.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.924+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3699329087 shutdown_connections 2024-04-26T22:04:32.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.925+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3699329087 wait complete. 2024-04-26T22:04:32.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.927+0000 7f9bfdebd640 1 Processor -- start 2024-04-26T22:04:32.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.928+0000 7f9bfdebd640 1 -- start start 2024-04-26T22:04:32.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.929+0000 7f9bfdebd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 0x7f9bf00417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:32.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.929+0000 7f9bfdebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf00dbed0 0x7f9bf002c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:32.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.929+0000 7f9bfdebd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf002ca40 0x7f9bf0044300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:32.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.929+0000 7f9bfdebd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00315e0 con 0x7f9bf00d55b0 2024-04-26T22:04:32.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.929+0000 7f9bfdebd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf002e0b0 con 0x7f9bf00dbed0 2024-04-26T22:04:32.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.929+0000 7f9bfdebd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf0043420 con 0x7f9bf002ca40 2024-04-26T22:04:32.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.931+0000 7f9bf77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 0x7f9bf00417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:32.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 0x7f9bf00417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44788/0 (socket says 172.21.15.135:44788) 2024-04-26T22:04:32.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3004020067 learned_addr learned my addr 172.21.15.135:0/3004020067 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:32.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3004020067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf002ca40 msgr2=0x7f9bf0044300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:32.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf002ca40 0x7f9bf0044300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3004020067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf00dbed0 msgr2=0x7f9bf002c500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:32.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf00dbed0 0x7f9bf002c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:32.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.932+0000 7f9bf77fe640 1 -- 172.21.15.135:0/3004020067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf0043700 con 0x7f9bf00d55b0 2024-04-26T22:04:32.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.933+0000 7f9bf77fe640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 0x7f9bf00417a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:32.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.933+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4040030 con 0x7f9bf00d55b0 2024-04-26T22:04:32.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.933+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4038040 con 0x7f9bf00d55b0 2024-04-26T22:04:32.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.933+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4037070 con 0x7f9bf00d55b0 2024-04-26T22:04:32.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.934+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf00439e0 con 0x7f9bf00d55b0 2024-04-26T22:04:32.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.934+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf0043fa0 con 0x7f9bf00d55b0 2024-04-26T22:04:32.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.936+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf0006ef0 con 0x7f9bf00d55b0 2024-04-26T22:04:32.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.939+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9be4043070 con 0x7f9bf00d55b0 2024-04-26T22:04:32.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.940+0000 7f9be3fff640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9bdc06c9c0 0x7f9bdc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:32.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.940+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f9be40ac470 con 0x7f9bf00d55b0 2024-04-26T22:04:32.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.941+0000 7f9bf6ffd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9bdc06c9c0 0x7f9bdc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:32.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.941+0000 7f9bf6ffd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9bdc06c9c0 0x7f9bdc06ee80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00d5230 tx=0x7f9bf806bd40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:32.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:32.948+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9be407a3a0 con 0x7f9bf00d55b0 2024-04-26T22:04:33.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.421+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bf0041ce0 con 0x7f9bf00d55b0 2024-04-26T22:04:33.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.422+0000 7f9be3fff640 1 -- 172.21.15.135:0/3004020067 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6500 (secure 0 0 0) 0x7f9be4079ca0 con 0x7f9bf00d55b0 2024-04-26T22:04:33.422 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:33.423 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:33.423 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 38 2024-04-26T22:04:33.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.426+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9bdc06c9c0 msgr2=0x7f9bdc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:33.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.426+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9bdc06c9c0 0x7f9bdc06ee80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00d5230 tx=0x7f9bf806bd40 comp rx=0 tx=0).stop 2024-04-26T22:04:33.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.426+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 msgr2=0x7f9bf00417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:33.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.426+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 0x7f9bf00417a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be40046c0 comp rx=0 tx=0).stop 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 shutdown_connections 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9bdc06c9c0 0x7f9bdc06ee80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf002ca40 0x7f9bf0044300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf00dbed0 0x7f9bf002c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 --2- 172.21.15.135:0/3004020067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf00d55b0 0x7f9bf00417a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 >> 172.21.15.135:0/3004020067 conn(0x7f9bf0005ec0 msgr2=0x7f9bf00d3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 shutdown_connections 2024-04-26T22:04:33.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:33.427+0000 7f9bfdebd640 1 -- 172.21.15.135:0/3004020067 wait complete. 2024-04-26T22:04:33.448 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:04:35.449 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:35.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.632+0000 7f04d320b640 1 Processor -- start 2024-04-26T22:04:35.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.635+0000 7f04d320b640 1 -- start start 2024-04-26T22:04:35.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d320b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f04c40dae00 0x7f04c400ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:35.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d320b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c40076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:35.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d320b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c400f130 con 0x7f04c40352f0 2024-04-26T22:04:35.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d320b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f04c400e760 con 0x7f04c40dae00 2024-04-26T22:04:35.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d320b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f04c404a920 con 0x7f04c4007210 2024-04-26T22:04:35.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d2a0a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c40076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:35.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d2a0a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c40076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:49728/0 (socket says 172.21.15.135:49728) 2024-04-26T22:04:35.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d2a0a640 1 -- 172.21.15.135:0/971723046 learned_addr learned my addr 172.21.15.135:0/971723046 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:35.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.637+0000 7f04d1a08640 1 --2- 172.21.15.135:0/971723046 >> v2:172.21.15.139:3300/0 conn(0x7f04c40dae00 0x7f04c400ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:35.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.638+0000 7f04d2a0a640 1 -- 172.21.15.135:0/971723046 >> v2:172.21.15.139:3300/0 conn(0x7f04c40dae00 msgr2=0x7f04c400ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:35.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.638+0000 7f04d2a0a640 1 --2- 172.21.15.135:0/971723046 >> v2:172.21.15.139:3300/0 conn(0x7f04c40dae00 0x7f04c400ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:35.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.638+0000 7f04d2a0a640 1 -- 172.21.15.135:0/971723046 >> v1:172.21.15.139:6789/0 conn(0x7f04c40352f0 legacy=0x7f04c404dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:35.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.638+0000 7f04d2a0a640 1 -- 172.21.15.135:0/971723046 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c404ac00 con 0x7f04c4007210 2024-04-26T22:04:35.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.639+0000 7f04d2a0a640 1 --2- 172.21.15.135:0/971723046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c40076a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009870 tx=0x7f04bc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d172507a0bf088 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:35.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.640+0000 7f04d1207640 1 -- 172.21.15.135:0/971723046 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc037070 con 0x7f04c4007210 2024-04-26T22:04:35.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.640+0000 7f04d1207640 1 -- 172.21.15.135:0/971723046 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc032440 con 0x7f04c4007210 2024-04-26T22:04:35.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.641+0000 7f04d320b640 1 -- 172.21.15.135:0/971723046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 msgr2=0x7f04c40076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:35.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.641+0000 7f04d320b640 1 --2- 172.21.15.135:0/971723046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c40076a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009870 tx=0x7f04bc02a210 comp rx=0 tx=0).stop 2024-04-26T22:04:35.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.642+0000 7f04d320b640 1 -- 172.21.15.135:0/971723046 shutdown_connections 2024-04-26T22:04:35.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.642+0000 7f04d320b640 1 --2- 172.21.15.135:0/971723046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c40076a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:35.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.642+0000 7f04d320b640 1 --2- 172.21.15.135:0/971723046 >> v2:172.21.15.139:3300/0 conn(0x7f04c40dae00 0x7f04c400ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:35.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.642+0000 7f04d320b640 1 -- 172.21.15.135:0/971723046 >> 172.21.15.135:0/971723046 conn(0x7f04c4031730 msgr2=0x7f04c4031b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:35.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.643+0000 7f04d320b640 1 -- 172.21.15.135:0/971723046 shutdown_connections 2024-04-26T22:04:35.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.643+0000 7f04d320b640 1 -- 172.21.15.135:0/971723046 wait complete. 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.646+0000 7f04d320b640 1 Processor -- start 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.646+0000 7f04d320b640 1 -- start start 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.647+0000 7f04d320b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c4170900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.647+0000 7f04d320b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04c40352f0 0x7f04c40d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d320b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 0x7f04c40db770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d2209640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c4170900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d2209640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c4170900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:49742/0 (socket says 172.21.15.135:49742) 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d1a08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04c40352f0 0x7f04c40d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:35.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d1a08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04c40352f0 0x7f04c40d9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:42954/0 (socket says 172.21.15.135:42954) 2024-04-26T22:04:35.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d2a0a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 0x7f04c40db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:35.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.648+0000 7f04d2a0a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 0x7f04c40db770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44790/0 (socket says 172.21.15.135:44790) 2024-04-26T22:04:35.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.649+0000 7f04d2209640 1 -- 172.21.15.135:0/3224583651 learned_addr learned my addr 172.21.15.135:0/3224583651 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:35.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.649+0000 7f04d320b640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40db240 con 0x7f04c40dae00 2024-04-26T22:04:35.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.649+0000 7f04d320b640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40356d0 con 0x7f04c40352f0 2024-04-26T22:04:35.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.649+0000 7f04d320b640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40322d0 con 0x7f04c4007210 2024-04-26T22:04:35.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.652+0000 7f04d2a0a640 1 -- 172.21.15.135:0/3224583651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 msgr2=0x7f04c4170900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:35.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.652+0000 7f04d2a0a640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c4170900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:35.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.652+0000 7f04d2a0a640 1 -- 172.21.15.135:0/3224583651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04c40352f0 msgr2=0x7f04c40d9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:35.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.652+0000 7f04d2a0a640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04c40352f0 0x7f04c40d9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:35.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.652+0000 7f04d2a0a640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c40da610 con 0x7f04c40dae00 2024-04-26T22:04:35.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.653+0000 7f04d2a0a640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 0x7f04c40db770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f04c40dea30 tx=0x7f04bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:35.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.653+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc037040 con 0x7f04c40dae00 2024-04-26T22:04:35.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.653+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc042070 con 0x7f04c40dae00 2024-04-26T22:04:35.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.653+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0327f0 con 0x7f04c40dae00 2024-04-26T22:04:35.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.654+0000 7f04d320b640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c4046150 con 0x7f04c40dae00 2024-04-26T22:04:35.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.654+0000 7f04d320b640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c4170100 con 0x7f04c40dae00 2024-04-26T22:04:35.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.656+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b4005290 con 0x7f04c40dae00 2024-04-26T22:04:35.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.660+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04bc009330 con 0x7f04c40dae00 2024-04-26T22:04:35.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.660+0000 7f04c2ffd640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f04b006c9c0 0x7f04b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:35.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.660+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 7096+0+0 (secure 0 0 0) 0x7f04bc0ac670 con 0x7f04c40dae00 2024-04-26T22:04:35.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.661+0000 7f04d2209640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f04b006c9c0 0x7f04b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:35.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.662+0000 7f04d2209640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f04b006c9c0 0x7f04b006ee80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04c4046760 tx=0x7f04c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:35.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:35.667+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04bc079ec0 con 0x7f04c40dae00 2024-04-26T22:04:36.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.165+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04b4005580 con 0x7f04c40dae00 2024-04-26T22:04:36.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.166+0000 7f04c2ffd640 1 -- 172.21.15.135:0/3224583651 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6500 (secure 0 0 0) 0x7f04bc07e1c0 con 0x7f04c40dae00 2024-04-26T22:04:36.169 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:36.169 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:03:15.024868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5268":{"gid":5268,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.135:6835/2910412510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2910412510},{"type":"v1","addr":"172.21.15.135:6835","nonce":2910412510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5268,"qdb_cluster":[5268]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:36.170 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 38 2024-04-26T22:04:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f04b006c9c0 msgr2=0x7f04b006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f04b006c9c0 0x7f04b006ee80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04c4046760 tx=0x7f04c8009040 comp rx=0 tx=0).stop 2024-04-26T22:04:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 msgr2=0x7f04c40db770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 0x7f04c40db770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f04c40dea30 tx=0x7f04bc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:04:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 shutdown_connections 2024-04-26T22:04:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f04b006c9c0 0x7f04b006ee80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:36.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04c40dae00 0x7f04c40db770 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:36.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04c40352f0 0x7f04c40d9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:36.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 --2- 172.21.15.135:0/3224583651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04c4007210 0x7f04c4170900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:36.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 >> 172.21.15.135:0/3224583651 conn(0x7f04c4031730 msgr2=0x7f04c4033830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:36.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 shutdown_connections 2024-04-26T22:04:36.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:36.173+0000 7f04c0ff9640 1 -- 172.21.15.135:0/3224583651 wait complete. 2024-04-26T22:04:36.194 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:04:38.196 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:38.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.370+0000 7f559490a640 1 Processor -- start 2024-04-26T22:04:38.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.370+0000 7f559490a640 1 -- start start 2024-04-26T22:04:38.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.372+0000 7f559490a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f559015a280 0x7f55900c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:38.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.372+0000 7f559490a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5590158180 0x7f55901585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:38.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.372+0000 7f559490a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55900c8a30 con 0x7f559013ffb0 2024-04-26T22:04:38.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.372+0000 7f559490a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5590072be0 con 0x7f5590158180 2024-04-26T22:04:38.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.372+0000 7f559490a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f55900c9640 con 0x7f559015a280 2024-04-26T22:04:38.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5590158180 0x7f55901585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:38.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5590158180 0x7f55901585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44792/0 (socket says 172.21.15.135:44792) 2024-04-26T22:04:38.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 -- 172.21.15.135:0/4082038230 learned_addr learned my addr 172.21.15.135:0/4082038230 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:38.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 -- 172.21.15.135:0/4082038230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f559015a280 msgr2=0x7f55900c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:38.373 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 --2- 172.21.15.135:0/4082038230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f559015a280 0x7f55900c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 -- 172.21.15.135:0/4082038230 >> v1:172.21.15.135:6789/0 conn(0x7f559013ffb0 legacy=0x7f5590072ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:38.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.373+0000 7f558ed9d640 1 -- 172.21.15.135:0/4082038230 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55900c8d10 con 0x7f5590158180 2024-04-26T22:04:38.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.375+0000 7f558ed9d640 1 --2- 172.21.15.135:0/4082038230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5590158180 0x7f55901585f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5580009870 tx=0x7f558002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d486fe12844aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:38.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.375+0000 7f558d59a640 1 -- 172.21.15.135:0/4082038230 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580037070 con 0x7f5590158180 2024-04-26T22:04:38.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.375+0000 7f558d59a640 1 -- 172.21.15.135:0/4082038230 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5580032440 con 0x7f5590158180 2024-04-26T22:04:38.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.375+0000 7f558d59a640 1 -- 172.21.15.135:0/4082038230 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580032810 con 0x7f5590158180 2024-04-26T22:04:38.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.376+0000 7f559490a640 1 -- 172.21.15.135:0/4082038230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5590158180 msgr2=0x7f55901585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:38.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.376+0000 7f559490a640 1 --2- 172.21.15.135:0/4082038230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5590158180 0x7f55901585f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5580009870 tx=0x7f558002a180 comp rx=0 tx=0).stop 2024-04-26T22:04:38.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.376+0000 7f559490a640 1 -- 172.21.15.135:0/4082038230 shutdown_connections 2024-04-26T22:04:38.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.376+0000 7f559490a640 1 --2- 172.21.15.135:0/4082038230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f559015a280 0x7f55900c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.376+0000 7f559490a640 1 --2- 172.21.15.135:0/4082038230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5590158180 0x7f55901585f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.376+0000 7f559490a640 1 -- 172.21.15.135:0/4082038230 >> 172.21.15.135:0/4082038230 conn(0x7f559006dc10 msgr2=0x7f559006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:38.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.377+0000 7f559490a640 1 -- 172.21.15.135:0/4082038230 shutdown_connections 2024-04-26T22:04:38.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.377+0000 7f559490a640 1 -- 172.21.15.135:0/4082038230 wait complete. 2024-04-26T22:04:38.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.379+0000 7f559490a640 1 Processor -- start 2024-04-26T22:04:38.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.379+0000 7f559490a640 1 -- start start 2024-04-26T22:04:38.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f559490a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f559013ffb0 0x7f55901f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f559490a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5590158180 0x7f559013ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f559490a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 0x7f559013b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f559490a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55900c92d0 con 0x7f559015a280 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f559490a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55900c87e0 con 0x7f559013ffb0 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f559490a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5590073700 con 0x7f5590158180 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f558ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 0x7f559013b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f558ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 0x7f559013b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44800/0 (socket says 172.21.15.135:44800) 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.380+0000 7f558ed9d640 1 -- 172.21.15.135:0/3703930809 learned_addr learned my addr 172.21.15.135:0/3703930809 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558dd9b640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5590158180 0x7f559013ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558ed9d640 1 -- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5590158180 msgr2=0x7f559013ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558ed9d640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5590158180 0x7f559013ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558ed9d640 1 -- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f559013ffb0 msgr2=0x7f55901f43b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558ed9d640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f559013ffb0 0x7f55901f43b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558ed9d640 1 -- 172.21.15.135:0/3703930809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55901f4df0 con 0x7f559015a280 2024-04-26T22:04:38.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.381+0000 7f558ed9d640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 0x7f559013b070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5580009870 tx=0x7f55800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:38.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.382+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580041070 con 0x7f559015a280 2024-04-26T22:04:38.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.382+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f558003c950 con 0x7f559015a280 2024-04-26T22:04:38.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.382+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580037040 con 0x7f559015a280 2024-04-26T22:04:38.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.382+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55900ad130 con 0x7f559015a280 2024-04-26T22:04:38.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.382+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55900ad6f0 con 0x7f559015a280 2024-04-26T22:04:38.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.383+0000 7f557cff9640 1 -- 172.21.15.135:0/3703930809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568005290 con 0x7f559015a280 2024-04-26T22:04:38.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.385+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5580044070 con 0x7f559015a280 2024-04-26T22:04:38.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.386+0000 7f557effd640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f557006c920 0x7f557006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:38.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.386+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f55800acff0 con 0x7f559015a280 2024-04-26T22:04:38.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.387+0000 7f558e59c640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f557006c920 0x7f557006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:38.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.390+0000 7f558e59c640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f557006c920 0x7f557006ede0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f558800ab90 tx=0x7f5588009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:38.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.393+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f558007aec0 con 0x7f559015a280 2024-04-26T22:04:38.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.894+0000 7f557cff9640 1 -- 172.21.15.135:0/3703930809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5568005580 con 0x7f559015a280 2024-04-26T22:04:38.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.895+0000 7f557effd640 1 -- 172.21.15.135:0/3703930809 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5664 (secure 0 0 0) 0x7f558007a7c0 con 0x7f559015a280 2024-04-26T22:04:38.896 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:38.896 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:37.805754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:38.896 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 39 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f557006c920 msgr2=0x7f557006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f557006c920 0x7f557006ede0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f558800ab90 tx=0x7f5588009230 comp rx=0 tx=0).stop 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 msgr2=0x7f559013b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 0x7f559013b070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5580009870 tx=0x7f55800046c0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 shutdown_connections 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f557006c920 0x7f557006ede0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559015a280 0x7f559013b070 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5590158180 0x7f559013ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 --2- 172.21.15.135:0/3703930809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f559013ffb0 0x7f55901f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:38.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.900+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 >> 172.21.15.135:0/3703930809 conn(0x7f559006dc10 msgr2=0x7f55900b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:38.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.901+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 shutdown_connections 2024-04-26T22:04:38.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:38.901+0000 7f559490a640 1 -- 172.21.15.135:0/3703930809 wait complete. 2024-04-26T22:04:38.920 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-04-26T22:04:38.920 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-04-26T22:04:38.920 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:39.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.086+0000 7f2475293640 1 Processor -- start 2024-04-26T22:04:39.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.086+0000 7f2475293640 1 -- start start 2024-04-26T22:04:39.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.087+0000 7f2475293640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24700abfc0 0x7f247015e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f2475293640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f24700c3750 0x7f247015da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f2475293640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247015edd0 con 0x7f247015dfa0 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f2475293640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f247015eba0 con 0x7f24700abfc0 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f2475293640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f247006e3d0 con 0x7f24700c3750 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24700abfc0 0x7f247015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24700abfc0 0x7f247015e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44922/0 (socket says 172.21.15.135:44922) 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 -- 172.21.15.135:0/3219242823 learned_addr learned my addr 172.21.15.135:0/3219242823 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:39.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246e7fc640 1 --2- 172.21.15.135:0/3219242823 >> v2:172.21.15.139:3300/0 conn(0x7f24700c3750 0x7f247015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 -- 172.21.15.135:0/3219242823 >> v2:172.21.15.139:3300/0 conn(0x7f24700c3750 msgr2=0x7f247015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 --2- 172.21.15.135:0/3219242823 >> v2:172.21.15.139:3300/0 conn(0x7f24700c3750 0x7f247015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 -- 172.21.15.135:0/3219242823 >> v1:172.21.15.139:6789/0 conn(0x7f247015dfa0 legacy=0x7f247006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.088+0000 7f246effd640 1 -- 172.21.15.135:0/3219242823 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247015f0b0 con 0x7f24700abfc0 2024-04-26T22:04:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.090+0000 7f246effd640 1 --2- 172.21.15.135:0/3219242823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24700abfc0 0x7f247015e660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f24700abaf0 tx=0x7f246002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f1baf1488aae15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.090+0000 7f246dffb640 1 -- 172.21.15.135:0/3219242823 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460037070 con 0x7f24700abfc0 2024-04-26T22:04:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.090+0000 7f246dffb640 1 -- 172.21.15.135:0/3219242823 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2460032440 con 0x7f24700abfc0 2024-04-26T22:04:39.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.090+0000 7f246dffb640 1 -- 172.21.15.135:0/3219242823 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460032830 con 0x7f24700abfc0 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.091+0000 7f2475293640 1 -- 172.21.15.135:0/3219242823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24700abfc0 msgr2=0x7f247015e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.091+0000 7f2475293640 1 --2- 172.21.15.135:0/3219242823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24700abfc0 0x7f247015e660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f24700abaf0 tx=0x7f246002a290 comp rx=0 tx=0).stop 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.091+0000 7f2475293640 1 -- 172.21.15.135:0/3219242823 shutdown_connections 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.091+0000 7f2475293640 1 --2- 172.21.15.135:0/3219242823 >> v2:172.21.15.139:3300/0 conn(0x7f24700c3750 0x7f247015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.091+0000 7f2475293640 1 --2- 172.21.15.135:0/3219242823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24700abfc0 0x7f247015e660 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.092+0000 7f2475293640 1 -- 172.21.15.135:0/3219242823 >> 172.21.15.135:0/3219242823 conn(0x7f247006c310 msgr2=0x7f247006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.092+0000 7f2475293640 1 -- 172.21.15.135:0/3219242823 shutdown_connections 2024-04-26T22:04:39.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.092+0000 7f2475293640 1 -- 172.21.15.135:0/3219242823 wait complete. 2024-04-26T22:04:39.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.094+0000 7f2475293640 1 Processor -- start 2024-04-26T22:04:39.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.094+0000 7f2475293640 1 -- start start 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.095+0000 7f2475293640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24700abfc0 0x7f24700c2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.095+0000 7f2475293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24700c3750 0x7f24700c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.095+0000 7f2475293640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f247015dfa0 0x7f24701f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.095+0000 7f2475293640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f247015e3b0 con 0x7f247015dfa0 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24700abfc0 0x7f24700c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24700abfc0 0x7f24700c2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:54020/0 (socket says 172.21.15.135:54020) 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246effd640 1 -- 172.21.15.135:0/1019071269 learned_addr learned my addr 172.21.15.135:0/1019071269 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246f7fe640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f247015dfa0 0x7f24701f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f2475293640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f247006e620 con 0x7f24700abfc0 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f2475293640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f247015c560 con 0x7f24700c3750 2024-04-26T22:04:39.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246e7fc640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24700c3750 0x7f24700c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246f7fe640 1 -- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24700c3750 msgr2=0x7f24700c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.096+0000 7f246f7fe640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24700c3750 0x7f24700c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.097+0000 7f246f7fe640 1 -- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24700abfc0 msgr2=0x7f24700c2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.097+0000 7f246f7fe640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24700abfc0 0x7f24700c2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.097+0000 7f246f7fe640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2470074c10 con 0x7f247015dfa0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.097+0000 7f246f7fe640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f247015dfa0 0x7f24701f3e30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f246800c730 tx=0x7f246800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.098+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468016020 con 0x7f247015dfa0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.098+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2468006d50 con 0x7f247015dfa0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.098+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468003bb0 con 0x7f247015dfa0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.098+0000 7f2475293640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24700bd660 con 0x7f247015dfa0 2024-04-26T22:04:39.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.098+0000 7f2475293640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24701f0d30 con 0x7f247015dfa0 2024-04-26T22:04:39.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.100+0000 7f2475293640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2450005290 con 0x7f247015dfa0 2024-04-26T22:04:39.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.101+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24680050d0 con 0x7f247015dfa0 2024-04-26T22:04:39.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.101+0000 7f24577fe640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f245c06c9c0 0x7f245c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:39.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.101+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f246808b620 con 0x7f247015dfa0 2024-04-26T22:04:39.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.102+0000 7f246effd640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f245c06c9c0 0x7f245c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:39.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.103+0000 7f246effd640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f245c06c9c0 0x7f245c06ee80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f246002a2d0 tx=0x7f246002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:39.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.110+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2470074c10 con 0x7f247015dfa0 2024-04-26T22:04:39.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.559+0000 7f2475293640 1 -- 172.21.15.135:0/1019071269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2450005580 con 0x7f247015dfa0 2024-04-26T22:04:39.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.561+0000 7f24577fe640 1 -- 172.21.15.135:0/1019071269 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5664 (secure 0 0 0) 0x7f2468058e30 con 0x7f247015dfa0 2024-04-26T22:04:39.561 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:39.561 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:37.805754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:39.561 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 39 2024-04-26T22:04:39.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.564+0000 7f24557fa640 1 -- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f245c06c9c0 msgr2=0x7f245c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:39.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.564+0000 7f24557fa640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f245c06c9c0 0x7f245c06ee80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f246002a2d0 tx=0x7f246002e000 comp rx=0 tx=0).stop 2024-04-26T22:04:39.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 -- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f247015dfa0 msgr2=0x7f24701f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:39.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f247015dfa0 0x7f24701f3e30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f246800c730 tx=0x7f246800cbf0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 -- 172.21.15.135:0/1019071269 shutdown_connections 2024-04-26T22:04:39.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f245c06c9c0 0x7f245c06ee80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f247015dfa0 0x7f24701f3e30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24700c3750 0x7f24700c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 --2- 172.21.15.135:0/1019071269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24700abfc0 0x7f24700c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:39.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 -- 172.21.15.135:0/1019071269 >> 172.21.15.135:0/1019071269 conn(0x7f247006c310 msgr2=0x7f24700c5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:39.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.565+0000 7f24557fa640 1 -- 172.21.15.135:0/1019071269 shutdown_connections 2024-04-26T22:04:39.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:39.566+0000 7f24557fa640 1 -- 172.21.15.135:0/1019071269 wait complete. 2024-04-26T22:04:39.587 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 39, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5272": { "addr": "172.21.15.135:6835/3214490783", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 3214490783, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 3214490783, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5272, "incarnation": 39, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:04:37.805754+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5272 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 37, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4973": { "addr": "172.21.15.139:6835/302589348", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 302589348, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 302589348, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4973, "incarnation": 34, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:04:22.870489+0000", "qdb_cluster": [ 4973 ], "qdb_leader": 4973, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4973 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:04:41.588 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:41.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.789+0000 7f90e6293640 1 Processor -- start 2024-04-26T22:04:41.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.790+0000 7f90e6293640 1 -- start start 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.791+0000 7f90e6293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e015abc0 0x7f90e00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.791+0000 7f90e6293640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f90e00bf4f0 0x7f90e00af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.791+0000 7f90e6293640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e015c150 con 0x7f90e00c8510 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.791+0000 7f90e6293640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f90e00c70e0 con 0x7f90e00bf4f0 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.791+0000 7f90e6293640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f90e00b1630 con 0x7f90e015abc0 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.792+0000 7f90e4830640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f90e00c8510 0x7f90e00b09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41552/0 (socket says 172.21.15.135:41552) 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.792+0000 7f90e4830640 1 -- 172.21.15.135:0/3698133378 learned_addr learned my addr 172.21.15.135:0/3698133378 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.793+0000 7f90dffff640 1 --2- 172.21.15.135:0/3698133378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e015abc0 0x7f90e00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.793+0000 7f90deffd640 1 -- 172.21.15.135:0/3698133378 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847176203 0 0) 0x7f90e015c150 con 0x7f90e00c8510 2024-04-26T22:04:41.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.793+0000 7f90deffd640 1 -- 172.21.15.135:0/3698133378 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90d0003520 con 0x7f90e00c8510 2024-04-26T22:04:41.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.794+0000 7f90dffff640 1 -- 172.21.15.135:0/3698133378 >> v2:172.21.15.135:3300/0 conn(0x7f90e00bf4f0 msgr2=0x7f90e00af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:41.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.794+0000 7f90dffff640 1 --2- 172.21.15.135:0/3698133378 >> v2:172.21.15.135:3300/0 conn(0x7f90e00bf4f0 0x7f90e00af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:41.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.794+0000 7f90dffff640 1 -- 172.21.15.135:0/3698133378 >> v1:172.21.15.135:6789/0 conn(0x7f90e00c8510 legacy=0x7f90e00b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:41.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.794+0000 7f90dffff640 1 -- 172.21.15.135:0/3698133378 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e015bde0 con 0x7f90e015abc0 2024-04-26T22:04:41.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.795+0000 7f90dffff640 1 --2- 172.21.15.135:0/3698133378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e015abc0 0x7f90e00b1f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f90d40097a0 tx=0x7f90d402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b5f0ff9eedc6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:41.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.795+0000 7f90deffd640 1 -- 172.21.15.135:0/3698133378 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4037070 con 0x7f90e015abc0 2024-04-26T22:04:41.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.796+0000 7f90deffd640 1 -- 172.21.15.135:0/3698133378 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d4032440 con 0x7f90e015abc0 2024-04-26T22:04:41.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.796+0000 7f90deffd640 1 -- 172.21.15.135:0/3698133378 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4032830 con 0x7f90e015abc0 2024-04-26T22:04:41.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.797+0000 7f90e6293640 1 -- 172.21.15.135:0/3698133378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e015abc0 msgr2=0x7f90e00b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:41.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.797+0000 7f90e6293640 1 --2- 172.21.15.135:0/3698133378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e015abc0 0x7f90e00b1f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f90d40097a0 tx=0x7f90d402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:04:41.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.797+0000 7f90e6293640 1 -- 172.21.15.135:0/3698133378 shutdown_connections 2024-04-26T22:04:41.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.797+0000 7f90e6293640 1 --2- 172.21.15.135:0/3698133378 >> v2:172.21.15.135:3300/0 conn(0x7f90e00bf4f0 0x7f90e00af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:41.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.797+0000 7f90e6293640 1 --2- 172.21.15.135:0/3698133378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e015abc0 0x7f90e00b1f00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:41.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.797+0000 7f90e6293640 1 -- 172.21.15.135:0/3698133378 >> 172.21.15.135:0/3698133378 conn(0x7f90e00c8d20 msgr2=0x7f90e00c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:41.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.798+0000 7f90e6293640 1 -- 172.21.15.135:0/3698133378 shutdown_connections 2024-04-26T22:04:41.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.798+0000 7f90e6293640 1 -- 172.21.15.135:0/3698133378 wait complete. 2024-04-26T22:04:41.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.800+0000 7f90e6293640 1 Processor -- start 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.800+0000 7f90e6293640 1 -- start start 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e6293640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e00bf4f0 0x7f90e01fa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e6293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e00c8510 0x7f90e00c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e6293640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 0x7f90e01fca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e6293640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90e00c8b40 con 0x7f90e015abc0 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e6293640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90e00c89c0 con 0x7f90e00bf4f0 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e6293640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f90e00befc0 con 0x7f90e00c8510 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e4830640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 0x7f90e01fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e4830640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 0x7f90e01fca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44952/0 (socket says 172.21.15.135:44952) 2024-04-26T22:04:41.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.802+0000 7f90e4830640 1 -- 172.21.15.135:0/2202077186 learned_addr learned my addr 172.21.15.135:0/2202077186 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:41.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.807+0000 7f90e4830640 1 -- 172.21.15.135:0/2202077186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e00c8510 msgr2=0x7f90e00c5930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:41.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.807+0000 7f90e4830640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e00c8510 0x7f90e00c5930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:41.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.807+0000 7f90e4830640 1 -- 172.21.15.135:0/2202077186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e00bf4f0 msgr2=0x7f90e01fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:41.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.807+0000 7f90e4830640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e00bf4f0 0x7f90e01fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:41.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.808+0000 7f90e4830640 1 -- 172.21.15.135:0/2202077186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e00ad650 con 0x7f90e015abc0 2024-04-26T22:04:41.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.808+0000 7f90e4830640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 0x7f90e01fca00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90d800cb40 tx=0x7f90d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:41.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.809+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8017070 con 0x7f90e015abc0 2024-04-26T22:04:41.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.809+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d800aca0 con 0x7f90e015abc0 2024-04-26T22:04:41.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.809+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8012620 con 0x7f90e015abc0 2024-04-26T22:04:41.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.809+0000 7f90e6293640 1 -- 172.21.15.135:0/2202077186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e00ad930 con 0x7f90e015abc0 2024-04-26T22:04:41.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.810+0000 7f90e6293640 1 -- 172.21.15.135:0/2202077186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e00c5410 con 0x7f90e015abc0 2024-04-26T22:04:41.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.815+0000 7f90e6293640 1 -- 172.21.15.135:0/2202077186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e00c5610 con 0x7f90e015abc0 2024-04-26T22:04:41.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.818+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90d801a070 con 0x7f90e015abc0 2024-04-26T22:04:41.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.819+0000 7f90dcff9640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90c406c9c0 0x7f90c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:41.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.819+0000 7f90dffff640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90c406c9c0 0x7f90c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:41.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.820+0000 7f90dffff640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90c406c9c0 0x7f90c406ee80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90d4034000 tx=0x7f90d402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:41.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.821+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f90d808beb0 con 0x7f90e015abc0 2024-04-26T22:04:41.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:41.824+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90e00c5610 con 0x7f90e015abc0 2024-04-26T22:04:42.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.285+0000 7f90e6293640 1 -- 172.21.15.135:0/2202077186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90e00af1a0 con 0x7f90e015abc0 2024-04-26T22:04:42.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.286+0000 7f90dcff9640 1 -- 172.21.15.135:0/2202077186 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5664 (secure 0 0 0) 0x7f90d8059d90 con 0x7f90e015abc0 2024-04-26T22:04:42.288 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:42.288 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:37.805754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:42.288 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 39 2024-04-26T22:04:42.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.292+0000 7f90ca7fc640 1 -- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90c406c9c0 msgr2=0x7f90c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:42.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.292+0000 7f90ca7fc640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90c406c9c0 0x7f90c406ee80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90d4034000 tx=0x7f90d402c000 comp rx=0 tx=0).stop 2024-04-26T22:04:42.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 -- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 msgr2=0x7f90e01fca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:42.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 0x7f90e01fca00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90d800cb40 tx=0x7f90d800a060 comp rx=0 tx=0).stop 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 -- 172.21.15.135:0/2202077186 shutdown_connections 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f90c406c9c0 0x7f90c406ee80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015abc0 0x7f90e01fca00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90e00c8510 0x7f90e00c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 --2- 172.21.15.135:0/2202077186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90e00bf4f0 0x7f90e01fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.293+0000 7f90ca7fc640 1 -- 172.21.15.135:0/2202077186 >> 172.21.15.135:0/2202077186 conn(0x7f90e00c8d20 msgr2=0x7f90e00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.294+0000 7f90ca7fc640 1 -- 172.21.15.135:0/2202077186 shutdown_connections 2024-04-26T22:04:42.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:42.294+0000 7f90ca7fc640 1 -- 172.21.15.135:0/2202077186 wait complete. 2024-04-26T22:04:42.314 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:04:44.315 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:44.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.526+0000 7f2f3167e640 1 Processor -- start 2024-04-26T22:04:44.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.526+0000 7f2f3167e640 1 -- start start 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f3167e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f2c0b0e80 0x7f2f2c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f3167e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2f2c0af390 0x7f2f2c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f3167e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f2c0cbd10 con 0x7f2f2c155330 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f3167e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2c0cbac0 con 0x7f2f2c0af390 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f3167e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2c0cbff0 con 0x7f2f2c0b0e80 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f23fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2f2c155330 0x7f2f2c15e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41554/0 (socket says 172.21.15.135:41554) 2024-04-26T22:04:44.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.527+0000 7f2f23fff640 1 -- 172.21.15.135:0/1453892225 learned_addr learned my addr 172.21.15.135:0/1453892225 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915095862 0 0) 0x7f2f2c0cbd10 con 0x7f2f2c155330 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f18003520 con 0x7f2f2c155330 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2710374363 0 0) 0x7f2f18003520 con 0x7f2f2c155330 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f2c0b0e80 msgr2=0x7f2f2c0b1260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f2c0b0e80 0x7f2f2c0b1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 >> v2:172.21.15.135:3300/0 conn(0x7f2f2c0af390 msgr2=0x7f2f2c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 --2- 172.21.15.135:0/1453892225 >> v2:172.21.15.135:3300/0 conn(0x7f2f2c0af390 0x7f2f2c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c15b260 con 0x7f2f2c155330 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.529+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2f14001670 con 0x7f2f2c155330 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 >> v1:172.21.15.135:6789/0 conn(0x7f2f2c155330 legacy=0x7f2f2c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f180039b0 0x7f2f18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f180053c0 0x7f2f1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0cbac0 con 0x7f2f180053c0 2024-04-26T22:04:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0cbff0 con 0x7f2f180039b0 2024-04-26T22:04:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1800c2b0 con 0x7f2f18004800 2024-04-26T22:04:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.530+0000 7f2f23fff640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.531+0000 7f2f2b7fe640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f180053c0 0x7f2f1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:44.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.531+0000 7f2f23fff640 1 -- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f180039b0 msgr2=0x7f2f18003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:44.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.531+0000 7f2f23fff640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f180039b0 0x7f2f18003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:44.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.531+0000 7f2f23fff640 1 -- 172.21.15.135:0/1453892225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f180053c0 msgr2=0x7f2f1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:44.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.531+0000 7f2f23fff640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f180053c0 0x7f2f1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:44.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.531+0000 7f2f23fff640 1 -- 172.21.15.135:0/1453892225 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f18003520 con 0x7f2f18004800 2024-04-26T22:04:44.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.534+0000 7f2f23fff640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f18004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c160 tx=0x7f2f18005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5132140718bab8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:44.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.539+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14007ca0 con 0x7f2f18004800 2024-04-26T22:04:44.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.539+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f14005af0 con 0x7f2f18004800 2024-04-26T22:04:44.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.539+0000 7f2f2a7fc640 1 -- 172.21.15.135:0/1453892225 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14007070 con 0x7f2f18004800 2024-04-26T22:04:44.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.540+0000 7f2f3167e640 1 -- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 msgr2=0x7f2f18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:44.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.540+0000 7f2f3167e640 1 --2- 172.21.15.135:0/1453892225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f18004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c160 tx=0x7f2f18005830 comp rx=0 tx=0).stop 2024-04-26T22:04:44.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.544+0000 7f2f2b7fe640 1 -- 172.21.15.135:0/1453892225 reap_dead start 2024-04-26T22:04:44.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.548+0000 7f2f3167e640 1 -- 172.21.15.135:0/1453892225 shutdown_connections 2024-04-26T22:04:44.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.548+0000 7f2f3167e640 1 -- 172.21.15.135:0/1453892225 >> 172.21.15.135:0/1453892225 conn(0x7f2f2c06cb90 msgr2=0x7f2f2c0ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:44.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.556+0000 7f2f3167e640 1 -- 172.21.15.135:0/1453892225 shutdown_connections 2024-04-26T22:04:44.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.557+0000 7f2f3167e640 1 -- 172.21.15.135:0/1453892225 wait complete. 2024-04-26T22:04:44.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.559+0000 7f2f3167e640 1 Processor -- start 2024-04-26T22:04:44.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.559+0000 7f2f3167e640 1 -- start start 2024-04-26T22:04:44.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.560+0000 7f2f3167e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f2c1f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.560+0000 7f2f3167e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 0x7f2f2c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.560+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 0x7f2f2c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:44.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.560+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 0x7f2f2c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44974/0 (socket says 172.21.15.135:44974) 2024-04-26T22:04:44.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.560+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f2c1f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:44.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.560+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f2c1f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48388/0 (socket says 172.21.15.135:48388) 2024-04-26T22:04:44.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.561+0000 7f2f23fff640 1 -- 172.21.15.135:0/612268111 learned_addr learned my addr 172.21.15.135:0/612268111 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:44.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.561+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f2c15e750 0x7f2f2c1f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0c9970 con 0x7f2f2c0c1300 2024-04-26T22:04:44.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0c97f0 con 0x7f2f2c15e750 2024-04-26T22:04:44.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2c0738b0 con 0x7f2f18004800 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f23fff640 1 -- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 msgr2=0x7f2f2c1f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f23fff640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f2c1f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f23fff640 1 -- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f2c15e750 msgr2=0x7f2f2c1f9200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f23fff640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f2c15e750 0x7f2f2c1f9200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.562+0000 7f2f23fff640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c15f390 con 0x7f2f2c0c1300 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.563+0000 7f2f23fff640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 0x7f2f2c15e210 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c280 tx=0x7f2f2c15b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.563+0000 7f2f2affd640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f2c1f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.564+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14001630 con 0x7f2f2c0c1300 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.564+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f14042070 con 0x7f2f2c0c1300 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.564+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14007270 con 0x7f2f2c0c1300 2024-04-26T22:04:44.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.564+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c15ebe0 con 0x7f2f2c0c1300 2024-04-26T22:04:44.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.566+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1f9f60 con 0x7f2f2c0c1300 2024-04-26T22:04:44.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.567+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f14007410 con 0x7f2f2c0c1300 2024-04-26T22:04:44.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.568+0000 7f2f237fe640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2f0c06c9f0 0x7f2f0c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:44.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.568+0000 7f2f2affd640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2f0c06c9f0 0x7f2f0c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:44.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.569+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c076650 con 0x7f2f2c0c1300 2024-04-26T22:04:44.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.569+0000 7f2f2affd640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2f0c06c9f0 0x7f2f0c06eeb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0c67a0 tx=0x7f2f1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:44.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.570+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f2f140ac4c0 con 0x7f2f2c0c1300 2024-04-26T22:04:44.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:44.580+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f14030380 con 0x7f2f2c0c1300 2024-04-26T22:04:45.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.057+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f2c0507a0 con 0x7f2f2c0c1300 2024-04-26T22:04:45.058 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.058+0000 7f2f237fe640 1 -- 172.21.15.135:0/612268111 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5664 (secure 0 0 0) 0x7f2f1407a3a0 con 0x7f2f2c0c1300 2024-04-26T22:04:45.058 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:45.058 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:37.805754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:45.059 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 39 2024-04-26T22:04:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2f0c06c9f0 msgr2=0x7f2f0c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2f0c06c9f0 0x7f2f0c06eeb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0c67a0 tx=0x7f2f1c008040 comp rx=0 tx=0).stop 2024-04-26T22:04:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 msgr2=0x7f2f2c15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 0x7f2f2c15e210 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c280 tx=0x7f2f2c15b540 comp rx=0 tx=0).stop 2024-04-26T22:04:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 shutdown_connections 2024-04-26T22:04:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2f0c06c9f0 0x7f2f0c06eeb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f2c15e750 0x7f2f2c1f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f2c0c1300 0x7f2f2c15e210 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 --2- 172.21.15.135:0/612268111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f18004800 0x7f2f2c1f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 >> 172.21.15.135:0/612268111 conn(0x7f2f2c06cb90 msgr2=0x7f2f2c0c7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 shutdown_connections 2024-04-26T22:04:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:45.062+0000 7f2f3167e640 1 -- 172.21.15.135:0/612268111 wait complete. 2024-04-26T22:04:45.085 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:04:47.086 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:47.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.294+0000 7f5aa1f3f640 1 Processor -- start 2024-04-26T22:04:47.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.296+0000 7f5aa1f3f640 1 -- start start 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5aa1f3f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5a9c150200 0x7f5a9c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5aa1f3f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5aa1f3f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9c06ff20 con 0x7f5a9c0ac680 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5aa1f3f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9c0c4bd0 con 0x7f5a9c150200 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5aa1f3f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9c06f4e0 con 0x7f5a9c0c2cd0 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5a9affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5a9affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c1533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48402/0 (socket says 172.21.15.135:48402) 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5a9affd640 1 -- 172.21.15.135:0/710583109 learned_addr learned my addr 172.21.15.135:0/710583109 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:47.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.297+0000 7f5a9b7fe640 1 --2- 172.21.15.135:0/710583109 >> v2:172.21.15.139:3300/0 conn(0x7f5a9c150200 0x7f5a9c0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:47.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.298+0000 7f5a9affd640 1 -- 172.21.15.135:0/710583109 >> v2:172.21.15.139:3300/0 conn(0x7f5a9c150200 msgr2=0x7f5a9c0ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:47.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.298+0000 7f5a9affd640 1 --2- 172.21.15.135:0/710583109 >> v2:172.21.15.139:3300/0 conn(0x7f5a9c150200 0x7f5a9c0ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.298+0000 7f5a9affd640 1 -- 172.21.15.135:0/710583109 >> v1:172.21.15.139:6789/0 conn(0x7f5a9c0ac680 legacy=0x7f5a9c156210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:47.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.298+0000 7f5a9affd640 1 -- 172.21.15.135:0/710583109 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c0c48f0 con 0x7f5a9c0c2cd0 2024-04-26T22:04:47.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.299+0000 7f5a9affd640 1 --2- 172.21.15.135:0/710583109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c1533b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00a910 tx=0x7f5a8c00a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad200a3f708db0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:47.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.300+0000 7f5a9a7fc640 1 -- 172.21.15.135:0/710583109 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c038070 con 0x7f5a9c0c2cd0 2024-04-26T22:04:47.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.300+0000 7f5a9a7fc640 1 -- 172.21.15.135:0/710583109 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8c030480 con 0x7f5a9c0c2cd0 2024-04-26T22:04:47.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.300+0000 7f5a9a7fc640 1 -- 172.21.15.135:0/710583109 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c030890 con 0x7f5a9c0c2cd0 2024-04-26T22:04:47.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.303+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/710583109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 msgr2=0x7f5a9c1533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:47.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.303+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/710583109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c1533b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00a910 tx=0x7f5a8c00a250 comp rx=0 tx=0).stop 2024-04-26T22:04:47.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.304+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/710583109 shutdown_connections 2024-04-26T22:04:47.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.304+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/710583109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c1533b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.304+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/710583109 >> v2:172.21.15.139:3300/0 conn(0x7f5a9c150200 0x7f5a9c0ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.304+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/710583109 >> 172.21.15.135:0/710583109 conn(0x7f5a9c0c82d0 msgr2=0x7f5a9c0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:47.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.311+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/710583109 shutdown_connections 2024-04-26T22:04:47.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.311+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/710583109 wait complete. 2024-04-26T22:04:47.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.313+0000 7f5aa1f3f640 1 Processor -- start 2024-04-26T22:04:47.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.313+0000 7f5aa1f3f640 1 -- start start 2024-04-26T22:04:47.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5aa1f3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 0x7f5a9c14cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:47.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5aa1f3f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5aa1f3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a9c150200 0x7f5a9c073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c0730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48414/0 (socket says 172.21.15.135:48414) 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 0x7f5a9c14cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 0x7f5a9c14cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:54048/0 (socket says 172.21.15.135:54048) 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a9c150200 0x7f5a9c073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a9c150200 0x7f5a9c073600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44982/0 (socket says 172.21.15.135:44982) 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5aa1f3f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c150760 con 0x7f5a9c150200 2024-04-26T22:04:47.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5aa1f3f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1505e0 con 0x7f5a9c0ac680 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5aa1f3f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9c0c8dc0 con 0x7f5a9c0c2cd0 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.315+0000 7f5a9affd640 1 -- 172.21.15.135:0/2445501096 learned_addr learned my addr 172.21.15.135:0/2445501096 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a9b7fe640 1 -- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 msgr2=0x7f5a9c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a9b7fe640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c0730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a9b7fe640 1 -- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a9c150200 msgr2=0x7f5a9c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a9b7fe640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a9c150200 0x7f5a9c073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a9b7fe640 1 -- 172.21.15.135:0/2445501096 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c14a7e0 con 0x7f5a9c0ac680 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a9b7fe640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 0x7f5a9c14cd10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a940074a0 tx=0x7f5a94007960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:47.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.316+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400d2d0 con 0x7f5a9c0ac680 2024-04-26T22:04:47.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.317+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c073b40 con 0x7f5a9c0ac680 2024-04-26T22:04:47.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.317+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c14a540 con 0x7f5a9c0ac680 2024-04-26T22:04:47.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.319+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9c0507a0 con 0x7f5a9c0ac680 2024-04-26T22:04:47.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.319+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9400d470 con 0x7f5a9c0ac680 2024-04-26T22:04:47.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.319+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400b940 con 0x7f5a9c0ac680 2024-04-26T22:04:47.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.322+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a9401d020 con 0x7f5a9c0ac680 2024-04-26T22:04:47.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.323+0000 7f5a83fff640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5a8806c8f0 0x7f5a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:47.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.323+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f5a9408d000 con 0x7f5a9c0ac680 2024-04-26T22:04:47.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.323+0000 7f5a9affd640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5a8806c8f0 0x7f5a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:47.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.324+0000 7f5a9affd640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5a8806c8f0 0x7f5a8806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c150990 tx=0x7f5a8c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:47.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.330+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a94024070 con 0x7f5a9c0ac680 2024-04-26T22:04:47.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.783+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a9c06fb70 con 0x7f5a9c0ac680 2024-04-26T22:04:47.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.784+0000 7f5a83fff640 1 -- 172.21.15.135:0/2445501096 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5664 (secure 0 0 0) 0x7f5a9c073b40 con 0x7f5a9c0ac680 2024-04-26T22:04:47.783 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:47.784 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:46.162644+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:47.784 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 41 2024-04-26T22:04:47.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.786+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5a8806c8f0 msgr2=0x7f5a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:47.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.786+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5a8806c8f0 0x7f5a8806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c150990 tx=0x7f5a8c02e000 comp rx=0 tx=0).stop 2024-04-26T22:04:47.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.786+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 msgr2=0x7f5a9c14cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:47.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.786+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 0x7f5a9c14cd10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a940074a0 tx=0x7f5a94007960 comp rx=0 tx=0).stop 2024-04-26T22:04:47.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.786+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 shutdown_connections 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5a8806c8f0 0x7f5a8806edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a9c150200 0x7f5a9c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a9c0c2cd0 0x7f5a9c0730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 --2- 172.21.15.135:0/2445501096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a9c0ac680 0x7f5a9c14cd10 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a940074a0 tx=0x7f5a94007960 comp rx=0 tx=0).stop 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 >> 172.21.15.135:0/2445501096 conn(0x7f5a9c0c82d0 msgr2=0x7f5a9c0aad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 shutdown_connections 2024-04-26T22:04:47.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:47.787+0000 7f5aa1f3f640 1 -- 172.21.15.135:0/2445501096 wait complete. 2024-04-26T22:04:47.802 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:04:49.803 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:49.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.982+0000 7f483e934640 1 Processor -- start 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.982+0000 7f483e934640 1 -- start start 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.983+0000 7f483e934640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4838154d20 0x7f483815b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.983+0000 7f483e934640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f48380cb6e0 0x7f4838073940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.983+0000 7f483e934640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483815e460 con 0x7f4838073e80 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.983+0000 7f483e934640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f483815e700 con 0x7f4838154d20 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.983+0000 7f483e934640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f483815e9e0 con 0x7f48380cb6e0 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.984+0000 7f483d932640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4838154d20 0x7f483815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:49.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.984+0000 7f483d932640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4838154d20 0x7f483815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47192/0 (socket says 172.21.15.135:47192) 2024-04-26T22:04:49.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.984+0000 7f483d932640 1 -- 172.21.15.135:0/1100994576 learned_addr learned my addr 172.21.15.135:0/1100994576 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:49.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.984+0000 7f483d131640 1 --2- 172.21.15.135:0/1100994576 >> v2:172.21.15.139:3300/0 conn(0x7f48380cb6e0 0x7f4838073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:49.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.985+0000 7f483d932640 1 -- 172.21.15.135:0/1100994576 >> v2:172.21.15.139:3300/0 conn(0x7f48380cb6e0 msgr2=0x7f4838073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:49.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.985+0000 7f483d932640 1 --2- 172.21.15.135:0/1100994576 >> v2:172.21.15.139:3300/0 conn(0x7f48380cb6e0 0x7f4838073940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:49.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.985+0000 7f483d932640 1 -- 172.21.15.135:0/1100994576 >> v1:172.21.15.135:6789/0 conn(0x7f4838073e80 legacy=0x7f48380ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:49.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.985+0000 7f483d932640 1 -- 172.21.15.135:0/1100994576 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483815ecc0 con 0x7f4838154d20 2024-04-26T22:04:49.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.989+0000 7f483d932640 1 --2- 172.21.15.135:0/1100994576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838154d20 0x7f483815b950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4838076c10 tx=0x7f483402a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ec86a689476efa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:49.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.989+0000 7f483c930640 1 -- 172.21.15.135:0/1100994576 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834037070 con 0x7f4838154d20 2024-04-26T22:04:49.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.990+0000 7f483c930640 1 -- 172.21.15.135:0/1100994576 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834032440 con 0x7f4838154d20 2024-04-26T22:04:49.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.990+0000 7f483e934640 1 -- 172.21.15.135:0/1100994576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838154d20 msgr2=0x7f483815b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:49.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 --2- 172.21.15.135:0/1100994576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838154d20 0x7f483815b950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4838076c10 tx=0x7f483402a400 comp rx=0 tx=0).stop 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 -- 172.21.15.135:0/1100994576 shutdown_connections 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 --2- 172.21.15.135:0/1100994576 >> v2:172.21.15.139:3300/0 conn(0x7f48380cb6e0 0x7f4838073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 --2- 172.21.15.135:0/1100994576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838154d20 0x7f483815b950 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 -- 172.21.15.135:0/1100994576 >> 172.21.15.135:0/1100994576 conn(0x7f48380c8da0 msgr2=0x7f48380c91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 -- 172.21.15.135:0/1100994576 shutdown_connections 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.991+0000 7f483e934640 1 -- 172.21.15.135:0/1100994576 wait complete. 2024-04-26T22:04:49.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.993+0000 7f483e934640 1 Processor -- start 2024-04-26T22:04:49.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.993+0000 7f483e934640 1 -- start start 2024-04-26T22:04:49.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.994+0000 7f483e934640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838073e80 0x7f4838159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:49.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.994+0000 7f483e934640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48380cb6e0 0x7f4838159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:49.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.994+0000 7f483e934640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 0x7f48381f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:49.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.994+0000 7f483e934640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f483815f1e0 con 0x7f4838073e80 2024-04-26T22:04:49.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.994+0000 7f483e934640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48380c6780 con 0x7f4838154d20 2024-04-26T22:04:49.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.994+0000 7f483e934640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f483815f360 con 0x7f48380cb6e0 2024-04-26T22:04:49.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.995+0000 7f483e133640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 0x7f48381f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:49.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.995+0000 7f483e133640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 0x7f48381f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:41988/0 (socket says 172.21.15.135:41988) 2024-04-26T22:04:49.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.995+0000 7f483e133640 1 -- 172.21.15.135:0/46826859 learned_addr learned my addr 172.21.15.135:0/46826859 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:50.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.996+0000 7f483e133640 1 -- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48380cb6e0 msgr2=0x7f4838159570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:50.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.996+0000 7f483e133640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48380cb6e0 0x7f4838159570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.996+0000 7f483d932640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838073e80 0x7f4838159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.996+0000 7f483e133640 1 -- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838073e80 msgr2=0x7f4838159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:50.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.996+0000 7f483e133640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838073e80 0x7f4838159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.996+0000 7f483e133640 1 -- 172.21.15.135:0/46826859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4838155c00 con 0x7f4838154d20 2024-04-26T22:04:50.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.997+0000 7f483d932640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838073e80 0x7f4838159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:50.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.997+0000 7f483e133640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 0x7f48381f0450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f482800c390 tx=0x7f482800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:50.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.997+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828017070 con 0x7f4838154d20 2024-04-26T22:04:50.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.997+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381583a0 con 0x7f4838154d20 2024-04-26T22:04:50.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.998+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483815dfc0 con 0x7f4838154d20 2024-04-26T22:04:50.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.998+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48280035f0 con 0x7f4838154d20 2024-04-26T22:04:50.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:49.998+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280039e0 con 0x7f4838154d20 2024-04-26T22:04:50.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.003+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4820005290 con 0x7f4838154d20 2024-04-26T22:04:50.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.005+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4828003b80 con 0x7f4838154d20 2024-04-26T22:04:50.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.006+0000 7f482e7fc640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f481c06c8f0 0x7f481c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.006+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f482808b3b0 con 0x7f4838154d20 2024-04-26T22:04:50.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.006+0000 7f483d932640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f481c06c8f0 0x7f481c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.007+0000 7f483d932640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f481c06c8f0 0x7f481c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4834034000 tx=0x7f483402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:50.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.017+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4828059290 con 0x7f4838154d20 2024-04-26T22:04:50.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.518+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4820005580 con 0x7f4838154d20 2024-04-26T22:04:50.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.520+0000 7f482e7fc640 1 -- 172.21.15.135:0/46826859 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5671 (secure 0 0 0) 0x7f4828058b90 con 0x7f4838154d20 2024-04-26T22:04:50.521 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:50.521 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:50.522 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 42 2024-04-26T22:04:50.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.525+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f481c06c8f0 msgr2=0x7f481c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:50.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.525+0000 7f483e934640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f481c06c8f0 0x7f481c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4834034000 tx=0x7f483402e000 comp rx=0 tx=0).stop 2024-04-26T22:04:50.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.525+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 msgr2=0x7f48381f0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:50.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.525+0000 7f483e934640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 0x7f48381f0450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f482800c390 tx=0x7f482800c850 comp rx=0 tx=0).stop 2024-04-26T22:04:50.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 shutdown_connections 2024-04-26T22:04:50.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f481c06c8f0 0x7f481c06edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4838154d20 0x7f48381f0450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48380cb6e0 0x7f4838159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 --2- 172.21.15.135:0/46826859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4838073e80 0x7f4838159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 >> 172.21.15.135:0/46826859 conn(0x7f48380c8da0 msgr2=0x7f4838153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:50.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 shutdown_connections 2024-04-26T22:04:50.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.526+0000 7f483e934640 1 -- 172.21.15.135:0/46826859 wait complete. 2024-04-26T22:04:50.548 INFO:tasks.mds_thrash.fs.[a]:mds.c has gained rank=0, replacing gid=5268 2024-04-26T22:04:50.548 INFO:tasks.mds_thrash.fs.[a]:waiting for 0 secs before reviving mds.a 2024-04-26T22:04:50.548 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-04-26T22:04:50.548 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:04:50.548 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:04:50.551 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:04:50.551 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:50.701 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-26T22:04:50.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.765+0000 7f5756231640 1 Processor -- start 2024-04-26T22:04:50.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.766+0000 7f5756231640 1 -- start start 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.767+0000 7f5756231640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 0x7f57480d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.767+0000 7f5756231640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57480decc0 0x7f57480e10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.767+0000 7f5756231640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57480468d0 con 0x7f57480d3c40 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.767+0000 7f5756231640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5748047660 con 0x7f57480decc0 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.767+0000 7f5756231640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f57480d65d0 con 0x7f57480d6a60 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.768+0000 7f5754a2e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 0x7f57480d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.768+0000 7f5754a2e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 0x7f57480d6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:42948/0 (socket says 172.21.15.135:42948) 2024-04-26T22:04:50.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.768+0000 7f5754a2e640 1 -- 172.21.15.135:0/2111599949 learned_addr learned my addr 172.21.15.135:0/2111599949 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:50.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.769+0000 7f5754a2e640 1 -- 172.21.15.135:0/2111599949 >> v2:172.21.15.139:3300/0 conn(0x7f57480decc0 msgr2=0x7f57480e10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:50.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.769+0000 7f5754a2e640 1 --2- 172.21.15.135:0/2111599949 >> v2:172.21.15.139:3300/0 conn(0x7f57480decc0 0x7f57480e10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.769+0000 7f5754a2e640 1 -- 172.21.15.135:0/2111599949 >> v1:172.21.15.135:6789/0 conn(0x7f57480d3c40 legacy=0x7f574804ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:50.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.773+0000 7f5747fff640 1 -- 172.21.15.135:0/2111599949 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2594238083 0 0) 0x7f57480468d0 con 0x7f57480d3c40 2024-04-26T22:04:50.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.775+0000 7f5754a2e640 1 -- 172.21.15.135:0/2111599949 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5748046bb0 con 0x7f57480d6a60 2024-04-26T22:04:50.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.776+0000 7f5754a2e640 1 --2- 172.21.15.135:0/2111599949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 0x7f57480d6ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f574000a910 tx=0x7f574002e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c7b9f101e4645b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:50.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.778+0000 7f5747fff640 1 -- 172.21.15.135:0/2111599949 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740031070 con 0x7f57480d6a60 2024-04-26T22:04:50.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.778+0000 7f5747fff640 1 -- 172.21.15.135:0/2111599949 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740035750 con 0x7f57480d6a60 2024-04-26T22:04:50.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.778+0000 7f5747fff640 1 -- 172.21.15.135:0/2111599949 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740035b60 con 0x7f57480d6a60 2024-04-26T22:04:50.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.779+0000 7f5756231640 1 -- 172.21.15.135:0/2111599949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 msgr2=0x7f57480d6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:50.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.779+0000 7f5756231640 1 --2- 172.21.15.135:0/2111599949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 0x7f57480d6ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f574000a910 tx=0x7f574002e8e0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.782+0000 7f5756231640 1 -- 172.21.15.135:0/2111599949 shutdown_connections 2024-04-26T22:04:50.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.782+0000 7f5756231640 1 --2- 172.21.15.135:0/2111599949 >> v2:172.21.15.139:3300/0 conn(0x7f57480decc0 0x7f57480e10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.782+0000 7f5756231640 1 --2- 172.21.15.135:0/2111599949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480d6a60 0x7f57480d6ee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.782+0000 7f5756231640 1 -- 172.21.15.135:0/2111599949 >> 172.21.15.135:0/2111599949 conn(0x7f5748005e90 msgr2=0x7f57480062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:50.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.783+0000 7f5756231640 1 -- 172.21.15.135:0/2111599949 shutdown_connections 2024-04-26T22:04:50.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.783+0000 7f5756231640 1 -- 172.21.15.135:0/2111599949 wait complete. 2024-04-26T22:04:50.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.785+0000 7f5756231640 1 Processor -- start 2024-04-26T22:04:50.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.785+0000 7f5756231640 1 -- start start 2024-04-26T22:04:50.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.786+0000 7f5756231640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480d3c40 0x7f57480cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.786+0000 7f5756231640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 0x7f574804a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.786+0000 7f5756231640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480decc0 0x7f574804ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.787+0000 7f5756231640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5748046360 con 0x7f57480d6a60 2024-04-26T22:04:50.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.787+0000 7f5756231640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f57480461e0 con 0x7f57480d3c40 2024-04-26T22:04:50.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.787+0000 7f5756231640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5748046e90 con 0x7f57480decc0 2024-04-26T22:04:50.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.788+0000 7f5755a30640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480decc0 0x7f574804ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.788+0000 7f575522f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480d3c40 0x7f57480cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.788+0000 7f5754a2e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 0x7f574804a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.788+0000 7f5754a2e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 0x7f574804a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47228/0 (socket says 172.21.15.135:47228) 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.788+0000 7f5754a2e640 1 -- 172.21.15.135:0/2791621568 learned_addr learned my addr 172.21.15.135:0/2791621568 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.789+0000 7f5754a2e640 1 -- 172.21.15.135:0/2791621568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480decc0 msgr2=0x7f574804ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.789+0000 7f5754a2e640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480decc0 0x7f574804ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.789+0000 7f5754a2e640 1 -- 172.21.15.135:0/2791621568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480d3c40 msgr2=0x7f57480cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.789+0000 7f5754a2e640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480d3c40 0x7f57480cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.789+0000 7f5754a2e640 1 -- 172.21.15.135:0/2791621568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57480cd250 con 0x7f57480d6a60 2024-04-26T22:04:50.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.789+0000 7f5754a2e640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 0x7f574804a5e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f574000aa30 tx=0x7f5740004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:50.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.790+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740035e50 con 0x7f57480d6a60 2024-04-26T22:04:50.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.790+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574816ce40 con 0x7f57480d6a60 2024-04-26T22:04:50.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.790+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574816d120 con 0x7f57480d6a60 2024-04-26T22:04:50.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.794+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740045070 con 0x7f57480d6a60 2024-04-26T22:04:50.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.794+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740038d90 con 0x7f57480d6a60 2024-04-26T22:04:50.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.794+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5740040470 con 0x7f57480d6a60 2024-04-26T22:04:50.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.795+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5730005290 con 0x7f57480d6a60 2024-04-26T22:04:50.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.795+0000 7f5745ffb640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f573406cb60 0x7f573406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:50.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.796+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f57400ad260 con 0x7f57480d6a60 2024-04-26T22:04:50.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.798+0000 7f575522f640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f573406cb60 0x7f573406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:50.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.799+0000 7f575522f640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f573406cb60 0x7f573406f020 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f574c000f60 tx=0x7f574c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:50.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:50.804+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5740010070 con 0x7f57480d6a60 2024-04-26T22:04:51.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.269+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5730005580 con 0x7f57480d6a60 2024-04-26T22:04:51.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.270+0000 7f5745ffb640 1 -- 172.21.15.135:0/2791621568 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7f574007b250 con 0x7f57480d6a60 2024-04-26T22:04:51.270 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:04:51.270 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:04:51.270 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:04:51.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.273+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f573406cb60 msgr2=0x7f573406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.273+0000 7f5756231640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f573406cb60 0x7f573406f020 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f574c000f60 tx=0x7f574c009040 comp rx=0 tx=0).stop 2024-04-26T22:04:51.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.273+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 msgr2=0x7f574804a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.273+0000 7f5756231640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 0x7f574804a5e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f574000aa30 tx=0x7f5740004730 comp rx=0 tx=0).stop 2024-04-26T22:04:51.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.275+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 shutdown_connections 2024-04-26T22:04:51.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.275+0000 7f5756231640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f573406cb60 0x7f573406f020 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.275+0000 7f5756231640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57480decc0 0x7f574804ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.275+0000 7f5756231640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57480d6a60 0x7f574804a5e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.275+0000 7f5756231640 1 --2- 172.21.15.135:0/2791621568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480d3c40 0x7f57480cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.275+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 >> 172.21.15.135:0/2791621568 conn(0x7f5748005e90 msgr2=0x7f57480d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.276+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 shutdown_connections 2024-04-26T22:04:51.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:04:51.276+0000 7f5756231640 1 -- 172.21.15.135:0/2791621568 wait complete. 2024-04-26T22:04:51.294 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-04-26T22:04:51.294 INFO:tasks.mds_thrash.fs.[a]:waiting for 67 secs before thrashing 2024-04-26T22:04:51.966 INFO:tasks.ceph.mds.c.smithi135.stderr:2024-04-26T22:04:51.966+0000 7f1970160640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:04:59.554 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:04:59.557 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:05:29.749 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:05:29.755 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:05:34.030 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.218+0000 7f7467fff640 1 Processor -- start 2024-04-26T22:05:34.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.220+0000 7f7467fff640 1 -- start start 2024-04-26T22:05:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.223+0000 7f7467fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600dbce0 0x7f74600474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.224+0000 7f7467fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f746000a8b0 0x7f746000ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.224+0000 7f7467fff640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746000bb50 con 0x7f74600df180 2024-04-26T22:05:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.224+0000 7f7467fff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f746000a420 con 0x7f746000a8b0 2024-04-26T22:05:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.224+0000 7f7467fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7460048060 con 0x7f74600dbce0 2024-04-26T22:05:34.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.225+0000 7f74677fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f746000a8b0 0x7f746000ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.226+0000 7f7466ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600dbce0 0x7f74600474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.226+0000 7f74677fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f746000a8b0 0x7f746000ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:35406/0 (socket says 172.21.15.135:35406) 2024-04-26T22:05:34.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.226+0000 7f74677fe640 1 -- 172.21.15.135:0/3797756905 learned_addr learned my addr 172.21.15.135:0/3797756905 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:34.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.227+0000 7f74677fe640 1 -- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600dbce0 msgr2=0x7f74600474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:34.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.227+0000 7f74677fe640 1 --2- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600dbce0 0x7f74600474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.227+0000 7f74677fe640 1 -- 172.21.15.135:0/3797756905 >> v1:172.21.15.139:6789/0 conn(0x7f74600df180 legacy=0x7f7460006ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:34.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.227+0000 7f74677fe640 1 -- 172.21.15.135:0/3797756905 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746000a0f0 con 0x7f746000a8b0 2024-04-26T22:05:34.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.230+0000 7f74677fe640 1 --2- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 0x7f746000ad00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f745c009870 tx=0x7f745c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f68471bf08843f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.231+0000 7f7465ffb640 1 -- 172.21.15.135:0/3797756905 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c037070 con 0x7f746000a8b0 2024-04-26T22:05:34.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.232+0000 7f7465ffb640 1 -- 172.21.15.135:0/3797756905 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c032440 con 0x7f746000a8b0 2024-04-26T22:05:34.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.232+0000 7f7465ffb640 1 -- 172.21.15.135:0/3797756905 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c032810 con 0x7f746000a8b0 2024-04-26T22:05:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.233+0000 7f7467fff640 1 -- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 msgr2=0x7f746000ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.233+0000 7f7467fff640 1 --2- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 0x7f746000ad00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f745c009870 tx=0x7f745c02a180 comp rx=0 tx=0).stop 2024-04-26T22:05:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.233+0000 7f7467fff640 1 -- 172.21.15.135:0/3797756905 shutdown_connections 2024-04-26T22:05:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.233+0000 7f7467fff640 1 --2- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 0x7f746000ad00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.233+0000 7f7467fff640 1 --2- 172.21.15.135:0/3797756905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600dbce0 0x7f74600474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.233+0000 7f7467fff640 1 -- 172.21.15.135:0/3797756905 >> 172.21.15.135:0/3797756905 conn(0x7f7460048250 msgr2=0x7f7460048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.235+0000 7f7467fff640 1 -- 172.21.15.135:0/3797756905 shutdown_connections 2024-04-26T22:05:34.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.241+0000 7f7467fff640 1 -- 172.21.15.135:0/3797756905 wait complete. 2024-04-26T22:05:34.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.243+0000 7f7467fff640 1 Processor -- start 2024-04-26T22:05:34.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.243+0000 7f7467fff640 1 -- start start 2024-04-26T22:05:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.245+0000 7f7467fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 0x7f7460043360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.245+0000 7f7467fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74600dbce0 0x7f7460030830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.245+0000 7f7467fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 0x7f746017ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.245+0000 7f7467fff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f746000b240 con 0x7f74600dbce0 2024-04-26T22:05:34.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.245+0000 7f7467fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7460047d80 con 0x7f746000a8b0 2024-04-26T22:05:34.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.245+0000 7f74677fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 0x7f746017ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.246+0000 7f74677fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 0x7f746017ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:56748/0 (socket says 172.21.15.135:56748) 2024-04-26T22:05:34.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.246+0000 7f74677fe640 1 -- 172.21.15.135:0/1045518231 learned_addr learned my addr 172.21.15.135:0/1045518231 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:34.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.246+0000 7f7467fff640 1 -- 172.21.15.135:0/1045518231 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f74600dc0c0 con 0x7f74600df180 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.246+0000 7f74667fc640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74600dbce0 0x7f7460030830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74667fc640 1 -- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 msgr2=0x7f746017ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74667fc640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 0x7f746017ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74667fc640 1 -- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 msgr2=0x7f7460043360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74667fc640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 0x7f7460043360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74667fc640 1 -- 172.21.15.135:0/1045518231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7460041870 con 0x7f74600dbce0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74667fc640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74600dbce0 0x7f7460030830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f745800b410 tx=0x7f745800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f74677fe640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 0x7f746017ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.247+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458015020 con 0x7f74600dbce0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.248+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7458011830 con 0x7f74600dbce0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.248+0000 7f7467fff640 1 -- 172.21.15.135:0/1045518231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74600428a0 con 0x7f74600dbce0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.248+0000 7f7467fff640 1 -- 172.21.15.135:0/1045518231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7460042b60 con 0x7f74600dbce0 2024-04-26T22:05:34.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.248+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458011be0 con 0x7f74600dbce0 2024-04-26T22:05:34.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.250+0000 7f7467fff640 1 -- 172.21.15.135:0/1045518231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7448005290 con 0x7f74600dbce0 2024-04-26T22:05:34.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.257+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7458018020 con 0x7f74600dbce0 2024-04-26T22:05:34.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.257+0000 7f744f7fe640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f745406c9c0 0x7f745406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.257+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f745808c670 con 0x7f74600dbce0 2024-04-26T22:05:34.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.261+0000 7f7466ffd640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f745406c9c0 0x7f745406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.262+0000 7f7466ffd640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f745406c9c0 0x7f745406ee80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7468051fc0 tx=0x7f746806b9a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.262+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f745801f020 con 0x7f74600dbce0 2024-04-26T22:05:34.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.762+0000 7f7467fff640 1 -- 172.21.15.135:0/1045518231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7448005580 con 0x7f74600dbce0 2024-04-26T22:05:34.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.763+0000 7f744f7fe640 1 -- 172.21.15.135:0/1045518231 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7f7460041870 con 0x7f74600dbce0 2024-04-26T22:05:34.766 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:34.766 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:34.766 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:34.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.771+0000 7f744d7fa640 1 -- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f745406c9c0 msgr2=0x7f745406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.771+0000 7f744d7fa640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f745406c9c0 0x7f745406ee80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7468051fc0 tx=0x7f746806b9a0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.771+0000 7f744d7fa640 1 -- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74600dbce0 msgr2=0x7f7460030830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.771+0000 7f744d7fa640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74600dbce0 0x7f7460030830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f745800b410 tx=0x7f745800b8d0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 -- 172.21.15.135:0/1045518231 shutdown_connections 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f745406c9c0 0x7f745406ee80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74600df180 0x7f746017ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74600dbce0 0x7f7460030830 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 --2- 172.21.15.135:0/1045518231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f746000a8b0 0x7f7460043360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 -- 172.21.15.135:0/1045518231 >> 172.21.15.135:0/1045518231 conn(0x7f7460048250 msgr2=0x7f74600071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.777+0000 7f744d7fa640 1 -- 172.21.15.135:0/1045518231 shutdown_connections 2024-04-26T22:05:34.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:34.778+0000 7f744d7fa640 1 -- 172.21.15.135:0/1045518231 wait complete. 2024-04-26T22:05:34.810 INFO:tasks.mds_thrash.fs.[b]:kill mds.b (rank=0) 2024-04-26T22:05:34.810 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T22:05:34.810 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:05:34.827 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T22:05:34.828 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.017+0000 7fe071a7e640 1 Processor -- start 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.017+0000 7fe071a7e640 1 -- start start 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.018+0000 7fe071a7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe06c13ddd0 0x7fe06c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.018+0000 7fe071a7e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe06c0707e0 con 0x7fe06c070f70 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.018+0000 7fe071a7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe06c155450 con 0x7fe06c071cc0 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.018+0000 7fe071a7e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe06c155730 con 0x7fe06c13ddd0 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.020+0000 7fe06bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe06c070f70 0x7fe06c15f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:36200/0 (socket says 172.21.15.135:36200) 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.020+0000 7fe06bfff640 1 -- 172.21.15.135:0/2106122454 learned_addr learned my addr 172.21.15.135:0/2106122454 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.021+0000 7fe06b7fe640 1 --2- 172.21.15.135:0/2106122454 >> v2:172.21.15.139:3300/0 conn(0x7fe06c13ddd0 0x7fe06c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:35.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.021+0000 7fe06a7fc640 1 -- 172.21.15.135:0/2106122454 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369856574 0 0) 0x7fe06c0707e0 con 0x7fe06c070f70 2024-04-26T22:05:35.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.021+0000 7fe06a7fc640 1 -- 172.21.15.135:0/2106122454 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe054003520 con 0x7fe06c070f70 2024-04-26T22:05:35.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.021+0000 7fe06b7fe640 1 -- 172.21.15.135:0/2106122454 >> v1:172.21.15.139:6789/0 conn(0x7fe06c071cc0 legacy=0x7fe06c073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.021+0000 7fe06b7fe640 1 -- 172.21.15.135:0/2106122454 >> v1:172.21.15.135:6789/0 conn(0x7fe06c070f70 legacy=0x7fe06c15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.021+0000 7fe06b7fe640 1 -- 172.21.15.135:0/2106122454 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c155a10 con 0x7fe06c13ddd0 2024-04-26T22:05:35.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.024+0000 7fe06b7fe640 1 --2- 172.21.15.135:0/2106122454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c13ddd0 0x7fe06c154b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe064009d90 tx=0x7fe06402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec0e649399db323 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:35.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.025+0000 7fe06a7fc640 1 -- 172.21.15.135:0/2106122454 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064007740 con 0x7fe06c13ddd0 2024-04-26T22:05:35.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.025+0000 7fe06a7fc640 1 -- 172.21.15.135:0/2106122454 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0640078e0 con 0x7fe06c13ddd0 2024-04-26T22:05:35.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.026+0000 7fe071a7e640 1 -- 172.21.15.135:0/2106122454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c13ddd0 msgr2=0x7fe06c154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.026+0000 7fe071a7e640 1 --2- 172.21.15.135:0/2106122454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c13ddd0 0x7fe06c154b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe064009d90 tx=0x7fe06402a660 comp rx=0 tx=0).stop 2024-04-26T22:05:35.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.030+0000 7fe071a7e640 1 -- 172.21.15.135:0/2106122454 shutdown_connections 2024-04-26T22:05:35.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.030+0000 7fe071a7e640 1 --2- 172.21.15.135:0/2106122454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c13ddd0 0x7fe06c154b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.030+0000 7fe071a7e640 1 -- 172.21.15.135:0/2106122454 >> 172.21.15.135:0/2106122454 conn(0x7fe06c0ca2f0 msgr2=0x7fe06c0ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:35.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.032+0000 7fe071a7e640 1 -- 172.21.15.135:0/2106122454 shutdown_connections 2024-04-26T22:05:35.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.032+0000 7fe071a7e640 1 -- 172.21.15.135:0/2106122454 wait complete. 2024-04-26T22:05:35.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.034+0000 7fe071a7e640 1 Processor -- start 2024-04-26T22:05:35.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.034+0000 7fe071a7e640 1 -- start start 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.035+0000 7fe071a7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c070f70 0x7fe06c1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.035+0000 7fe071a7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe06c071cc0 0x7fe06c1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.035+0000 7fe071a7e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06c13ddd0 0x7fe06c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.035+0000 7fe071a7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c13e330 con 0x7fe06c071cc0 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.035+0000 7fe071a7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c13e1b0 con 0x7fe06c070f70 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.035+0000 7fe071a7e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe06c155040 con 0x7fe06c13ddd0 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.036+0000 7fe06bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06c13ddd0 0x7fe06c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.036+0000 7fe06bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06c13ddd0 0x7fe06c156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:56760/0 (socket says 172.21.15.135:56760) 2024-04-26T22:05:35.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.036+0000 7fe06bfff640 1 -- 172.21.15.135:0/304182633 learned_addr learned my addr 172.21.15.135:0/304182633 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.036+0000 7fe06affd640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe06c071cc0 0x7fe06c1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.037+0000 7fe06affd640 1 -- 172.21.15.135:0/304182633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06c13ddd0 msgr2=0x7fe06c156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.037+0000 7fe06affd640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06c13ddd0 0x7fe06c156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.037+0000 7fe06affd640 1 -- 172.21.15.135:0/304182633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c070f70 msgr2=0x7fe06c1f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.037+0000 7fe06affd640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c070f70 0x7fe06c1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.037+0000 7fe06affd640 1 -- 172.21.15.135:0/304182633 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c156b60 con 0x7fe06c071cc0 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.037+0000 7fe06affd640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe06c071cc0 0x7fe06c1560e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c340 tx=0x7fe06000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.038+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060017070 con 0x7fe06c071cc0 2024-04-26T22:05:35.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.038+0000 7fe071a7e640 1 -- 172.21.15.135:0/304182633 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c1f9e00 con 0x7fe06c071cc0 2024-04-26T22:05:35.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.038+0000 7fe071a7e640 1 -- 172.21.15.135:0/304182633 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c1f9b20 con 0x7fe06c071cc0 2024-04-26T22:05:35.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.042+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0600052f0 con 0x7fe06c071cc0 2024-04-26T22:05:35.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.042+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060003830 con 0x7fe06c071cc0 2024-04-26T22:05:35.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.044+0000 7fe071a7e640 1 -- 172.21.15.135:0/304182633 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04c005290 con 0x7fe06c071cc0 2024-04-26T22:05:35.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.058+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe06001a070 con 0x7fe06c071cc0 2024-04-26T22:05:35.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.063+0000 7fe053fff640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe05806c9c0 0x7fe05806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:35.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.064+0000 7fe06b7fe640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe05806c9c0 0x7fe05806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:35.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.065+0000 7fe06b7fe640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe05806c9c0 0x7fe05806ee80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe064034000 tx=0x7fe06402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:35.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.065+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7fe06008b800 con 0x7fe06c071cc0 2024-04-26T22:05:35.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.065+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe060021020 con 0x7fe06c071cc0 2024-04-26T22:05:35.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.533+0000 7fe071a7e640 1 -- 172.21.15.135:0/304182633 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe04c005580 con 0x7fe06c071cc0 2024-04-26T22:05:35.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.535+0000 7fe053fff640 1 -- 172.21.15.135:0/304182633 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7fe060059150 con 0x7fe06c071cc0 2024-04-26T22:05:35.535 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:35.536 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:35.536 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:35.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.540+0000 7fe051ffb640 1 -- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe05806c9c0 msgr2=0x7fe05806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.540+0000 7fe051ffb640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe05806c9c0 0x7fe05806ee80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe064034000 tx=0x7fe06402e000 comp rx=0 tx=0).stop 2024-04-26T22:05:35.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.540+0000 7fe051ffb640 1 -- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe06c071cc0 msgr2=0x7fe06c1560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.540+0000 7fe051ffb640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe06c071cc0 0x7fe06c1560e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c340 tx=0x7fe06000c800 comp rx=0 tx=0).stop 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.541+0000 7fe051ffb640 1 -- 172.21.15.135:0/304182633 shutdown_connections 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.541+0000 7fe051ffb640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe05806c9c0 0x7fe05806ee80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.541+0000 7fe051ffb640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06c13ddd0 0x7fe06c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.541+0000 7fe051ffb640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe06c071cc0 0x7fe06c1560e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.541+0000 7fe051ffb640 1 --2- 172.21.15.135:0/304182633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe06c070f70 0x7fe06c1f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.541+0000 7fe051ffb640 1 -- 172.21.15.135:0/304182633 >> 172.21.15.135:0/304182633 conn(0x7fe06c0ca2f0 msgr2=0x7fe06c13ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.542+0000 7fe051ffb640 1 -- 172.21.15.135:0/304182633 shutdown_connections 2024-04-26T22:05:35.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:35.542+0000 7fe051ffb640 1 -- 172.21.15.135:0/304182633 wait complete. 2024-04-26T22:05:35.563 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:37.564 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:37.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.758+0000 7f0188fbe640 1 Processor -- start 2024-04-26T22:05:37.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.759+0000 7f0188fbe640 1 -- start start 2024-04-26T22:05:37.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.760+0000 7f0188fbe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f017c0dbce0 0x7f017c0474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0188fbe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f017c00a8b0 0x7f017c00ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0188fbe640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f017c00bb50 con 0x7f017c0df180 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0188fbe640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f017c00a420 con 0x7f017c00a8b0 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0183fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f017c00a8b0 0x7f017c00ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f01837fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f017c0dbce0 0x7f017c0474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0183fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f017c00a8b0 0x7f017c00ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53282/0 (socket says 172.21.15.135:53282) 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0183fff640 1 -- 172.21.15.135:0/3108802734 learned_addr learned my addr 172.21.15.135:0/3108802734 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f01837fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f017c0dbce0 0x7f017c0474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:56768/0 (socket says 172.21.15.135:56768) 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.761+0000 7f0188fbe640 1 -- 172.21.15.135:0/3108802734 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f017c048060 con 0x7f017c0dbce0 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.762+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 178279371 0 0) 0x7f017c00bb50 con 0x7f017c0df180 2024-04-26T22:05:37.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.762+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f016c003520 con 0x7f017c0df180 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3110939488 0 0) 0x7f016c003520 con 0x7f017c0df180 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f017c0dbce0 msgr2=0x7f017c0474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f017c0dbce0 0x7f017c0474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 >> v2:172.21.15.135:3300/0 conn(0x7f017c00a8b0 msgr2=0x7f017c00ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 --2- 172.21.15.135:0/3108802734 >> v2:172.21.15.135:3300/0 conn(0x7f017c00a8b0 0x7f017c00ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c00a0f0 con 0x7f017c0df180 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f0174001670 con 0x7f017c0df180 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 >> v1:172.21.15.135:6789/0 conn(0x7f017c0df180 legacy=0x7f017c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c0039b0 0x7f016c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f016c004800 0x7f016c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0051c0 0x7f016c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f017c00a420 con 0x7f016c004800 2024-04-26T22:05:37.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f017c048060 con 0x7f016c0051c0 2024-04-26T22:05:37.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.763+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f016c00c0e0 con 0x7f016c0039b0 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.764+0000 7f0182ffd640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f016c004800 0x7f016c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.764+0000 7f0182ffd640 1 -- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c0039b0 msgr2=0x7f016c003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.764+0000 7f0182ffd640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c0039b0 0x7f016c003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.764+0000 7f0182ffd640 1 -- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0051c0 msgr2=0x7f016c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.764+0000 7f0182ffd640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0051c0 0x7f016c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.765+0000 7f0182ffd640 1 -- 172.21.15.135:0/3108802734 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c003520 con 0x7f016c004800 2024-04-26T22:05:37.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.766+0000 7f0182ffd640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f016c004800 0x7f016c004c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0174007f90 tx=0x7f016c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe4dc17f4cbaf25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:37.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.766+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174036070 con 0x7f016c004800 2024-04-26T22:05:37.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.767+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0174031620 con 0x7f016c004800 2024-04-26T22:05:37.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.767+0000 7f01827fc640 1 -- 172.21.15.135:0/3108802734 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174007bb0 con 0x7f016c004800 2024-04-26T22:05:37.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.768+0000 7f0188fbe640 1 -- 172.21.15.135:0/3108802734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f016c004800 msgr2=0x7f016c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:37.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.768+0000 7f0188fbe640 1 --2- 172.21.15.135:0/3108802734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f016c004800 0x7f016c004c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0174007f90 tx=0x7f016c005830 comp rx=0 tx=0).stop 2024-04-26T22:05:37.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.768+0000 7f0183fff640 1 -- 172.21.15.135:0/3108802734 reap_dead start 2024-04-26T22:05:37.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.770+0000 7f0188fbe640 1 -- 172.21.15.135:0/3108802734 shutdown_connections 2024-04-26T22:05:37.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.770+0000 7f0188fbe640 1 -- 172.21.15.135:0/3108802734 >> 172.21.15.135:0/3108802734 conn(0x7f017c048250 msgr2=0x7f017c048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:37.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.772+0000 7f0188fbe640 1 -- 172.21.15.135:0/3108802734 shutdown_connections 2024-04-26T22:05:37.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.772+0000 7f0188fbe640 1 -- 172.21.15.135:0/3108802734 wait complete. 2024-04-26T22:05:37.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.774+0000 7f0188fbe640 1 Processor -- start 2024-04-26T22:05:37.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.774+0000 7f0188fbe640 1 -- start start 2024-04-26T22:05:37.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.775+0000 7f0188fbe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c004800 0x7f017c030ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.775+0000 7f0188fbe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 0x7f017c044bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0182ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 0x7f017c044bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0182ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 0x7f017c044bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53308/0 (socket says 172.21.15.135:53308) 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0182ffd640 1 -- 172.21.15.135:0/4195451916 learned_addr learned my addr 172.21.15.135:0/4195451916 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0188fbe640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f017c045130 0x7f017c0d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f017c00b240 con 0x7f017c04a640 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f017c047d80 con 0x7f017c045130 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.777+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f017c0dc0c0 con 0x7f016c004800 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.778+0000 7f0182ffd640 1 -- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c004800 msgr2=0x7f017c030ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.778+0000 7f0182ffd640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c004800 0x7f017c030ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.778+0000 7f0182ffd640 1 -- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f017c045130 msgr2=0x7f017c0d7f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:05:37.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.778+0000 7f0182ffd640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f017c045130 0x7f017c0d7f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:37.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.778+0000 7f0182ffd640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c02e520 con 0x7f017c04a640 2024-04-26T22:05:37.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.778+0000 7f0182ffd640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 0x7f017c044bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f017400c1d0 tx=0x7f017c00a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:37.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.779+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174036040 con 0x7f017c04a640 2024-04-26T22:05:37.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.779+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0174005270 con 0x7f017c04a640 2024-04-26T22:05:37.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.779+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740035f0 con 0x7f017c04a640 2024-04-26T22:05:37.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.779+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c02e800 con 0x7f017c04a640 2024-04-26T22:05:37.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.782+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0174003790 con 0x7f017c04a640 2024-04-26T22:05:37.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.782+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c02f590 con 0x7f017c04a640 2024-04-26T22:05:37.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.783+0000 7f016bfff640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f016c06e5c0 0x7f016c070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:37.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.783+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f01740ac130 con 0x7f017c04a640 2024-04-26T22:05:37.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.783+0000 7f01837fe640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f016c06e5c0 0x7f016c070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:37.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.784+0000 7f01837fe640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f016c06e5c0 0x7f016c070a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0184069b60 tx=0x7f0184076870 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:37.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.789+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c02f790 con 0x7f017c04a640 2024-04-26T22:05:37.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:37.799+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f017c02f790 con 0x7f017c04a640 2024-04-26T22:05:38.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.318+0000 7f0188fbe640 1 -- 172.21.15.135:0/4195451916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f017c005390 con 0x7f017c04a640 2024-04-26T22:05:38.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.320+0000 7f016bfff640 1 -- 172.21.15.135:0/4195451916 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7f017407a130 con 0x7f017c04a640 2024-04-26T22:05:38.322 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:38.322 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:38.322 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.329+0000 7f0169ffb640 1 -- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f016c06e5c0 msgr2=0x7f016c070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.329+0000 7f0169ffb640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f016c06e5c0 0x7f016c070a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0184069b60 tx=0x7f0184076870 comp rx=0 tx=0).stop 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.329+0000 7f0169ffb640 1 -- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 msgr2=0x7f017c044bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.329+0000 7f0169ffb640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 0x7f017c044bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f017400c1d0 tx=0x7f017c00a390 comp rx=0 tx=0).stop 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.329+0000 7f0169ffb640 1 -- 172.21.15.135:0/4195451916 shutdown_connections 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.329+0000 7f0169ffb640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f016c06e5c0 0x7f016c070a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:38.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.330+0000 7f0169ffb640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f017c045130 0x7f017c0d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.330+0000 7f0169ffb640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f017c04a640 0x7f017c044bf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.330+0000 7f0169ffb640 1 --2- 172.21.15.135:0/4195451916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f016c004800 0x7f017c030ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.330+0000 7f0169ffb640 1 -- 172.21.15.135:0/4195451916 >> 172.21.15.135:0/4195451916 conn(0x7f017c048250 msgr2=0x7f017c0055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.330+0000 7f0169ffb640 1 -- 172.21.15.135:0/4195451916 shutdown_connections 2024-04-26T22:05:38.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:38.330+0000 7f0169ffb640 1 -- 172.21.15.135:0/4195451916 wait complete. 2024-04-26T22:05:38.359 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:40.360 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:40.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.547+0000 7fe73d32b640 1 Processor -- start 2024-04-26T22:05:40.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.547+0000 7fe73d32b640 1 -- start start 2024-04-26T22:05:40.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.549+0000 7fe73d32b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe73806d590 0x7fe7381539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.550+0000 7fe73d32b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe738157be0 0x7fe738150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.550+0000 7fe73d32b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe738153ff0 con 0x7fe7380c5560 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.550+0000 7fe73d32b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe7381542d0 con 0x7fe738157be0 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.550+0000 7fe73d32b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe7381545b0 con 0x7fe73806d590 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe73806d590 0x7fe7381539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe73806d590 0x7fe7381539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:57032/0 (socket says 172.21.15.135:57032) 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 -- 172.21.15.135:0/618585026 learned_addr learned my addr 172.21.15.135:0/618585026 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 -- 172.21.15.135:0/618585026 >> v2:172.21.15.135:3300/0 conn(0x7fe738157be0 msgr2=0x7fe738150cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 --2- 172.21.15.135:0/618585026 >> v2:172.21.15.135:3300/0 conn(0x7fe738157be0 0x7fe738150cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 -- 172.21.15.135:0/618585026 >> v1:172.21.15.135:6789/0 conn(0x7fe7380c5560 legacy=0x7fe7381574d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:40.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.552+0000 7fe736ffd640 1 -- 172.21.15.135:0/618585026 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe738154890 con 0x7fe73806d590 2024-04-26T22:05:40.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.554+0000 7fe736ffd640 1 --2- 172.21.15.135:0/618585026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe7381539a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe7380c69c0 tx=0x7fe7280328b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34a385e2e522db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:40.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.554+0000 7fe735ffb640 1 -- 172.21.15.135:0/618585026 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72803f070 con 0x7fe73806d590 2024-04-26T22:05:40.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.554+0000 7fe735ffb640 1 -- 172.21.15.135:0/618585026 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7280114e0 con 0x7fe73806d590 2024-04-26T22:05:40.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.554+0000 7fe735ffb640 1 -- 172.21.15.135:0/618585026 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72803a680 con 0x7fe73806d590 2024-04-26T22:05:40.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.555+0000 7fe73d32b640 1 -- 172.21.15.135:0/618585026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 msgr2=0x7fe7381539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:40.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.555+0000 7fe73d32b640 1 --2- 172.21.15.135:0/618585026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe7381539a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe7380c69c0 tx=0x7fe7280328b0 comp rx=0 tx=0).stop 2024-04-26T22:05:40.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.556+0000 7fe73d32b640 1 -- 172.21.15.135:0/618585026 shutdown_connections 2024-04-26T22:05:40.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.556+0000 7fe73d32b640 1 --2- 172.21.15.135:0/618585026 >> v2:172.21.15.135:3300/0 conn(0x7fe738157be0 0x7fe738150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:40.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.556+0000 7fe73d32b640 1 --2- 172.21.15.135:0/618585026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe7381539a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:40.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.556+0000 7fe73d32b640 1 -- 172.21.15.135:0/618585026 >> 172.21.15.135:0/618585026 conn(0x7fe7380c5ce0 msgr2=0x7fe7380c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:40.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.560+0000 7fe73d32b640 1 -- 172.21.15.135:0/618585026 shutdown_connections 2024-04-26T22:05:40.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.560+0000 7fe73d32b640 1 -- 172.21.15.135:0/618585026 wait complete. 2024-04-26T22:05:40.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.562+0000 7fe73d32b640 1 Processor -- start 2024-04-26T22:05:40.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.562+0000 7fe73d32b640 1 -- start start 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.563+0000 7fe73d32b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe738152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.563+0000 7fe73d32b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 0x7fe7381e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 0x7fe7381e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 0x7fe7381e97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48480/0 (socket says 172.21.15.135:48480) 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe736ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe738152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 -- 172.21.15.135:0/734699419 learned_addr learned my addr 172.21.15.135:0/734699419 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe73d32b640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe738157be0 0x7fe7381e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7380c3c70 con 0x7fe7380c5560 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe738074b30 con 0x7fe73806d590 2024-04-26T22:05:40.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7380c3df0 con 0x7fe738157be0 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 -- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe738157be0 msgr2=0x7fe7381e9d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe738157be0 0x7fe7381e9d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 -- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 msgr2=0x7fe738152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe738152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.564+0000 7fe7367fc640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe738155540 con 0x7fe7380c5560 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.565+0000 7fe7367fc640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 0x7fe7381e97e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00b700 tx=0x7fe72c00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.565+0000 7fe736ffd640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe738152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.565+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c019070 con 0x7fe7380c5560 2024-04-26T22:05:40.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.566+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381e9140 con 0x7fe7380c5560 2024-04-26T22:05:40.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.568+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7380af1f0 con 0x7fe7380c5560 2024-04-26T22:05:40.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.569+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c00c800 con 0x7fe7380c5560 2024-04-26T22:05:40.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.569+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00cbf0 con 0x7fe7380c5560 2024-04-26T22:05:40.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.570+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe72c01c070 con 0x7fe7380c5560 2024-04-26T22:05:40.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.571+0000 7fe71f7fe640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe72406c9f0 0x7fe72406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:40.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.571+0000 7fe736ffd640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe72406c9f0 0x7fe72406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:40.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.572+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7fe72c08b440 con 0x7fe7380c5560 2024-04-26T22:05:40.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.572+0000 7fe736ffd640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe72406c9f0 0x7fe72406eeb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe72803c000 tx=0x7fe728008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:40.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.573+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe718005290 con 0x7fe7380c5560 2024-04-26T22:05:40.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:40.584+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe738155540 con 0x7fe7380c5560 2024-04-26T22:05:41.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.073+0000 7fe73d32b640 1 -- 172.21.15.135:0/734699419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe718005850 con 0x7fe7380c5560 2024-04-26T22:05:41.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.075+0000 7fe71f7fe640 1 -- 172.21.15.135:0/734699419 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7fe72c059380 con 0x7fe7380c5560 2024-04-26T22:05:41.076 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:41.076 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:41.077 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.080+0000 7fe71d7fa640 1 -- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe72406c9f0 msgr2=0x7fe72406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.080+0000 7fe71d7fa640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe72406c9f0 0x7fe72406eeb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe72803c000 tx=0x7fe728008040 comp rx=0 tx=0).stop 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.080+0000 7fe71d7fa640 1 -- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 msgr2=0x7fe7381e97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.080+0000 7fe71d7fa640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 0x7fe7381e97e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00b700 tx=0x7fe72c00bbc0 comp rx=0 tx=0).stop 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 -- 172.21.15.135:0/734699419 shutdown_connections 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe72406c9f0 0x7fe72406eeb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:41.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe738157be0 0x7fe7381e9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:41.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7380c5560 0x7fe7381e97e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:41.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 --2- 172.21.15.135:0/734699419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe73806d590 0x7fe738152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:41.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 -- 172.21.15.135:0/734699419 >> 172.21.15.135:0/734699419 conn(0x7fe7380c5ce0 msgr2=0x7fe7380c7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:41.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 -- 172.21.15.135:0/734699419 shutdown_connections 2024-04-26T22:05:41.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:41.081+0000 7fe71d7fa640 1 -- 172.21.15.135:0/734699419 wait complete. 2024-04-26T22:05:41.100 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:43.101 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:43.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.310+0000 7f1a338b0640 1 Processor -- start 2024-04-26T22:05:43.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.311+0000 7f1a338b0640 1 -- start start 2024-04-26T22:05:43.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.312+0000 7f1a338b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1a2c0c5720 0x7f1a2c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:43.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.312+0000 7f1a338b0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a2c156e50 0x7f1a2c152b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:43.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.312+0000 7f1a338b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2c0c4400 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.312+0000 7f1a338b0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2c156120 con 0x7f1a2c156e50 2024-04-26T22:05:43.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.312+0000 7f1a338b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1a2c1569c0 con 0x7f1a2c0c5720 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a2c156e50 0x7f1a2c152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a2c156e50 0x7f1a2c152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48486/0 (socket says 172.21.15.135:48486) 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 -- 172.21.15.135:0/3357298518 learned_addr learned my addr 172.21.15.135:0/3357298518 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 -- 172.21.15.135:0/3357298518 >> v2:172.21.15.139:3300/0 conn(0x7f1a2c0c5720 msgr2=0x7f1a2c14fd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 --2- 172.21.15.135:0/3357298518 >> v2:172.21.15.139:3300/0 conn(0x7f1a2c0c5720 0x7f1a2c14fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 -- 172.21.15.135:0/3357298518 >> v1:172.21.15.139:6789/0 conn(0x7f1a2c0c3f20 legacy=0x7f1a2c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.313+0000 7f1a30e4b640 1 -- 172.21.15.135:0/3357298518 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c156370 con 0x7f1a2c156e50 2024-04-26T22:05:43.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.314+0000 7f1a30e4b640 1 --2- 172.21.15.135:0/3357298518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 0x7f1a2c152b30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0c3630 tx=0x7f1a1c032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f114ea736408261 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.316+0000 7f1a23fff640 1 -- 172.21.15.135:0/3357298518 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c039070 con 0x7f1a2c156e50 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.316+0000 7f1a23fff640 1 -- 172.21.15.135:0/3357298518 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c033b10 con 0x7f1a2c156e50 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.317+0000 7f1a338b0640 1 -- 172.21.15.135:0/3357298518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 msgr2=0x7f1a2c152b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.317+0000 7f1a338b0640 1 --2- 172.21.15.135:0/3357298518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 0x7f1a2c152b30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0c3630 tx=0x7f1a1c032dc0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.317+0000 7f1a338b0640 1 -- 172.21.15.135:0/3357298518 shutdown_connections 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.317+0000 7f1a338b0640 1 --2- 172.21.15.135:0/3357298518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 0x7f1a2c152b30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.317+0000 7f1a338b0640 1 --2- 172.21.15.135:0/3357298518 >> v2:172.21.15.139:3300/0 conn(0x7f1a2c0c5720 0x7f1a2c14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.317+0000 7f1a338b0640 1 -- 172.21.15.135:0/3357298518 >> 172.21.15.135:0/3357298518 conn(0x7f1a2c0c89b0 msgr2=0x7f1a2c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:43.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.318+0000 7f1a338b0640 1 -- 172.21.15.135:0/3357298518 shutdown_connections 2024-04-26T22:05:43.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.322+0000 7f1a338b0640 1 -- 172.21.15.135:0/3357298518 wait complete. 2024-04-26T22:05:43.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.323+0000 7f1a338b0640 1 Processor -- start 2024-04-26T22:05:43.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.328+0000 7f1a338b0640 1 -- start start 2024-04-26T22:05:43.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a338b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 0x7f1a2c14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a338b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c0c5720 0x7f1a2c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a338b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 0x7f1a2c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a338b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0c3aa0 con 0x7f1a2c156e50 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a338b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0ae1e0 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a338b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2c0c5b00 con 0x7f1a2c0c5720 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.329+0000 7f1a3164c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 0x7f1a2c14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 0x7f1a2c14c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:57050/0 (socket says 172.21.15.135:57050) 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 -- 172.21.15.135:0/137906526 learned_addr learned my addr 172.21.15.135:0/137906526 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:43.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 -- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c0c5720 msgr2=0x7f1a2c0ab8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:43.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c0c5720 0x7f1a2c0ab8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 -- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 msgr2=0x7f1a2c143d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:43.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 0x7f1a2c143d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.330+0000 7f1a3164c640 1 -- 172.21.15.135:0/137906526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c0ad110 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.332+0000 7f1a3164c640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 0x7f1a2c14c1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400ba50 tx=0x7f1a2400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:43.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.333+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24019070 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.334+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c14caa0 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.334+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c14d060 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.334+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2400cc00 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.334+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24014440 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.336+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c1439d0 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.337+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a2401c070 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.337+0000 7f1a21ffb640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1a1006c9c0 0x7f1a1006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:43.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.338+0000 7f1a30e4b640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1a1006c9c0 0x7f1a1006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:43.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.339+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f1a2408bf30 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.339+0000 7f1a30e4b640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1a1006c9c0 0x7f1a1006ee80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c03e000 tx=0x7f1a1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:43.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.345+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a24059e10 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.792+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a2c076100 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.793+0000 7f1a21ffb640 1 -- 172.21.15.135:0/137906526 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7f1a24059710 con 0x7f1a2c0c3f20 2024-04-26T22:05:43.793 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:43.793 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:43.793 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1a1006c9c0 msgr2=0x7f1a1006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1a1006c9c0 0x7f1a1006ee80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c03e000 tx=0x7f1a1c008040 comp rx=0 tx=0).stop 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 msgr2=0x7f1a2c14c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 0x7f1a2c14c1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400ba50 tx=0x7f1a2400bf10 comp rx=0 tx=0).stop 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 shutdown_connections 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1a1006c9c0 0x7f1a1006ee80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a2c156e50 0x7f1a2c143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c0c5720 0x7f1a2c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 --2- 172.21.15.135:0/137906526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0c3f20 0x7f1a2c14c1a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:43.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 >> 172.21.15.135:0/137906526 conn(0x7f1a2c0c89b0 msgr2=0x7f1a2c14f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:43.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 shutdown_connections 2024-04-26T22:05:43.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:43.796+0000 7f1a338b0640 1 -- 172.21.15.135:0/137906526 wait complete. 2024-04-26T22:05:43.811 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:45.812 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:45.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.997+0000 7f8a54446640 1 Processor -- start 2024-04-26T22:05:45.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.997+0000 7f8a54446640 1 -- start start 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.998+0000 7f8a54446640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8a4c0ab380 0x7f8a4c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.998+0000 7f8a54446640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a4c0c3e00 con 0x7f8a4c150300 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.998+0000 7f8a54446640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a4c0c40e0 con 0x7f8a4c0c4c90 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.998+0000 7f8a54446640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8a4c06e3d0 con 0x7f8a4c0ab380 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.999+0000 7f8a521e2640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8a4c150300 0x7f8a4c0c3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57272/0 (socket says 172.21.15.135:57272) 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.999+0000 7f8a521e2640 1 -- 172.21.15.135:0/120821625 learned_addr learned my addr 172.21.15.135:0/120821625 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:05:46.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:45.999+0000 7f8a519e1640 1 --2- 172.21.15.135:0/120821625 >> v2:172.21.15.139:3300/0 conn(0x7f8a4c0ab380 0x7f8a4c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:46.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.000+0000 7f8a519e1640 1 -- 172.21.15.135:0/120821625 >> v1:172.21.15.139:6789/0 conn(0x7f8a4c0c4c90 legacy=0x7f8a4c06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:46.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.000+0000 7f8a519e1640 1 -- 172.21.15.135:0/120821625 >> v1:172.21.15.135:6789/0 conn(0x7f8a4c150300 legacy=0x7f8a4c0c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:46.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.000+0000 7f8a519e1640 1 -- 172.21.15.135:0/120821625 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c0c43c0 con 0x7f8a4c0ab380 2024-04-26T22:05:46.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.001+0000 7f8a519e1640 1 --2- 172.21.15.135:0/120821625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c152cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c06e550 tx=0x7f8a3c02b2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307d6c19b29300e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:46.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.003+0000 7f8a511e0640 1 -- 172.21.15.135:0/120821625 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c038070 con 0x7f8a4c0ab380 2024-04-26T22:05:46.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.003+0000 7f8a511e0640 1 -- 172.21.15.135:0/120821625 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3c033440 con 0x7f8a4c0ab380 2024-04-26T22:05:46.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.003+0000 7f8a511e0640 1 -- 172.21.15.135:0/120821625 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c033810 con 0x7f8a4c0ab380 2024-04-26T22:05:46.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.004+0000 7f8a54446640 1 -- 172.21.15.135:0/120821625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0ab380 msgr2=0x7f8a4c152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:46.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.004+0000 7f8a54446640 1 --2- 172.21.15.135:0/120821625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c152cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c06e550 tx=0x7f8a3c02b2f0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.005+0000 7f8a54446640 1 -- 172.21.15.135:0/120821625 shutdown_connections 2024-04-26T22:05:46.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.005+0000 7f8a54446640 1 --2- 172.21.15.135:0/120821625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c152cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.005+0000 7f8a54446640 1 -- 172.21.15.135:0/120821625 >> 172.21.15.135:0/120821625 conn(0x7f8a4c06c310 msgr2=0x7f8a4c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:46.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.005+0000 7f8a54446640 1 -- 172.21.15.135:0/120821625 shutdown_connections 2024-04-26T22:05:46.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.005+0000 7f8a54446640 1 -- 172.21.15.135:0/120821625 wait complete. 2024-04-26T22:05:46.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.007+0000 7f8a54446640 1 Processor -- start 2024-04-26T22:05:46.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.007+0000 7f8a54446640 1 -- start start 2024-04-26T22:05:46.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.008+0000 7f8a54446640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:46.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.008+0000 7f8a521e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:46.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a521e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c0c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48490/0 (socket says 172.21.15.135:48490) 2024-04-26T22:05:46.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a521e2640 1 -- 172.21.15.135:0/1495223864 learned_addr learned my addr 172.21.15.135:0/1495223864 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:46.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a54446640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0c4c90 0x7f8a4c1f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a54446640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a4c150300 0x7f8a4c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a54446640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c06f250 con 0x7f8a4c0ab380 2024-04-26T22:05:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a519e1640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0c4c90 0x7f8a4c1f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.009+0000 7f8a529e3640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a4c150300 0x7f8a4c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a54446640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c06e6b0 con 0x7f8a4c0c4c90 2024-04-26T22:05:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a54446640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4c06f3d0 con 0x7f8a4c150300 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a521e2640 1 -- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a4c150300 msgr2=0x7f8a4c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a521e2640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a4c150300 0x7f8a4c15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a521e2640 1 -- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0c4c90 msgr2=0x7f8a4c1f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a521e2640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0c4c90 0x7f8a4c1f1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.010+0000 7f8a521e2640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c1498f0 con 0x7f8a4c0ab380 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.011+0000 7f8a521e2640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c0c7180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800c800 tx=0x7f8a4800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.012+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48018070 con 0x7f8a4c0ab380 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.012+0000 7f8a519e1640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0c4c90 0x7f8a4c1f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.012+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a480072d0 con 0x7f8a4c0ab380 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.012+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480047e0 con 0x7f8a4c0ab380 2024-04-26T22:05:46.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.012+0000 7f8a529e3640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a4c150300 0x7f8a4c15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:46.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.012+0000 7f8a54446640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c149050 con 0x7f8a4c0ab380 2024-04-26T22:05:46.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.020+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a48004980 con 0x7f8a4c0ab380 2024-04-26T22:05:46.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.020+0000 7f8a42ffd640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8a3006c920 0x7f8a3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:46.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.021+0000 7f8a519e1640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8a3006c920 0x7f8a3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:46.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.021+0000 7f8a54446640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c149610 con 0x7f8a4c0ab380 2024-04-26T22:05:46.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.022+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f8a48056af0 con 0x7f8a4c0ab380 2024-04-26T22:05:46.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.023+0000 7f8a519e1640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8a3006c920 0x7f8a3006ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c035000 tx=0x7f8a3c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:46.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.025+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4c0c3420 con 0x7f8a4c0ab380 2024-04-26T22:05:46.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.037+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a48059150 con 0x7f8a4c0ab380 2024-04-26T22:05:46.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.529+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a4c14b430 con 0x7f8a4c0ab380 2024-04-26T22:05:46.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.531+0000 7f8a42ffd640 1 -- 172.21.15.135:0/1495223864 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7f8a48058a50 con 0x7f8a4c0ab380 2024-04-26T22:05:46.531 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:46.531 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:46.531 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:46.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.534+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8a3006c920 msgr2=0x7f8a3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:46.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.534+0000 7f8a40ff9640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8a3006c920 0x7f8a3006ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c035000 tx=0x7f8a3c02f000 comp rx=0 tx=0).stop 2024-04-26T22:05:46.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.534+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 msgr2=0x7f8a4c0c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.534+0000 7f8a40ff9640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c0c7180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800c800 tx=0x7f8a4800ccc0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 shutdown_connections 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8a3006c920 0x7f8a3006ede0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a4c150300 0x7f8a4c15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4c0c4c90 0x7f8a4c1f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 --2- 172.21.15.135:0/1495223864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c0ab380 0x7f8a4c0c7180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 >> 172.21.15.135:0/1495223864 conn(0x7f8a4c06c310 msgr2=0x7f8a4c06eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 shutdown_connections 2024-04-26T22:05:46.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:46.535+0000 7f8a40ff9640 1 -- 172.21.15.135:0/1495223864 wait complete. 2024-04-26T22:05:46.554 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:48.555 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.731+0000 7fb1b7fff640 1 Processor -- start 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.731+0000 7fb1b7fff640 1 -- start start 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.732+0000 7fb1b7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb1b00d3ab0 0x7fb1b00d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.732+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 0x7fb1b00e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.732+0000 7fb1b7fff640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b0007050 con 0x7fb1b00d6d10 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.732+0000 7fb1b7fff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b0006e00 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.732+0000 7fb1b7fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b0007330 con 0x7fb1b00dece0 2024-04-26T22:05:48.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.733+0000 7fb1b77fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 0x7fb1b00e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:48.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.733+0000 7fb1b77fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 0x7fb1b00e10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:52308/0 (socket says 172.21.15.135:52308) 2024-04-26T22:05:48.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.733+0000 7fb1b77fe640 1 -- 172.21.15.135:0/3936452146 learned_addr learned my addr 172.21.15.135:0/3936452146 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:48.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.733+0000 7fb1b6ffd640 1 --2- 172.21.15.135:0/3936452146 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00d3ab0 0x7fb1b00d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:48.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.734+0000 7fb1b77fe640 1 -- 172.21.15.135:0/3936452146 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00d3ab0 msgr2=0x7fb1b00d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:48.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.734+0000 7fb1b77fe640 1 --2- 172.21.15.135:0/3936452146 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00d3ab0 0x7fb1b00d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:48.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.734+0000 7fb1b77fe640 1 -- 172.21.15.135:0/3936452146 >> v1:172.21.15.139:6789/0 conn(0x7fb1b00d6d10 legacy=0x7fb1b00465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:48.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.734+0000 7fb1b77fe640 1 -- 172.21.15.135:0/3936452146 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b00d6510 con 0x7fb1b00dece0 2024-04-26T22:05:48.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.735+0000 7fb1b77fe640 1 --2- 172.21.15.135:0/3936452146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 0x7fb1b00e10d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1b806bd80 tx=0x7fb1b80af6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8de4a875067fe8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:48.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.739+0000 7fb1b5ffb640 1 -- 172.21.15.135:0/3936452146 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80bd070 con 0x7fb1b00dece0 2024-04-26T22:05:48.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.739+0000 7fb1b5ffb640 1 -- 172.21.15.135:0/3936452146 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b80b5540 con 0x7fb1b00dece0 2024-04-26T22:05:48.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.739+0000 7fb1b5ffb640 1 -- 172.21.15.135:0/3936452146 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80b59b0 con 0x7fb1b00dece0 2024-04-26T22:05:48.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.741+0000 7fb1b7fff640 1 -- 172.21.15.135:0/3936452146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 msgr2=0x7fb1b00e10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:48.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.742+0000 7fb1b7fff640 1 --2- 172.21.15.135:0/3936452146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 0x7fb1b00e10d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1b806bd80 tx=0x7fb1b80af6e0 comp rx=0 tx=0).stop 2024-04-26T22:05:48.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.744+0000 7fb1b7fff640 1 -- 172.21.15.135:0/3936452146 shutdown_connections 2024-04-26T22:05:48.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.744+0000 7fb1b7fff640 1 --2- 172.21.15.135:0/3936452146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00dece0 0x7fb1b00e10d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:48.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.744+0000 7fb1b7fff640 1 --2- 172.21.15.135:0/3936452146 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00d3ab0 0x7fb1b00d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:48.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.744+0000 7fb1b7fff640 1 -- 172.21.15.135:0/3936452146 >> 172.21.15.135:0/3936452146 conn(0x7fb1b00062e0 msgr2=0x7fb1b00066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.744+0000 7fb1b7fff640 1 -- 172.21.15.135:0/3936452146 shutdown_connections 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.744+0000 7fb1b7fff640 1 -- 172.21.15.135:0/3936452146 wait complete. 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.746+0000 7fb1b7fff640 1 Processor -- start 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.747+0000 7fb1b7fff640 1 -- start start 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 0x7fb1b004a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 0x7fb1b004a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 0x7fb1b004a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36296/0 (socket says 172.21.15.135:36296) 2024-04-26T22:05:48.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b6ffd640 1 -- 172.21.15.135:0/131509092 learned_addr learned my addr 172.21.15.135:0/131509092 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b7fff640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00d6d10 0x7fb1b00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b7fff640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00dece0 0x7fb1b00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0046e30 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0046cb0 con 0x7fb1b00dece0 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.748+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b0009a60 con 0x7fb1b00d6d10 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.749+0000 7fb1b67fc640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00d6d10 0x7fb1b00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.749+0000 7fb1b6ffd640 1 -- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00d6d10 msgr2=0x7fb1b00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.749+0000 7fb1b6ffd640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00d6d10 0x7fb1b00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.749+0000 7fb1b6ffd640 1 -- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00dece0 msgr2=0x7fb1b00cb2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.749+0000 7fb1b6ffd640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00dece0 0x7fb1b00cb2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.749+0000 7fb1b6ffd640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b0049ac0 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.750+0000 7fb1b6ffd640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 0x7fb1b004a870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800d960 tx=0x7fb1a800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:48.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.751+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a801a070 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.751+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b00cb830 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.751+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b0049d60 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.753+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b00057e0 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.764+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a800c7f0 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.765+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800cbc0 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.765+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1a801d070 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.765+0000 7fb19ffff640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb1a406ca20 0x7fb1a406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:48.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.766+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7fb1a808bae0 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.766+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1a8024080 con 0x7fb1b00d3ab0 2024-04-26T22:05:48.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.767+0000 7fb1b67fc640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb1a406ca20 0x7fb1a406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:48.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:48.768+0000 7fb1b67fc640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb1a406ca20 0x7fb1a406eee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00469a0 tx=0x7fb1ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:49.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.242+0000 7fb1b7fff640 1 -- 172.21.15.135:0/131509092 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1b00466f0 con 0x7fb1b00d3ab0 2024-04-26T22:05:49.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.246+0000 7fb19ffff640 1 -- 172.21.15.135:0/131509092 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7fb1a80599c0 con 0x7fb1b00d3ab0 2024-04-26T22:05:49.247 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:49.247 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:49.247 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 -- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb1a406ca20 msgr2=0x7fb1a406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb1a406ca20 0x7fb1a406eee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00469a0 tx=0x7fb1ac009040 comp rx=0 tx=0).stop 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 -- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 msgr2=0x7fb1b004a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 0x7fb1b004a870 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800d960 tx=0x7fb1a800de20 comp rx=0 tx=0).stop 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 -- 172.21.15.135:0/131509092 shutdown_connections 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb1a406ca20 0x7fb1a406eee0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00469a0 tx=0x7fb1ac009040 comp rx=0 tx=0).stop 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00dece0 0x7fb1b00cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b00d6d10 0x7fb1b00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 --2- 172.21.15.135:0/131509092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1b00d3ab0 0x7fb1b004a870 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:49.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 -- 172.21.15.135:0/131509092 >> 172.21.15.135:0/131509092 conn(0x7fb1b00062e0 msgr2=0x7fb1b00d7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:49.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.250+0000 7fb19dffb640 1 -- 172.21.15.135:0/131509092 shutdown_connections 2024-04-26T22:05:49.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:49.251+0000 7fb19dffb640 1 -- 172.21.15.135:0/131509092 wait complete. 2024-04-26T22:05:49.272 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:51.273 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.448+0000 7f49ce44e640 1 Processor -- start 2024-04-26T22:05:51.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.448+0000 7f49ce44e640 1 -- start start 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.449+0000 7f49ce44e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c0047240 0x7f49c0007bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.449+0000 7f49ce44e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49c004adc0 0x7f49c004bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.449+0000 7f49ce44e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c004a250 con 0x7f49c004a9e0 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.449+0000 7f49ce44e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f49c004b250 con 0x7f49c004adc0 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.449+0000 7f49ce44e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f49c004ba30 con 0x7f49c0047240 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.450+0000 7f49cd44c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f49c004a9e0 0x7f49c00db000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:38208/0 (socket says 172.21.15.135:38208) 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.450+0000 7f49cd44c640 1 -- 172.21.15.135:0/4044111726 learned_addr learned my addr 172.21.15.135:0/4044111726 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.450+0000 7f49ccc4b640 1 --2- 172.21.15.135:0/4044111726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c0047240 0x7f49c0007bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.450+0000 7f49cdc4d640 1 --2- 172.21.15.135:0/4044111726 >> v2:172.21.15.135:3300/0 conn(0x7f49c004adc0 0x7f49c004bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.451+0000 7f49ccc4b640 1 -- 172.21.15.135:0/4044111726 >> v2:172.21.15.135:3300/0 conn(0x7f49c004adc0 msgr2=0x7f49c004bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.451+0000 7f49ccc4b640 1 --2- 172.21.15.135:0/4044111726 >> v2:172.21.15.135:3300/0 conn(0x7f49c004adc0 0x7f49c004bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.451+0000 7f49ccc4b640 1 -- 172.21.15.135:0/4044111726 >> v1:172.21.15.139:6789/0 conn(0x7f49c004a9e0 legacy=0x7f49c00db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:51.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.451+0000 7f49ccc4b640 1 -- 172.21.15.135:0/4044111726 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c00476d0 con 0x7f49c0047240 2024-04-26T22:05:51.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.451+0000 7f49cdc4d640 1 --2- 172.21.15.135:0/4044111726 >> v2:172.21.15.135:3300/0 conn(0x7f49c004adc0 0x7f49c004bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:51.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.452+0000 7f49ccc4b640 1 --2- 172.21.15.135:0/4044111726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c0047240 0x7f49c0007bc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f49b80097a0 tx=0x7f49b802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7cb3fb20261a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:51.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.453+0000 7f49bffff640 1 -- 172.21.15.135:0/4044111726 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8037070 con 0x7f49c0047240 2024-04-26T22:05:51.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.453+0000 7f49bffff640 1 -- 172.21.15.135:0/4044111726 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b802ad10 con 0x7f49c0047240 2024-04-26T22:05:51.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.454+0000 7f49ce44e640 1 -- 172.21.15.135:0/4044111726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c0047240 msgr2=0x7f49c0007bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:51.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.454+0000 7f49ce44e640 1 --2- 172.21.15.135:0/4044111726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c0047240 0x7f49c0007bc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f49b80097a0 tx=0x7f49b802a080 comp rx=0 tx=0).stop 2024-04-26T22:05:51.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.454+0000 7f49ce44e640 1 -- 172.21.15.135:0/4044111726 shutdown_connections 2024-04-26T22:05:51.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.454+0000 7f49ce44e640 1 --2- 172.21.15.135:0/4044111726 >> v2:172.21.15.135:3300/0 conn(0x7f49c004adc0 0x7f49c004bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.454+0000 7f49ce44e640 1 --2- 172.21.15.135:0/4044111726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c0047240 0x7f49c0007bc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.454+0000 7f49ce44e640 1 -- 172.21.15.135:0/4044111726 >> 172.21.15.135:0/4044111726 conn(0x7f49c0049340 msgr2=0x7f49c0049750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:51.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.455+0000 7f49ce44e640 1 -- 172.21.15.135:0/4044111726 shutdown_connections 2024-04-26T22:05:51.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.455+0000 7f49ce44e640 1 -- 172.21.15.135:0/4044111726 wait complete. 2024-04-26T22:05:51.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.457+0000 7f49ce44e640 1 Processor -- start 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.457+0000 7f49ce44e640 1 -- start start 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.458+0000 7f49ce44e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 0x7f49c00c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.458+0000 7f49ce44e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49c004adc0 0x7f49c00ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.458+0000 7f49ce44e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c00cf1d0 0x7f49c00cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.458+0000 7f49ce44e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0047c90 con 0x7f49c004adc0 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.458+0000 7f49ce44e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49c004b6c0 con 0x7f49c00cf1d0 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.458+0000 7f49ce44e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f49c004c400 con 0x7f49c004a9e0 2024-04-26T22:05:51.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.459+0000 7f49ccc4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49c004adc0 0x7f49c00ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.459+0000 7f49cd44c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 0x7f49c00c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.459+0000 7f49cd44c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 0x7f49c00c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:52340/0 (socket says 172.21.15.135:52340) 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.459+0000 7f49cd44c640 1 -- 172.21.15.135:0/3017823107 learned_addr learned my addr 172.21.15.135:0/3017823107 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.459+0000 7f49ccc4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49c004adc0 0x7f49c00ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36324/0 (socket says 172.21.15.135:36324) 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.459+0000 7f49cdc4d640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c00cf1d0 0x7f49c00cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.460+0000 7f49cd44c640 1 -- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c00cf1d0 msgr2=0x7f49c00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.460+0000 7f49cd44c640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c00cf1d0 0x7f49c00cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.460+0000 7f49cd44c640 1 -- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49c004adc0 msgr2=0x7f49c00ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.460+0000 7f49cd44c640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49c004adc0 0x7f49c00ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.460+0000 7f49cd44c640 1 -- 172.21.15.135:0/3017823107 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c00c60a0 con 0x7f49c004a9e0 2024-04-26T22:05:51.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.461+0000 7f49cd44c640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 0x7f49c00c5b60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c780 tx=0x7f49c400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:51.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.462+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4018070 con 0x7f49c004a9e0 2024-04-26T22:05:51.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.462+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c0043400 con 0x7f49c004a9e0 2024-04-26T22:05:51.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.462+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c40072d0 con 0x7f49c004a9e0 2024-04-26T22:05:51.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.462+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c40047e0 con 0x7f49c004a9e0 2024-04-26T22:05:51.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.462+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c00cecc0 con 0x7f49c004a9e0 2024-04-26T22:05:51.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.465+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c00057e0 con 0x7f49c004a9e0 2024-04-26T22:05:51.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.470+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49c401b070 con 0x7f49c004a9e0 2024-04-26T22:05:51.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.470+0000 7f49bdffb640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f49ac06c9c0 0x7f49ac06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:51.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.471+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 7182+0+0 (secure 0 0 0) 0x7f49c408b800 con 0x7f49c004a9e0 2024-04-26T22:05:51.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.475+0000 7f49ccc4b640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f49ac06c9c0 0x7f49ac06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:51.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.476+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49c4059650 con 0x7f49c004a9e0 2024-04-26T22:05:51.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.476+0000 7f49ccc4b640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f49ac06c9c0 0x7f49ac06ee80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f49b8034000 tx=0x7f49b803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:51.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.923+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49c00435d0 con 0x7f49c004a9e0 2024-04-26T22:05:51.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.924+0000 7f49bdffb640 1 -- 172.21.15.135:0/3017823107 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6499 (secure 0 0 0) 0x7f49c4058f50 con 0x7f49c004a9e0 2024-04-26T22:05:51.927 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:51.927 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:04:22.870489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4973":{"gid":4973,"name":"b","rank":0,"incarnation":34,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/302589348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":302589348},{"type":"v1","addr":"172.21.15.139:6835","nonce":302589348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4973,"qdb_cluster":[4973]},"id":2}]} 2024-04-26T22:05:51.927 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 43 2024-04-26T22:05:51.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f49ac06c9c0 msgr2=0x7f49ac06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:51.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f49ac06c9c0 0x7f49ac06ee80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f49b8034000 tx=0x7f49b803c040 comp rx=0 tx=0).stop 2024-04-26T22:05:51.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 msgr2=0x7f49c00c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 0x7f49c00c5b60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c780 tx=0x7f49c400cc40 comp rx=0 tx=0).stop 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 shutdown_connections 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f49ac06c9c0 0x7f49ac06ee80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c00cf1d0 0x7f49c00cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49c004adc0 0x7f49c00ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 --2- 172.21.15.135:0/3017823107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49c004a9e0 0x7f49c00c5b60 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c780 tx=0x7f49c400cc40 comp rx=0 tx=0).stop 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.930+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 >> 172.21.15.135:0/3017823107 conn(0x7f49c0049340 msgr2=0x7f49c002d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:51.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.931+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 shutdown_connections 2024-04-26T22:05:51.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:51.931+0000 7f49ce44e640 1 -- 172.21.15.135:0/3017823107 wait complete. 2024-04-26T22:05:51.952 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:05:53.953 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:54.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.186+0000 7f996917f640 1 Processor -- start 2024-04-26T22:05:54.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.187+0000 7f996917f640 1 -- start start 2024-04-26T22:05:54.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.188+0000 7f996917f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9964155330 0x7f99640b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:54.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.188+0000 7f996917f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99640af390 0x7f996415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:54.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.188+0000 7f996917f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99640cbd10 con 0x7f99640b0e80 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.188+0000 7f996917f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99640cbac0 con 0x7f9964155330 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.188+0000 7f996917f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99640cbff0 con 0x7f99640af390 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.189+0000 7f9962d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f99640b0e80 0x7f996415e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:48338/0 (socket says 172.21.15.135:48338) 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.189+0000 7f9962d9d640 1 -- 172.21.15.135:0/546144024 learned_addr learned my addr 172.21.15.135:0/546144024 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.189+0000 7f996359e640 1 --2- 172.21.15.135:0/546144024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99640af390 0x7f996415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.190+0000 7f996359e640 1 -- 172.21.15.135:0/546144024 >> v2:172.21.15.139:3300/0 conn(0x7f9964155330 msgr2=0x7f99640b1260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.190+0000 7f996359e640 1 --2- 172.21.15.135:0/546144024 >> v2:172.21.15.139:3300/0 conn(0x7f9964155330 0x7f99640b1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.190+0000 7f996359e640 1 -- 172.21.15.135:0/546144024 >> v1:172.21.15.135:6789/0 conn(0x7f99640b0e80 legacy=0x7f996415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:54.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.190+0000 7f996359e640 1 -- 172.21.15.135:0/546144024 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996415b260 con 0x7f99640af390 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.193+0000 7f996359e640 1 --2- 172.21.15.135:0/546144024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99640af390 0x7f996415ba60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f995c009d90 tx=0x7f995c02a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989fa662504ce812 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.193+0000 7f9961d9b640 1 -- 172.21.15.135:0/546144024 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c007740 con 0x7f99640af390 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.194+0000 7f9961d9b640 1 -- 172.21.15.135:0/546144024 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c0078e0 con 0x7f99640af390 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.195+0000 7f996917f640 1 -- 172.21.15.135:0/546144024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99640af390 msgr2=0x7f996415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.195+0000 7f996917f640 1 --2- 172.21.15.135:0/546144024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99640af390 0x7f996415ba60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f995c009d90 tx=0x7f995c02a590 comp rx=0 tx=0).stop 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.199+0000 7f996917f640 1 -- 172.21.15.135:0/546144024 shutdown_connections 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.199+0000 7f996917f640 1 --2- 172.21.15.135:0/546144024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99640af390 0x7f996415ba60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.199+0000 7f996917f640 1 --2- 172.21.15.135:0/546144024 >> v2:172.21.15.139:3300/0 conn(0x7f9964155330 0x7f99640b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.199+0000 7f996917f640 1 -- 172.21.15.135:0/546144024 >> 172.21.15.135:0/546144024 conn(0x7f996406cb90 msgr2=0x7f99640ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:54.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.206+0000 7f996917f640 1 -- 172.21.15.135:0/546144024 shutdown_connections 2024-04-26T22:05:54.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.208+0000 7f996917f640 1 -- 172.21.15.135:0/546144024 wait complete. 2024-04-26T22:05:54.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.210+0000 7f996917f640 1 Processor -- start 2024-04-26T22:05:54.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.214+0000 7f996917f640 1 -- start start 2024-04-26T22:05:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.215+0000 7f996917f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99640af390 0x7f99640c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.215+0000 7f996917f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 0x7f9964149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.215+0000 7f996917f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9964155330 0x7f9964149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.215+0000 7f996917f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99640c9970 con 0x7f99640b0e80 2024-04-26T22:05:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.215+0000 7f996917f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99640c97f0 con 0x7f99640af390 2024-04-26T22:05:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.215+0000 7f996917f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99640738b0 con 0x7f9964155330 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 0x7f9964149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 0x7f9964149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36334/0 (socket says 172.21.15.135:36334) 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 -- 172.21.15.135:0/3930133392 learned_addr learned my addr 172.21.15.135:0/3930133392 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f9962d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99640af390 0x7f99640c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 -- 172.21.15.135:0/3930133392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9964155330 msgr2=0x7f9964149770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9964155330 0x7f9964149770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 -- 172.21.15.135:0/3930133392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99640af390 msgr2=0x7f99640c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99640af390 0x7f99640c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.216+0000 7f996259c640 1 -- 172.21.15.135:0/3930133392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99641f1fe0 con 0x7f99640b0e80 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.217+0000 7f996259c640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 0x7f9964149230 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f995400b3e0 tx=0x7f995400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:54.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.218+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954015020 con 0x7f99640b0e80 2024-04-26T22:05:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.218+0000 7f996917f640 1 -- 172.21.15.135:0/3930133392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99641f4bf0 con 0x7f99640b0e80 2024-04-26T22:05:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.218+0000 7f996917f640 1 -- 172.21.15.135:0/3930133392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641f2560 con 0x7f99640b0e80 2024-04-26T22:05:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.218+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9954011850 con 0x7f99640b0e80 2024-04-26T22:05:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.218+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954011c40 con 0x7f99640b0e80 2024-04-26T22:05:54.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.222+0000 7f996917f640 1 -- 172.21.15.135:0/3930133392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944005290 con 0x7f99640b0e80 2024-04-26T22:05:54.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.245+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9954018020 con 0x7f99640b0e80 2024-04-26T22:05:54.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.246+0000 7f994b7fe640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f995006c9f0 0x7f995006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:54.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.246+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f995408bff0 con 0x7f99640b0e80 2024-04-26T22:05:54.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.247+0000 7f9962d9d640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f995006c9f0 0x7f995006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:54.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.248+0000 7f9962d9d640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f995006c9f0 0x7f995006eeb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f995800c7f0 tx=0x7f9958009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:54.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.251+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f995401f020 con 0x7f99640b0e80 2024-04-26T22:05:54.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.842+0000 7f996917f640 1 -- 172.21.15.135:0/3930133392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9944005580 con 0x7f99640b0e80 2024-04-26T22:05:54.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.843+0000 7f994b7fe640 1 -- 172.21.15.135:0/3930133392 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5666 (secure 0 0 0) 0x7f99540172c0 con 0x7f99640b0e80 2024-04-26T22:05:54.844 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:54.844 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:52.818469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:05:54.844 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 44 2024-04-26T22:05:54.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.848+0000 7f99497fa640 1 -- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f995006c9f0 msgr2=0x7f995006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:54.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.848+0000 7f99497fa640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f995006c9f0 0x7f995006eeb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f995800c7f0 tx=0x7f9958009040 comp rx=0 tx=0).stop 2024-04-26T22:05:54.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.848+0000 7f99497fa640 1 -- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 msgr2=0x7f9964149230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:54.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.848+0000 7f99497fa640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 0x7f9964149230 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f995400b3e0 tx=0x7f995400b8a0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.849+0000 7f99497fa640 1 -- 172.21.15.135:0/3930133392 shutdown_connections 2024-04-26T22:05:54.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.849+0000 7f99497fa640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f995006c9f0 0x7f995006eeb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.849+0000 7f99497fa640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9964155330 0x7f9964149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.849+0000 7f99497fa640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99640b0e80 0x7f9964149230 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.849+0000 7f99497fa640 1 --2- 172.21.15.135:0/3930133392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99640af390 0x7f99640c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:54.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.849+0000 7f99497fa640 1 -- 172.21.15.135:0/3930133392 >> 172.21.15.135:0/3930133392 conn(0x7f996406cb90 msgr2=0x7f99640acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:54.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.851+0000 7f99497fa640 1 -- 172.21.15.135:0/3930133392 shutdown_connections 2024-04-26T22:05:54.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:54.851+0000 7f99497fa640 1 -- 172.21.15.135:0/3930133392 wait complete. 2024-04-26T22:05:54.875 INFO:tasks.mds_thrash.fs.[b]:mds.b down, removed from mdsmap 2024-04-26T22:05:54.875 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:05:54.875 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:55.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.137+0000 7fad1eadc640 1 Processor -- start 2024-04-26T22:05:55.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.137+0000 7fad1eadc640 1 -- start start 2024-04-26T22:05:55.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad1eadc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 0x7fad1806e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:55.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad1eadc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad180aa8c0 0x7fad180c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:55.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad1eadc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad1815c0d0 con 0x7fad180c3f70 2024-04-26T22:05:55.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad1eadc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fad180ab4b0 con 0x7fad180aa8c0 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad1eadc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad1815bdf0 con 0x7fad180c3650 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad17fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad180aa8c0 0x7fad180c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad17fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad180aa8c0 0x7fad180c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33796/0 (socket says 172.21.15.135:33796) 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.138+0000 7fad17fff640 1 -- 172.21.15.135:0/4251968769 learned_addr learned my addr 172.21.15.135:0/4251968769 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.139+0000 7fad17fff640 1 -- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 msgr2=0x7fad1806e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.139+0000 7fad17fff640 1 --2- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 0x7fad1806e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.139+0000 7fad17fff640 1 -- 172.21.15.135:0/4251968769 >> v1:172.21.15.135:6789/0 conn(0x7fad180c3f70 legacy=0x7fad180abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:55.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.139+0000 7fad17fff640 1 -- 172.21.15.135:0/4251968769 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad180ab750 con 0x7fad180aa8c0 2024-04-26T22:05:55.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.144+0000 7fad17fff640 1 --2- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad180c3a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e50 tx=0x7fad1002a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b610e9b94c21dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:55.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.148+0000 7fad177fe640 1 -- 172.21.15.135:0/4251968769 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10036070 con 0x7fad180aa8c0 2024-04-26T22:05:55.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.148+0000 7fad177fe640 1 -- 172.21.15.135:0/4251968769 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1002ae50 con 0x7fad180aa8c0 2024-04-26T22:05:55.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.148+0000 7fad177fe640 1 -- 172.21.15.135:0/4251968769 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10031820 con 0x7fad180aa8c0 2024-04-26T22:05:55.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.149+0000 7fad1eadc640 1 -- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 msgr2=0x7fad180c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:55.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.149+0000 7fad1eadc640 1 --2- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad180c3a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e50 tx=0x7fad1002a630 comp rx=0 tx=0).stop 2024-04-26T22:05:55.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.150+0000 7fad1eadc640 1 -- 172.21.15.135:0/4251968769 shutdown_connections 2024-04-26T22:05:55.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.150+0000 7fad1eadc640 1 --2- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad180c3a30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.150+0000 7fad1eadc640 1 --2- 172.21.15.135:0/4251968769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 0x7fad1806e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.150+0000 7fad1eadc640 1 -- 172.21.15.135:0/4251968769 >> 172.21.15.135:0/4251968769 conn(0x7fad180c82d0 msgr2=0x7fad180c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:55.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.150+0000 7fad1eadc640 1 -- 172.21.15.135:0/4251968769 shutdown_connections 2024-04-26T22:05:55.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.150+0000 7fad1eadc640 1 -- 172.21.15.135:0/4251968769 wait complete. 2024-04-26T22:05:55.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.152+0000 7fad1eadc640 1 Processor -- start 2024-04-26T22:05:55.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.152+0000 7fad1eadc640 1 -- start start 2024-04-26T22:05:55.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.153+0000 7fad1eadc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad18073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:55.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.153+0000 7fad1eadc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 0x7fad18143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:55.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.153+0000 7fad1eadc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad180c3f70 0x7fad181457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:55.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.153+0000 7fad1eadc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad180ac7e0 con 0x7fad180c3f70 2024-04-26T22:05:55.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.154+0000 7fad1eadc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad180ac660 con 0x7fad180aa8c0 2024-04-26T22:05:55.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.154+0000 7fad1eadc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad1815a6a0 con 0x7fad180c3650 2024-04-26T22:05:55.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.154+0000 7fad1c878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad18073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:55.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.155+0000 7fad1c878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad18073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:33798/0 (socket says 172.21.15.135:33798) 2024-04-26T22:05:55.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.155+0000 7fad1c878640 1 -- 172.21.15.135:0/1328478985 learned_addr learned my addr 172.21.15.135:0/1328478985 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:55.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.156+0000 7fad1c878640 1 -- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 msgr2=0x7fad18143940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:55.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.156+0000 7fad1c878640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 0x7fad18143940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.156+0000 7fad1c878640 1 -- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad180c3f70 msgr2=0x7fad181457d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:55.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.159+0000 7fad1d079640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad180c3f70 0x7fad181457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:55.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.159+0000 7fad1c878640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad180c3f70 0x7fad181457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.159+0000 7fad1c878640 1 -- 172.21.15.135:0/1328478985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad18074240 con 0x7fad180aa8c0 2024-04-26T22:05:55.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.159+0000 7fad1d079640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad180c3f70 0x7fad181457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:55.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.159+0000 7fad1c878640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad18073c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b450 tx=0x7fad0c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:55.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.160+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c015020 con 0x7fad180aa8c0 2024-04-26T22:05:55.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.160+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c011890 con 0x7fad180aa8c0 2024-04-26T22:05:55.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.160+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c011c80 con 0x7fad180aa8c0 2024-04-26T22:05:55.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.162+0000 7fad1eadc640 1 -- 172.21.15.135:0/1328478985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad18071bc0 con 0x7fad180aa8c0 2024-04-26T22:05:55.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.162+0000 7fad1eadc640 1 -- 172.21.15.135:0/1328478985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad18072180 con 0x7fad180aa8c0 2024-04-26T22:05:55.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.166+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad0c018020 con 0x7fad180aa8c0 2024-04-26T22:05:55.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.166+0000 7fad157fa640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad0406c9f0 0x7fad0406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:55.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.166+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7fad0c08bc50 con 0x7fad180aa8c0 2024-04-26T22:05:55.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.167+0000 7fad17fff640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad0406c9f0 0x7fad0406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:55.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.165+0000 7fad1eadc640 1 -- 172.21.15.135:0/1328478985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad180c4ac0 con 0x7fad180aa8c0 2024-04-26T22:05:55.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.171+0000 7fad17fff640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad0406c9f0 0x7fad0406eeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fad10033000 tx=0x7fad10009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.177+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad0c01f070 con 0x7fad180aa8c0 2024-04-26T22:05:55.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.662+0000 7fad1eadc640 1 -- 172.21.15.135:0/1328478985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad180507a0 con 0x7fad180aa8c0 2024-04-26T22:05:55.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.667+0000 7fad157fa640 1 -- 172.21.15.135:0/1328478985 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5666 (secure 0 0 0) 0x7fad0c00d040 con 0x7fad180aa8c0 2024-04-26T22:05:55.671 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:55.671 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:52.818469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:05:55.671 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 44 2024-04-26T22:05:55.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.673+0000 7facfeffd640 1 -- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad0406c9f0 msgr2=0x7fad0406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:55.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.673+0000 7facfeffd640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad0406c9f0 0x7fad0406eeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fad10033000 tx=0x7fad10009000 comp rx=0 tx=0).stop 2024-04-26T22:05:55.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.673+0000 7facfeffd640 1 -- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 msgr2=0x7fad18073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:55.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.673+0000 7facfeffd640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad18073c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b450 tx=0x7fad0c00b910 comp rx=0 tx=0).stop 2024-04-26T22:05:55.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 -- 172.21.15.135:0/1328478985 shutdown_connections 2024-04-26T22:05:55.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad0406c9f0 0x7fad0406eeb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad180c3f70 0x7fad181457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad180c3650 0x7fad18143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 --2- 172.21.15.135:0/1328478985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad180aa8c0 0x7fad18073c60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:55.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 -- 172.21.15.135:0/1328478985 >> 172.21.15.135:0/1328478985 conn(0x7fad180c82d0 msgr2=0x7fad180c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:55.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 -- 172.21.15.135:0/1328478985 shutdown_connections 2024-04-26T22:05:55.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:55.674+0000 7facfeffd640 1 -- 172.21.15.135:0/1328478985 wait complete. 2024-04-26T22:05:55.699 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 44, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 42, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5272": { "addr": "172.21.15.135:6835/3214490783", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 3214490783, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 3214490783, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5272, "incarnation": 39, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:04:48.111880+0000", "qdb_cluster": [ 5272 ], "qdb_leader": 5272, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5272 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 44, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5610": { "addr": "172.21.15.135:6837/2165063360", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 2165063360, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 2165063360, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5610, "incarnation": 44, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:05:52.818469+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5610 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:05:57.701 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:57.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.890+0000 7f97f2140640 1 Processor -- start 2024-04-26T22:05:57.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.890+0000 7f97f2140640 1 -- start start 2024-04-26T22:05:57.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.892+0000 7f97f2140640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f97ec0c3650 0x7f97ec06e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.892+0000 7f97f2140640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0abf50 0x7f97ec0ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.892+0000 7f97f2140640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ec15c0d0 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.892+0000 7f97f2140640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f97ec0ab4b0 con 0x7f97ec0c3650 2024-04-26T22:05:57.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.892+0000 7f97f2140640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f97ec15bdf0 con 0x7f97ec0abf50 2024-04-26T22:05:57.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.894+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0abf50 0x7f97ec0ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.894+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0abf50 0x7f97ec0ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:52364/0 (socket says 172.21.15.135:52364) 2024-04-26T22:05:57.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.894+0000 7f97ebfff640 1 -- 172.21.15.135:0/1045784220 learned_addr learned my addr 172.21.15.135:0/1045784220 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:57.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.894+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/1045784220 >> v2:172.21.15.135:3300/0 conn(0x7f97ec0c3650 0x7f97ec06e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.895+0000 7f97eb7fe640 1 -- 172.21.15.135:0/1045784220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0abf50 msgr2=0x7f97ec0ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:57.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.895+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/1045784220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0abf50 0x7f97ec0ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.895+0000 7f97eb7fe640 1 -- 172.21.15.135:0/1045784220 >> v1:172.21.15.139:6789/0 conn(0x7f97ec0aa8c0 legacy=0x7f97ec0c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.895+0000 7f97eb7fe640 1 -- 172.21.15.135:0/1045784220 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec0ab750 con 0x7f97ec0c3650 2024-04-26T22:05:57.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.896+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/1045784220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec06e760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0097a0 tx=0x7f97dc02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3b173211f71d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.899+0000 7f97ea7fc640 1 -- 172.21.15.135:0/1045784220 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc037070 con 0x7f97ec0c3650 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.899+0000 7f97ea7fc640 1 -- 172.21.15.135:0/1045784220 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc02adb0 con 0x7f97ec0c3650 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.899+0000 7f97ea7fc640 1 -- 172.21.15.135:0/1045784220 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc032660 con 0x7f97ec0c3650 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.900+0000 7f97f2140640 1 -- 172.21.15.135:0/1045784220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 msgr2=0x7f97ec06e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.900+0000 7f97f2140640 1 --2- 172.21.15.135:0/1045784220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec06e760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0097a0 tx=0x7f97dc02a060 comp rx=0 tx=0).stop 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.900+0000 7f97f2140640 1 -- 172.21.15.135:0/1045784220 shutdown_connections 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.900+0000 7f97f2140640 1 --2- 172.21.15.135:0/1045784220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0abf50 0x7f97ec0ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.900+0000 7f97f2140640 1 --2- 172.21.15.135:0/1045784220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec06e760 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.900+0000 7f97f2140640 1 -- 172.21.15.135:0/1045784220 >> 172.21.15.135:0/1045784220 conn(0x7f97ec0c82d0 msgr2=0x7f97ec0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:57.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.901+0000 7f97f2140640 1 -- 172.21.15.135:0/1045784220 shutdown_connections 2024-04-26T22:05:57.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.902+0000 7f97f2140640 1 -- 172.21.15.135:0/1045784220 wait complete. 2024-04-26T22:05:57.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.904+0000 7f97f2140640 1 Processor -- start 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.904+0000 7f97f2140640 1 -- start start 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97f2140640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0aa8c0 0x7f97ec06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97f2140640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97ec0abf50 0x7f97ec15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97f2140640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97f2140640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec0c42c0 con 0x7f97ec0c3650 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97f2140640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec0c4140 con 0x7f97ec0abf50 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97f2140640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f97ec15a6a0 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36354/0 (socket says 172.21.15.135:36354) 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.905+0000 7f97ebfff640 1 -- 172.21.15.135:0/2026211575 learned_addr learned my addr 172.21.15.135:0/2026211575 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:57.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.907+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0aa8c0 0x7f97ec06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.907+0000 7f97eaffd640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97ec0abf50 0x7f97ec15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.908+0000 7f97eb7fe640 1 -- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97ec0abf50 msgr2=0x7f97ec15c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.908+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97ec0abf50 0x7f97ec15c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.908+0000 7f97eb7fe640 1 -- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 msgr2=0x7f97ec15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:57.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.908+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:57.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.908+0000 7f97eb7fe640 1 -- 172.21.15.135:0/2026211575 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec15f2b0 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.912+0000 7f97eb7fe640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0aa8c0 0x7f97ec06fa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97ec076720 tx=0x7f97dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.912+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc042020 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.912+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc03d690 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.912+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc037040 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.913+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec147fc0 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.914+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec147160 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.915+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ec0507a0 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.918+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97dc045020 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.920+0000 7f97d3fff640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f97d806c8f0 0x7f97d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:57.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.920+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f97dc0acbf0 con 0x7f97ec0aa8c0 2024-04-26T22:05:57.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.921+0000 7f97eaffd640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f97d806c8f0 0x7f97d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:57.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.927+0000 7f97eaffd640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f97d806c8f0 0x7f97d806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f97ec177740 tx=0x7f97e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:57.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:57.936+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97dc04c030 con 0x7f97ec0aa8c0 2024-04-26T22:05:58.295 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:58.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.487+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97ec0c3b60 con 0x7f97ec0aa8c0 2024-04-26T22:05:58.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.489+0000 7f97d3fff640 1 -- 172.21.15.135:0/2026211575 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5674 (secure 0 0 0) 0x7f97dc07ab10 con 0x7f97ec0aa8c0 2024-04-26T22:05:58.491 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:58.491 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:05:58.493 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 47 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f97d806c8f0 msgr2=0x7f97d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f97d806c8f0 0x7f97d806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f97ec177740 tx=0x7f97e0009040 comp rx=0 tx=0).stop 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0aa8c0 msgr2=0x7f97ec06fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0aa8c0 0x7f97ec06fa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97ec076720 tx=0x7f97dc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 shutdown_connections 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f97d806c8f0 0x7f97d806edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec0c3650 0x7f97ec15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97ec0abf50 0x7f97ec15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 --2- 172.21.15.135:0/2026211575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97ec0aa8c0 0x7f97ec06fa20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.499+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 >> 172.21.15.135:0/2026211575 conn(0x7f97ec0c82d0 msgr2=0x7f97ec165c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:58.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.500+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 shutdown_connections 2024-04-26T22:05:58.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.500+0000 7f97f2140640 1 -- 172.21.15.135:0/2026211575 wait complete. 2024-04-26T22:05:58.526 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=4973 2024-04-26T22:05:58.526 INFO:tasks.mds_thrash.fs.[b]:waiting for 1 secs before reviving mds.b 2024-04-26T22:05:58.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.562+0000 7f901aa8f640 1 Processor -- start 2024-04-26T22:05:58.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.563+0000 7f901aa8f640 1 -- start start 2024-04-26T22:05:58.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.564+0000 7f901aa8f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c0474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:58.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.564+0000 7f901aa8f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f900c00a8b0 0x7f900c00ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:58.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.564+0000 7f901aa8f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c00bb50 con 0x7f900c0df180 2024-04-26T22:05:58.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.564+0000 7f901aa8f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f900c00a420 con 0x7f900c00a8b0 2024-04-26T22:05:58.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.564+0000 7f901aa8f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f900c048060 con 0x7f900c0dbce0 2024-04-26T22:05:58.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.565+0000 7f901a28e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f900c00a8b0 0x7f900c00ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:58.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.565+0000 7f901a28e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f900c00a8b0 0x7f900c00ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38658/0 (socket says 172.21.15.135:38658) 2024-04-26T22:05:58.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.565+0000 7f901a28e640 1 -- 172.21.15.135:0/2304995327 learned_addr learned my addr 172.21.15.135:0/2304995327 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:58.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.565+0000 7f9019a8d640 1 --2- 172.21.15.135:0/2304995327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c0474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:58.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.566+0000 7f901a28e640 1 -- 172.21.15.135:0/2304995327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 msgr2=0x7f900c0474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:58.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.566+0000 7f901a28e640 1 --2- 172.21.15.135:0/2304995327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c0474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.566+0000 7f901a28e640 1 -- 172.21.15.135:0/2304995327 >> v1:172.21.15.135:6789/0 conn(0x7f900c0df180 legacy=0x7f900c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:58.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.566+0000 7f901a28e640 1 -- 172.21.15.135:0/2304995327 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c00a0f0 con 0x7f900c00a8b0 2024-04-26T22:05:58.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.567+0000 7f901a28e640 1 --2- 172.21.15.135:0/2304995327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c00ad00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9004009870 tx=0x7f900402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797c99c560b64873 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:58.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.568+0000 7f9018a8b640 1 -- 172.21.15.135:0/2304995327 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004037070 con 0x7f900c00a8b0 2024-04-26T22:05:58.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.568+0000 7f9018a8b640 1 -- 172.21.15.135:0/2304995327 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9004032440 con 0x7f900c00a8b0 2024-04-26T22:05:58.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.568+0000 7f9018a8b640 1 -- 172.21.15.135:0/2304995327 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004032810 con 0x7f900c00a8b0 2024-04-26T22:05:58.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.572+0000 7f901aa8f640 1 -- 172.21.15.135:0/2304995327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 msgr2=0x7f900c00ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:58.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.572+0000 7f901aa8f640 1 --2- 172.21.15.135:0/2304995327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c00ad00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9004009870 tx=0x7f900402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.572+0000 7f901aa8f640 1 -- 172.21.15.135:0/2304995327 shutdown_connections 2024-04-26T22:05:58.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.572+0000 7f901aa8f640 1 --2- 172.21.15.135:0/2304995327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c00ad00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.572+0000 7f901aa8f640 1 --2- 172.21.15.135:0/2304995327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c0474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.572+0000 7f901aa8f640 1 -- 172.21.15.135:0/2304995327 >> 172.21.15.135:0/2304995327 conn(0x7f900c048250 msgr2=0x7f900c048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:58.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.575+0000 7f901aa8f640 1 -- 172.21.15.135:0/2304995327 shutdown_connections 2024-04-26T22:05:58.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.575+0000 7f901aa8f640 1 -- 172.21.15.135:0/2304995327 wait complete. 2024-04-26T22:05:58.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.577+0000 7f901aa8f640 1 Processor -- start 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.577+0000 7f901aa8f640 1 -- start start 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.578+0000 7f901aa8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c0dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.578+0000 7f901aa8f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c040550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.578+0000 7f901aa8f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f900c0df180 0x7f900c191630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.578+0000 7f901aa8f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f900c00b240 con 0x7f900c00a8b0 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.578+0000 7f901aa8f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f900c047d80 con 0x7f900c0df180 2024-04-26T22:05:58.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.578+0000 7f901aa8f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f900c0dc0c0 con 0x7f900c0dbce0 2024-04-26T22:05:58.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.579+0000 7f901928c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c040550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:58.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.579+0000 7f9019a8d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c0dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:58.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.579+0000 7f901928c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c040550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46828/0 (socket says 172.21.15.135:46828) 2024-04-26T22:05:58.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.579+0000 7f9019a8d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c0dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38674/0 (socket says 172.21.15.135:38674) 2024-04-26T22:05:58.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.579+0000 7f9019a8d640 1 -- 172.21.15.135:0/3015544715 learned_addr learned my addr 172.21.15.135:0/3015544715 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:58.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.580+0000 7f9019a8d640 1 -- 172.21.15.135:0/3015544715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 msgr2=0x7f900c040550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:58.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.580+0000 7f9019a8d640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c040550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.580+0000 7f9019a8d640 1 -- 172.21.15.135:0/3015544715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f900c0df180 msgr2=0x7f900c191630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:05:58.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.581+0000 7f9019a8d640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f900c0df180 0x7f900c191630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:58.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.581+0000 7f9019a8d640 1 -- 172.21.15.135:0/3015544715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c0db520 con 0x7f900c00a8b0 2024-04-26T22:05:58.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.581+0000 7f9019a8d640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c0dafe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9014000b60 tx=0x7f901406a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:58.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.581+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014072980 con 0x7f900c00a8b0 2024-04-26T22:05:58.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.582+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9014072b20 con 0x7f900c00a8b0 2024-04-26T22:05:58.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.582+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901406f400 con 0x7f900c00a8b0 2024-04-26T22:05:58.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.583+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c030380 con 0x7f900c00a8b0 2024-04-26T22:05:58.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.583+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c035460 con 0x7f900c00a8b0 2024-04-26T22:05:58.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.585+0000 7f8ff7fff640 1 -- 172.21.15.135:0/3015544715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c035250 con 0x7f900c00a8b0 2024-04-26T22:05:58.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.596+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9014096480 con 0x7f900c00a8b0 2024-04-26T22:05:58.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.597+0000 7f900a7fc640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8ff806c9c0 0x7f8ff806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:58.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.598+0000 7f901928c640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8ff806c9c0 0x7f8ff806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:58.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.598+0000 7f901928c640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8ff806c9c0 0x7f8ff806ee80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f900c030c30 tx=0x7f9010009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:58.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.599+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f90141092b0 con 0x7f900c00a8b0 2024-04-26T22:05:58.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:58.599+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f900c0db520 con 0x7f900c00a8b0 2024-04-26T22:05:59.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.094+0000 7f8ff7fff640 1 -- 172.21.15.135:0/3015544715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f900c005390 con 0x7f900c00a8b0 2024-04-26T22:05:59.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.096+0000 7f900a7fc640 1 -- 172.21.15.135:0/3015544715 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5674 (secure 0 0 0) 0x7f90140d6a50 con 0x7f900c00a8b0 2024-04-26T22:05:59.097 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:05:59.097 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:05:59.097 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 47 2024-04-26T22:05:59.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.101+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8ff806c9c0 msgr2=0x7f8ff806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.101+0000 7f901aa8f640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8ff806c9c0 0x7f8ff806ee80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f900c030c30 tx=0x7f9010009040 comp rx=0 tx=0).stop 2024-04-26T22:05:59.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.101+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 msgr2=0x7f900c0dafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.101+0000 7f901aa8f640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c0dafe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9014000b60 tx=0x7f901406a720 comp rx=0 tx=0).stop 2024-04-26T22:05:59.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.103+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 shutdown_connections 2024-04-26T22:05:59.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.103+0000 7f901aa8f640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8ff806c9c0 0x7f8ff806ee80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.103+0000 7f901aa8f640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f900c0df180 0x7f900c191630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.103+0000 7f901aa8f640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900c0dbce0 0x7f900c040550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.103+0000 7f901aa8f640 1 --2- 172.21.15.135:0/3015544715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900c00a8b0 0x7f900c0dafe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.103+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 >> 172.21.15.135:0/3015544715 conn(0x7f900c048250 msgr2=0x7f900c0e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:59.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.104+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 shutdown_connections 2024-04-26T22:05:59.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.104+0000 7f901aa8f640 1 -- 172.21.15.135:0/3015544715 wait complete. 2024-04-26T22:05:59.126 INFO:tasks.mds_thrash.fs.[a]:kill mds.c (rank=0) 2024-04-26T22:05:59.127 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:05:59.127 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:05:59.191 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:05:59.191 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:59.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.414+0000 7f996d578640 1 Processor -- start 2024-04-26T22:05:59.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.416+0000 7f996d578640 1 -- start start 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f996d578640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99600dbec0 0x7f9960007890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f996d578640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99600e2030 0x7f99600e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f996d578640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99600deb30 con 0x7f9960046ce0 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f996d578640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f99600dc2a0 con 0x7f99600e2030 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f996d578640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99600dee10 con 0x7f99600dbec0 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f9967fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99600dbec0 0x7f9960007890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f9967fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99600dbec0 0x7f9960007890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:44506/0 (socket says 172.21.15.135:44506) 2024-04-26T22:05:59.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.417+0000 7f9967fff640 1 -- 172.21.15.135:0/343471953 learned_addr learned my addr 172.21.15.135:0/343471953 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:59.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.418+0000 7f9967fff640 1 -- 172.21.15.135:0/343471953 >> v2:172.21.15.135:3300/0 conn(0x7f99600e2030 msgr2=0x7f99600e24e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:59.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.418+0000 7f9967fff640 1 --2- 172.21.15.135:0/343471953 >> v2:172.21.15.135:3300/0 conn(0x7f99600e2030 0x7f99600e24e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.418+0000 7f9967fff640 1 -- 172.21.15.135:0/343471953 >> v1:172.21.15.135:6789/0 conn(0x7f9960046ce0 legacy=0x7f996003f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.418+0000 7f9967fff640 1 -- 172.21.15.135:0/343471953 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99600df0f0 con 0x7f99600dbec0 2024-04-26T22:05:59.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.421+0000 7f9967fff640 1 --2- 172.21.15.135:0/343471953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99600dbec0 0x7f9960007890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9960048a90 tx=0x7f995c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374ce1fdd3df0972 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:59.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.422+0000 7f9966ffd640 1 -- 172.21.15.135:0/343471953 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c037070 con 0x7f99600dbec0 2024-04-26T22:05:59.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.422+0000 7f9966ffd640 1 -- 172.21.15.135:0/343471953 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c032440 con 0x7f99600dbec0 2024-04-26T22:05:59.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.422+0000 7f9966ffd640 1 -- 172.21.15.135:0/343471953 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c032850 con 0x7f99600dbec0 2024-04-26T22:05:59.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.423+0000 7f996d578640 1 -- 172.21.15.135:0/343471953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99600dbec0 msgr2=0x7f9960007890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.423+0000 7f996d578640 1 --2- 172.21.15.135:0/343471953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99600dbec0 0x7f9960007890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9960048a90 tx=0x7f995c02a230 comp rx=0 tx=0).stop 2024-04-26T22:05:59.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.433+0000 7f996d578640 1 -- 172.21.15.135:0/343471953 shutdown_connections 2024-04-26T22:05:59.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.433+0000 7f996d578640 1 --2- 172.21.15.135:0/343471953 >> v2:172.21.15.135:3300/0 conn(0x7f99600e2030 0x7f99600e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.433+0000 7f996d578640 1 --2- 172.21.15.135:0/343471953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99600dbec0 0x7f9960007890 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.433+0000 7f996d578640 1 -- 172.21.15.135:0/343471953 >> 172.21.15.135:0/343471953 conn(0x7f9960048ac0 msgr2=0x7f9960048ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:59.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.436+0000 7f996d578640 1 -- 172.21.15.135:0/343471953 shutdown_connections 2024-04-26T22:05:59.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.441+0000 7f996d578640 1 -- 172.21.15.135:0/343471953 wait complete. 2024-04-26T22:05:59.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.443+0000 7f996d578640 1 Processor -- start 2024-04-26T22:05:59.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.446+0000 7f996d578640 1 -- start start 2024-04-26T22:05:59.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.447+0000 7f996d578640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9960046ce0 0x7f99600dda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.447+0000 7f996d578640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 0x7f99600437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.447+0000 7f996d578640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99600e2030 0x7f99600da7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.447+0000 7f996d578640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99600de7e0 con 0x7f99600dbec0 2024-04-26T22:05:59.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.447+0000 7f996d578640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99600df6b0 con 0x7f9960046ce0 2024-04-26T22:05:59.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.447+0000 7f996d578640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f996003f720 con 0x7f99600e2030 2024-04-26T22:05:59.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.448+0000 7f99677fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 0x7f99600437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.448+0000 7f99677fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 0x7f99600437c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38688/0 (socket says 172.21.15.135:38688) 2024-04-26T22:05:59.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.448+0000 7f99677fe640 1 -- 172.21.15.135:0/1956517398 learned_addr learned my addr 172.21.15.135:0/1956517398 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:59.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f99677fe640 1 -- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99600e2030 msgr2=0x7f99600da7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f996cd77640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99600e2030 0x7f99600da7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f9967fff640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9960046ce0 0x7f99600dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f99677fe640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99600e2030 0x7f99600da7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f99677fe640 1 -- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9960046ce0 msgr2=0x7f99600dda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f99677fe640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9960046ce0 0x7f99600dda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f99677fe640 1 -- 172.21.15.135:0/1956517398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99600d7ed0 con 0x7f99600dbec0 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f996cd77640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99600e2030 0x7f99600da7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.451+0000 7f9967fff640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9960046ce0 0x7f99600dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.452+0000 7f99677fe640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 0x7f99600437c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f995400c3f0 tx=0x7f995400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:59.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.454+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954017070 con 0x7f99600dbec0 2024-04-26T22:05:59.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.455+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99600d81b0 con 0x7f99600dbec0 2024-04-26T22:05:59.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.455+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99600d8770 con 0x7f99600dbec0 2024-04-26T22:05:59.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.455+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540052f0 con 0x7f99600dbec0 2024-04-26T22:05:59.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.455+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954003830 con 0x7f99600dbec0 2024-04-26T22:05:59.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.459+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f995401a070 con 0x7f99600dbec0 2024-04-26T22:05:59.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.459+0000 7f9964ff9640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f994c06c940 0x7f994c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.460+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f995408b4e0 con 0x7f99600dbec0 2024-04-26T22:05:59.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.463+0000 7f9967fff640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f994c06c940 0x7f994c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.465+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9948005290 con 0x7f99600dbec0 2024-04-26T22:05:59.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.465+0000 7f9967fff640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f994c06c940 0x7f994c06ee00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f995c034000 tx=0x7f995c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:59.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.475+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9954059350 con 0x7f99600dbec0 2024-04-26T22:05:59.527 INFO:tasks.mds_thrash.fs.[b]:reviving mds.b 2024-04-26T22:05:59.527 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T22:05:59.527 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T22:05:59.530 INFO:tasks.ceph.mds.b:Started 2024-04-26T22:05:59.531 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:59.662 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-04-26T22:05:59.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.766+0000 7f544fcfc640 1 Processor -- start 2024-04-26T22:05:59.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.766+0000 7f544fcfc640 1 -- start start 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544fcfc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f54480c7f70 0x7f54480ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544fcfc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54480c7040 0x7f5448151940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544fcfc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54480ab960 con 0x7f544806fff0 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544fcfc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f54480ad8a0 con 0x7f54480c7f70 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544fcfc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f54480abc40 con 0x7f54480c7040 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544da98640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f544806fff0 0x7f54481551c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:33342/0 (socket says 172.21.15.135:33342) 2024-04-26T22:05:59.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.768+0000 7f544da98640 1 -- 172.21.15.135:0/1656470792 learned_addr learned my addr 172.21.15.135:0/1656470792 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1110178590 0 0) 0x7f54480ab960 con 0x7f544806fff0 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5434003520 con 0x7f544806fff0 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544e299640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54480c7040 0x7f5448151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 622555076 0 0) 0x7f5434003520 con 0x7f544806fff0 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54480c7040 msgr2=0x7f5448151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54480c7040 0x7f5448151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 >> v2:172.21.15.139:3300/0 conn(0x7f54480c7f70 msgr2=0x7f54480ad360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 --2- 172.21.15.135:0/1656470792 >> v2:172.21.15.139:3300/0 conn(0x7f54480c7f70 0x7f54480ad360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54480c6840 con 0x7f544806fff0 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f5440001a20 con 0x7f544806fff0 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 >> v1:172.21.15.135:6789/0 conn(0x7f544806fff0 legacy=0x7f54481551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5434004800 0x7f5434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340052e0 0x7f543400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54480abc40 con 0x7f5434004800 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5434010580 con 0x7f54340039b0 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.769+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5434010780 con 0x7f54340052e0 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544e299640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54480c7040 0x7f5448151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544da98640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544e299640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340052e0 0x7f543400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544da98640 1 -- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340052e0 msgr2=0x7f543400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544da98640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340052e0 0x7f543400fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544da98640 1 -- 172.21.15.135:0/1656470792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5434004800 msgr2=0x7f5434004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544da98640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5434004800 0x7f5434004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.770+0000 7f544da98640 1 -- 172.21.15.135:0/1656470792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5434003520 con 0x7f54340039b0 2024-04-26T22:05:59.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.776+0000 7f544da98640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f544000a7c0 tx=0x7f5434005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876bd98ed5832524 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:59.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.778+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544000a410 con 0x7f54340039b0 2024-04-26T22:05:59.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.778+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544002f070 con 0x7f54340039b0 2024-04-26T22:05:59.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.778+0000 7f544ca96640 1 -- 172.21.15.135:0/1656470792 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544002bcb0 con 0x7f54340039b0 2024-04-26T22:05:59.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.779+0000 7f544fcfc640 1 -- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340039b0 msgr2=0x7f5434003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.779+0000 7f544fcfc640 1 --2- 172.21.15.135:0/1656470792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f544000a7c0 tx=0x7f5434005830 comp rx=0 tx=0).stop 2024-04-26T22:05:59.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.779+0000 7f544e299640 1 -- 172.21.15.135:0/1656470792 reap_dead start 2024-04-26T22:05:59.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.780+0000 7f544fcfc640 1 -- 172.21.15.135:0/1656470792 shutdown_connections 2024-04-26T22:05:59.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.780+0000 7f544fcfc640 1 -- 172.21.15.135:0/1656470792 >> 172.21.15.135:0/1656470792 conn(0x7f54480c5230 msgr2=0x7f54480c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:59.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.780+0000 7f544fcfc640 1 -- 172.21.15.135:0/1656470792 shutdown_connections 2024-04-26T22:05:59.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.781+0000 7f544fcfc640 1 -- 172.21.15.135:0/1656470792 wait complete. 2024-04-26T22:05:59.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.782+0000 7f544fcfc640 1 Processor -- start 2024-04-26T22:05:59.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.783+0000 7f544fcfc640 1 -- start start 2024-04-26T22:05:59.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544fcfc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340039b0 0x7f54481e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544fcfc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54480c47e0 0x7f5448155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544fcfc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 0x7f54480c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544fcfc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5448070550 con 0x7f54480c3950 2024-04-26T22:05:59.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544fcfc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f54480703d0 con 0x7f54480c47e0 2024-04-26T22:05:59.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544e299640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 0x7f54480c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544e299640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 0x7f54480c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38696/0 (socket says 172.21.15.135:38696) 2024-04-26T22:05:59.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.784+0000 7f544e299640 1 -- 172.21.15.135:0/2639270087 learned_addr learned my addr 172.21.15.135:0/2639270087 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:05:59.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.785+0000 7f544fcfc640 1 -- 172.21.15.135:0/2639270087 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f54480c74c0 con 0x7f54340039b0 2024-04-26T22:05:59.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.785+0000 7f544da98640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340039b0 0x7f54481e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.785+0000 7f544e299640 1 -- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340039b0 msgr2=0x7f54481e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.785+0000 7f544e299640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340039b0 0x7f54481e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.785+0000 7f544e299640 1 -- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54480c47e0 msgr2=0x7f5448155980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.785+0000 7f544e299640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54480c47e0 0x7f5448155980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.786+0000 7f544e299640 1 -- 172.21.15.135:0/2639270087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54481e8900 con 0x7f54480c3950 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.786+0000 7f544e299640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 0x7f54480c3dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f544400afd0 tx=0x7f544400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.787+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444013850 con 0x7f54480c3950 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.787+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5444013a10 con 0x7f54480c3950 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.787+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444013e00 con 0x7f54480c3950 2024-04-26T22:05:59.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.789+0000 7f544fcfc640 1 -- 172.21.15.135:0/2639270087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54480c4d80 con 0x7f54480c3950 2024-04-26T22:05:59.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.794+0000 7f544fcfc640 1 -- 172.21.15.135:0/2639270087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54480730c0 con 0x7f54480c3950 2024-04-26T22:05:59.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.798+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54440243f0 con 0x7f54480c3950 2024-04-26T22:05:59.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.798+0000 7f543e7fc640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f542c06c9c0 0x7f542c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:59.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.798+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f544408c790 con 0x7f54480c3950 2024-04-26T22:05:59.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.799+0000 7f544da98640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f542c06c9c0 0x7f542c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:59.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.800+0000 7f544da98640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f542c06c9c0 0x7f542c06ee80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5440033000 tx=0x7f5440007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:59.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.801+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54480506c0 con 0x7f54480c3950 2024-04-26T22:05:59.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:05:59.812+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54481e8900 con 0x7f54480c3950 2024-04-26T22:06:00.004 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:06:00.008 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:06:00.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.022+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9948005850 con 0x7f99600dbec0 2024-04-26T22:06:00.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.023+0000 7f9964ff9640 1 -- 172.21.15.135:0/1956517398 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6502 (secure 0 0 0) 0x7f9954058c50 con 0x7f99600dbec0 2024-04-26T22:06:00.023 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:00.023 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:00.024 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 48 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f994c06c940 msgr2=0x7f994c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f994c06c940 0x7f994c06ee00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f995c034000 tx=0x7f995c02e000 comp rx=0 tx=0).stop 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 msgr2=0x7f99600437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 0x7f99600437c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f995400c3f0 tx=0x7f995400c8b0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 shutdown_connections 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f994c06c940 0x7f994c06ee00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99600e2030 0x7f99600da7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99600dbec0 0x7f99600437c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.027+0000 7f996d578640 1 --2- 172.21.15.135:0/1956517398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9960046ce0 0x7f99600dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.028+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 >> 172.21.15.135:0/1956517398 conn(0x7f9960048ac0 msgr2=0x7f996000ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:00.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.028+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 shutdown_connections 2024-04-26T22:06:00.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.028+0000 7f996d578640 1 -- 172.21.15.135:0/1956517398 wait complete. 2024-04-26T22:06:00.052 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:06:00.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.305+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54480c56b0 con 0x7f54480c3950 2024-04-26T22:06:00.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.306+0000 7f543e7fc640 1 -- 172.21.15.135:0/2639270087 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6502 (secure 0 0 0) 0x7f5444017030 con 0x7f54480c3950 2024-04-26T22:06:00.308 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:00.308 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:00.308 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 48 2024-04-26T22:06:00.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.311+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f542c06c9c0 msgr2=0x7f542c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:00.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.311+0000 7f542bfff640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f542c06c9c0 0x7f542c06ee80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5440033000 tx=0x7f5440007000 comp rx=0 tx=0).stop 2024-04-26T22:06:00.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.311+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 msgr2=0x7f54480c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.311+0000 7f542bfff640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 0x7f54480c3dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f544400afd0 tx=0x7f544400fa40 comp rx=0 tx=0).stop 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.312+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 shutdown_connections 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.312+0000 7f542bfff640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f542c06c9c0 0x7f542c06ee80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.312+0000 7f542bfff640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54480c3950 0x7f54480c3dd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.312+0000 7f542bfff640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54480c47e0 0x7f5448155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.312+0000 7f542bfff640 1 --2- 172.21.15.135:0/2639270087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54340039b0 0x7f54481e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:00.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.312+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 >> 172.21.15.135:0/2639270087 conn(0x7f54480c5230 msgr2=0x7f5448150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:00.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.314+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 shutdown_connections 2024-04-26T22:06:00.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:00.315+0000 7f542bfff640 1 -- 172.21.15.135:0/2639270087 wait complete. 2024-04-26T22:06:00.335 INFO:tasks.mds_thrash.fs.[b]:mds.b reported in up:standby state 2024-04-26T22:06:00.335 INFO:tasks.mds_thrash.fs.[b]:waiting for 52 secs before thrashing 2024-04-26T22:06:02.053 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:02.178 INFO:tasks.ceph.mds.a.smithi135.stderr:2024-04-26T22:06:02.178+0000 7fa342230640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:06:02.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.252+0000 7fe732578640 1 Processor -- start 2024-04-26T22:06:02.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.253+0000 7fe732578640 1 -- start start 2024-04-26T22:06:02.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.254+0000 7fe732578640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe72c158c20 0x7fe72c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:02.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.254+0000 7fe732578640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72c0cbde0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.254+0000 7fe732578640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72c0cc0c0 con 0x7fe72c0c90c0 2024-04-26T22:06:02.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.254+0000 7fe732578640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe72c15c150 con 0x7fe72c158c20 2024-04-26T22:06:02.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.254+0000 7fe72bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe72c0c90c0 0x7fe72c0cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:34222/0 (socket says 172.21.15.135:34222) 2024-04-26T22:06:02.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.254+0000 7fe72bfff640 1 -- 172.21.15.135:0/3834694038 learned_addr learned my addr 172.21.15.135:0/3834694038 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:02.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.255+0000 7fe730b15640 1 --2- 172.21.15.135:0/3834694038 >> v2:172.21.15.135:3300/0 conn(0x7fe72c158c20 0x7fe72c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:02.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.256+0000 7fe730b15640 1 -- 172.21.15.135:0/3834694038 >> v1:172.21.15.139:6789/0 conn(0x7fe72c0c90c0 legacy=0x7fe72c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:02.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.256+0000 7fe730b15640 1 -- 172.21.15.135:0/3834694038 >> v1:172.21.15.135:6789/0 conn(0x7fe72c0bfbb0 legacy=0x7fe72c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:02.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.256+0000 7fe730b15640 1 -- 172.21.15.135:0/3834694038 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0ca5d0 con 0x7fe72c158c20 2024-04-26T22:06:02.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.257+0000 7fe730b15640 1 --2- 172.21.15.135:0/3834694038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c158c20 0x7fe72c15bc10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe724009d90 tx=0x7fe72402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353c0ea01ed8d427 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:02.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.257+0000 7fe72affd640 1 -- 172.21.15.135:0/3834694038 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724031600 con 0x7fe72c158c20 2024-04-26T22:06:02.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.257+0000 7fe72affd640 1 -- 172.21.15.135:0/3834694038 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7240317a0 con 0x7fe72c158c20 2024-04-26T22:06:02.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.257+0000 7fe72affd640 1 -- 172.21.15.135:0/3834694038 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724031b70 con 0x7fe72c158c20 2024-04-26T22:06:02.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.259+0000 7fe732578640 1 -- 172.21.15.135:0/3834694038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c158c20 msgr2=0x7fe72c15bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:02.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.259+0000 7fe732578640 1 --2- 172.21.15.135:0/3834694038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c158c20 0x7fe72c15bc10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe724009d90 tx=0x7fe72402a5d0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.261+0000 7fe732578640 1 -- 172.21.15.135:0/3834694038 shutdown_connections 2024-04-26T22:06:02.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.261+0000 7fe732578640 1 --2- 172.21.15.135:0/3834694038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c158c20 0x7fe72c15bc10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.261+0000 7fe732578640 1 -- 172.21.15.135:0/3834694038 >> 172.21.15.135:0/3834694038 conn(0x7fe72c0ae010 msgr2=0x7fe72c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:02.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.266+0000 7fe732578640 1 -- 172.21.15.135:0/3834694038 shutdown_connections 2024-04-26T22:06:02.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.266+0000 7fe732578640 1 -- 172.21.15.135:0/3834694038 wait complete. 2024-04-26T22:06:02.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.268+0000 7fe732578640 1 Processor -- start 2024-04-26T22:06:02.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.269+0000 7fe732578640 1 -- start start 2024-04-26T22:06:02.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.269+0000 7fe732578640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 0x7fe72c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:02.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.269+0000 7fe732578640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72c0c90c0 0x7fe72c15e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:02.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe732578640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72c158c20 0x7fe72c0c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe732578640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0c94a0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe732578640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0ca380 con 0x7fe72c158c20 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe732578640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c0c9620 con 0x7fe72c0c90c0 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 0x7fe72c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 0x7fe72c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38714/0 (socket says 172.21.15.135:38714) 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.270+0000 7fe72bfff640 1 -- 172.21.15.135:0/3845508529 learned_addr learned my addr 172.21.15.135:0/3845508529 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.271+0000 7fe72bfff640 1 -- 172.21.15.135:0/3845508529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72c0c90c0 msgr2=0x7fe72c15e930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.271+0000 7fe72bfff640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72c0c90c0 0x7fe72c15e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.271+0000 7fe72bfff640 1 -- 172.21.15.135:0/3845508529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72c158c20 msgr2=0x7fe72c0c4aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.271+0000 7fe72bfff640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72c158c20 0x7fe72c0c4aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.271+0000 7fe72bfff640 1 -- 172.21.15.135:0/3845508529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c15f3b0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.271+0000 7fe72bfff640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 0x7fe72c15b280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c400 tx=0x7fe72000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.273+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720017070 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.274+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c15b8d0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.274+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720005ce0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.274+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c0c42a0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.275+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7200072f0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.277+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe72001a070 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.277+0000 7fe728ff9640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe71806c9c0 0x7fe71806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:02.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.278+0000 7fe72b7fe640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe71806c9c0 0x7fe71806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:02.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.279+0000 7fe72b7fe640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe71806c9c0 0x7fe71806ee80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe72c157b90 tx=0x7fe71c007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:02.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.279+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7fe72008b4b0 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.284+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c005290 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.293+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe720059330 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.769+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe70c005580 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.771+0000 7fe728ff9640 1 -- 172.21.15.135:0/3845508529 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6502 (secure 0 0 0) 0x7fe720058c30 con 0x7fe72c0bfbb0 2024-04-26T22:06:02.773 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:02.773 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:02.774 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 48 2024-04-26T22:06:02.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.778+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe71806c9c0 msgr2=0x7fe71806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.778+0000 7fe732578640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe71806c9c0 0x7fe71806ee80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe72c157b90 tx=0x7fe71c007990 comp rx=0 tx=0).stop 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.778+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 msgr2=0x7fe72c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.778+0000 7fe732578640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 0x7fe72c15b280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe72000c400 tx=0x7fe72000c8c0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 shutdown_connections 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe71806c9c0 0x7fe71806ee80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72c158c20 0x7fe72c0c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72c0c90c0 0x7fe72c15e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 --2- 172.21.15.135:0/3845508529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe72c0bfbb0 0x7fe72c15b280 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 >> 172.21.15.135:0/3845508529 conn(0x7fe72c0ae010 msgr2=0x7fe72c157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 shutdown_connections 2024-04-26T22:06:02.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:02.779+0000 7fe732578640 1 -- 172.21.15.135:0/3845508529 wait complete. 2024-04-26T22:06:02.802 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:06:04.803 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:04.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.990+0000 7f99dfa1d640 1 Processor -- start 2024-04-26T22:06:04.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.991+0000 7f99dfa1d640 1 -- start start 2024-04-26T22:06:04.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.992+0000 7f99dfa1d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99d80ae3d0 0x7f99d80c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:04.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.992+0000 7f99dfa1d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99d80ae7b0 0x7f99d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:04.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99dfa1d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d8070750 con 0x7f99d80a80a0 2024-04-26T22:06:04.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99dfa1d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f99d81565b0 con 0x7f99d80ae7b0 2024-04-26T22:06:04.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99dfa1d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99d8156800 con 0x7f99d80ae3d0 2024-04-26T22:06:04.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99dcfb8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f99d80a80a0 0x7f99d80c5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:34228/0 (socket says 172.21.15.135:34228) 2024-04-26T22:06:04.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99dcfb8640 1 -- 172.21.15.135:0/3550535400 learned_addr learned my addr 172.21.15.135:0/3550535400 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:04.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99ddfba640 1 --2- 172.21.15.135:0/3550535400 >> v2:172.21.15.135:3300/0 conn(0x7f99d80ae7b0 0x7f99d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.993+0000 7f99ddfba640 1 -- 172.21.15.135:0/3550535400 >> v2:172.21.15.139:3300/0 conn(0x7f99d80ae3d0 msgr2=0x7f99d80c8250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.994+0000 7f99ddfba640 1 --2- 172.21.15.135:0/3550535400 >> v2:172.21.15.139:3300/0 conn(0x7f99d80ae3d0 0x7f99d80c8250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.994+0000 7f99ddfba640 1 -- 172.21.15.135:0/3550535400 >> v1:172.21.15.139:6789/0 conn(0x7f99d80a80a0 legacy=0x7f99d80c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.994+0000 7f99ddfba640 1 -- 172.21.15.135:0/3550535400 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d8156ae0 con 0x7f99d80ae7b0 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.995+0000 7f99ddfba640 1 --2- 172.21.15.135:0/3550535400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae7b0 0x7f99d806ede0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f99d80c60b0 tx=0x7f99c802c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d459b91d4bc76f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.997+0000 7f99cffff640 1 -- 172.21.15.135:0/3550535400 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c803b070 con 0x7f99d80ae7b0 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.997+0000 7f99cffff640 1 -- 172.21.15.135:0/3550535400 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c80369b0 con 0x7f99d80ae7b0 2024-04-26T22:06:04.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.997+0000 7f99cffff640 1 -- 172.21.15.135:0/3550535400 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8036dc0 con 0x7f99d80ae7b0 2024-04-26T22:06:04.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.998+0000 7f99dfa1d640 1 -- 172.21.15.135:0/3550535400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae7b0 msgr2=0x7f99d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:04.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.998+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/3550535400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae7b0 0x7f99d806ede0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f99d80c60b0 tx=0x7f99c802c3f0 comp rx=0 tx=0).stop 2024-04-26T22:06:04.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.999+0000 7f99dfa1d640 1 -- 172.21.15.135:0/3550535400 shutdown_connections 2024-04-26T22:06:04.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.999+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/3550535400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae7b0 0x7f99d806ede0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:04.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.999+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/3550535400 >> v2:172.21.15.139:3300/0 conn(0x7f99d80ae3d0 0x7f99d80c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:04.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:04.999+0000 7f99dfa1d640 1 -- 172.21.15.135:0/3550535400 >> 172.21.15.135:0/3550535400 conn(0x7f99d80c89f0 msgr2=0x7f99d80c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:04.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.000+0000 7f99dfa1d640 1 -- 172.21.15.135:0/3550535400 shutdown_connections 2024-04-26T22:06:05.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.001+0000 7f99dfa1d640 1 -- 172.21.15.135:0/3550535400 wait complete. 2024-04-26T22:06:05.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.004+0000 7f99dfa1d640 1 Processor -- start 2024-04-26T22:06:05.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.004+0000 7f99dfa1d640 1 -- start start 2024-04-26T22:06:05.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.005+0000 7f99dfa1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d80a80a0 0x7f99d814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.006+0000 7f99dfa1d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae3d0 0x7f99d8155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.006+0000 7f99dd7b9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d80a80a0 0x7f99d814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.006+0000 7f99dd7b9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d80a80a0 0x7f99d814c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46858/0 (socket says 172.21.15.135:46858) 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.006+0000 7f99dd7b9640 1 -- 172.21.15.135:0/1176372956 learned_addr learned my addr 172.21.15.135:0/1176372956 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.007+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d80ae7b0 0x7f99d81e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.007+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99d80c5b90 con 0x7f99d80ae3d0 2024-04-26T22:06:05.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.007+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81570a0 con 0x7f99d80ae7b0 2024-04-26T22:06:05.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.007+0000 7f99dcfb8640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae3d0 0x7f99d8155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.007+0000 7f99ddfba640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d80ae7b0 0x7f99d81e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.008+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99d80c5d10 con 0x7f99d80a80a0 2024-04-26T22:06:05.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.009+0000 7f99dcfb8640 1 -- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d80a80a0 msgr2=0x7f99d814c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.009+0000 7f99dcfb8640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d80a80a0 0x7f99d814c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.009+0000 7f99dcfb8640 1 -- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d80ae7b0 msgr2=0x7f99d81e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.009+0000 7f99dcfb8640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d80ae7b0 0x7f99d81e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.009+0000 7f99dcfb8640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d814a560 con 0x7f99d80ae3d0 2024-04-26T22:06:05.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.010+0000 7f99dcfb8640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae3d0 0x7f99d8155910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000f60 tx=0x7f99d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.011+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4017070 con 0x7f99d80ae3d0 2024-04-26T22:06:05.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.011+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4005ce0 con 0x7f99d80ae3d0 2024-04-26T22:06:05.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.011+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4007310 con 0x7f99d80ae3d0 2024-04-26T22:06:05.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.012+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d814a840 con 0x7f99d80ae3d0 2024-04-26T22:06:05.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.012+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d814ae00 con 0x7f99d80ae3d0 2024-04-26T22:06:05.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.014+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b8005290 con 0x7f99d80ae3d0 2024-04-26T22:06:05.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.015+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99d401a070 con 0x7f99d80ae3d0 2024-04-26T22:06:05.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.016+0000 7f99cdffb640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f99bc06c940 0x7f99bc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.017+0000 7f99dd7b9640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f99bc06c940 0x7f99bc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.018+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f99d40569d0 con 0x7f99d80ae3d0 2024-04-26T22:06:05.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.019+0000 7f99dd7b9640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f99bc06c940 0x7f99bc06ee00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009fd0 tx=0x7f99d000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.026+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f99d4058930 con 0x7f99d80ae3d0 2024-04-26T22:06:05.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.516+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b8005580 con 0x7f99d80ae3d0 2024-04-26T22:06:05.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.518+0000 7f99cdffb640 1 -- 172.21.15.135:0/1176372956 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6502 (secure 0 0 0) 0x7f99d405cc30 con 0x7f99d80ae3d0 2024-04-26T22:06:05.518 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:05.519 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:05.519 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 48 2024-04-26T22:06:05.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f99bc06c940 msgr2=0x7f99bc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f99bc06c940 0x7f99bc06ee00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009fd0 tx=0x7f99d000b040 comp rx=0 tx=0).stop 2024-04-26T22:06:05.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae3d0 msgr2=0x7f99d8155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae3d0 0x7f99d8155910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000f60 tx=0x7f99d400c800 comp rx=0 tx=0).stop 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 shutdown_connections 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f99bc06c940 0x7f99bc06ee00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d80ae7b0 0x7f99d81e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99d80ae3d0 0x7f99d8155910 secure :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000f60 tx=0x7f99d400c800 comp rx=0 tx=0).stop 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 --2- 172.21.15.135:0/1176372956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d80a80a0 0x7f99d814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.527+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 >> 172.21.15.135:0/1176372956 conn(0x7f99d80c89f0 msgr2=0x7f99d80c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.529+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 shutdown_connections 2024-04-26T22:06:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:05.529+0000 7f99dfa1d640 1 -- 172.21.15.135:0/1176372956 wait complete. 2024-04-26T22:06:05.550 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:06:07.550 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:07.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.746+0000 7fe40161d640 1 Processor -- start 2024-04-26T22:06:07.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.746+0000 7fe40161d640 1 -- start start 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe40161d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe3fc156e50 0x7fe3fc0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe40161d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 0x7fe3fc14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe40161d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe3fc0694b0 0x7fe3fc152b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe40161d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe3fc156120 con 0x7fe3fc156e50 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe40161d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe3fc1569c0 con 0x7fe3fc0694b0 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe40161d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc0c4400 con 0x7fe3fc0c3f20 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.747+0000 7fe3fa7fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 0x7fe3fc14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 0x7fe3fc14fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46872/0 (socket says 172.21.15.135:46872) 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 -- 172.21.15.135:0/1338196208 learned_addr learned my addr 172.21.15.135:0/1338196208 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:07.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3faffd640 1 --2- 172.21.15.135:0/1338196208 >> v2:172.21.15.135:3300/0 conn(0x7fe3fc156e50 0x7fe3fc0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 -- 172.21.15.135:0/1338196208 >> v2:172.21.15.139:3300/0 conn(0x7fe3fc0694b0 msgr2=0x7fe3fc152b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 --2- 172.21.15.135:0/1338196208 >> v2:172.21.15.139:3300/0 conn(0x7fe3fc0694b0 0x7fe3fc152b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 -- 172.21.15.135:0/1338196208 >> v2:172.21.15.135:3300/0 conn(0x7fe3fc156e50 msgr2=0x7fe3fc0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 --2- 172.21.15.135:0/1338196208 >> v2:172.21.15.135:3300/0 conn(0x7fe3fc156e50 0x7fe3fc0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.748+0000 7fe3fa7fc640 1 -- 172.21.15.135:0/1338196208 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc156370 con 0x7fe3fc0c3f20 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.749+0000 7fe3faffd640 1 --2- 172.21.15.135:0/1338196208 >> v2:172.21.15.135:3300/0 conn(0x7fe3fc156e50 0x7fe3fc0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.750+0000 7fe3fa7fc640 1 --2- 172.21.15.135:0/1338196208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 0x7fe3fc14fd30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc156930 tx=0x7fe3f403b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61353091d13c2a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.751+0000 7fe3f9ffb640 1 -- 172.21.15.135:0/1338196208 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4047070 con 0x7fe3fc0c3f20 2024-04-26T22:06:07.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.751+0000 7fe3f9ffb640 1 -- 172.21.15.135:0/1338196208 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f4042600 con 0x7fe3fc0c3f20 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 -- 172.21.15.135:0/1338196208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 msgr2=0x7fe3fc14fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 --2- 172.21.15.135:0/1338196208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 0x7fe3fc14fd30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc156930 tx=0x7fe3f403b770 comp rx=0 tx=0).stop 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 -- 172.21.15.135:0/1338196208 shutdown_connections 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 --2- 172.21.15.135:0/1338196208 >> v2:172.21.15.139:3300/0 conn(0x7fe3fc0694b0 0x7fe3fc152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 --2- 172.21.15.135:0/1338196208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc0c3f20 0x7fe3fc14fd30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 --2- 172.21.15.135:0/1338196208 >> v2:172.21.15.135:3300/0 conn(0x7fe3fc156e50 0x7fe3fc0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.753+0000 7fe40161d640 1 -- 172.21.15.135:0/1338196208 >> 172.21.15.135:0/1338196208 conn(0x7fe3fc0c89b0 msgr2=0x7fe3fc0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.754+0000 7fe40161d640 1 -- 172.21.15.135:0/1338196208 shutdown_connections 2024-04-26T22:06:07.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.754+0000 7fe40161d640 1 -- 172.21.15.135:0/1338196208 wait complete. 2024-04-26T22:06:07.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.756+0000 7fe40161d640 1 Processor -- start 2024-04-26T22:06:07.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.757+0000 7fe40161d640 1 -- start start 2024-04-26T22:06:07.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe40161d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3fc0694b0 0x7fe3fc146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe40161d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3fc0c3f20 0x7fe3fc1496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe40161d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc156e50 0x7fe3fc152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe40161d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0c3aa0 con 0x7fe3fc0694b0 2024-04-26T22:06:07.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe40161d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0ae1e0 con 0x7fe3fc0c3f20 2024-04-26T22:06:07.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe40161d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc0c5bd0 con 0x7fe3fc156e50 2024-04-26T22:06:07.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe3fb7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc156e50 0x7fe3fc152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.758+0000 7fe3fb7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc156e50 0x7fe3fc152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:46880/0 (socket says 172.21.15.135:46880) 2024-04-26T22:06:07.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.759+0000 7fe3fb7fe640 1 -- 172.21.15.135:0/2899013760 learned_addr learned my addr 172.21.15.135:0/2899013760 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:07.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.759+0000 7fe3faffd640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3fc0694b0 0x7fe3fc146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.759+0000 7fe3fa7fc640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3fc0c3f20 0x7fe3fc1496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.760+0000 7fe3faffd640 1 -- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc156e50 msgr2=0x7fe3fc152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.760+0000 7fe3faffd640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc156e50 0x7fe3fc152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.760+0000 7fe3faffd640 1 -- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3fc0c3f20 msgr2=0x7fe3fc1496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.760+0000 7fe3faffd640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3fc0c3f20 0x7fe3fc1496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.760+0000 7fe3faffd640 1 -- 172.21.15.135:0/2899013760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc1521c0 con 0x7fe3fc0694b0 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.760+0000 7fe3faffd640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3fc0694b0 0x7fe3fc146510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c5e0 tx=0x7fe3ec00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.761+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec015070 con 0x7fe3fc0694b0 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.761+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec011800 con 0x7fe3fc0694b0 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.761+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011c10 con 0x7fe3fc0694b0 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.761+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc0ac280 con 0x7fe3fc0694b0 2024-04-26T22:06:07.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.761+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc147430 con 0x7fe3fc0694b0 2024-04-26T22:06:07.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.763+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc076100 con 0x7fe3fc0694b0 2024-04-26T22:06:07.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.774+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3ec018070 con 0x7fe3fc0694b0 2024-04-26T22:06:07.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.774+0000 7fe3e37fe640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe3e806c9f0 0x7fe3e806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.775+0000 7fe3fa7fc640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe3e806c9f0 0x7fe3e806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.778+0000 7fe3fa7fc640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe3e806c9f0 0x7fe3e806eeb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4044000 tx=0x7fe3f403f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.779+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7fe3ec08c4b0 con 0x7fe3fc0694b0 2024-04-26T22:06:07.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:07.779+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3ec08c8d0 con 0x7fe3fc0694b0 2024-04-26T22:06:08.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.278+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3fc0c3410 con 0x7fe3fc0694b0 2024-04-26T22:06:08.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.282+0000 7fe3e37fe640 1 -- 172.21.15.135:0/2899013760 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6502 (secure 0 0 0) 0x7fe3ec05a2a0 con 0x7fe3fc0694b0 2024-04-26T22:06:08.283 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:08.283 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:08.283 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 48 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.286+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe3e806c9f0 msgr2=0x7fe3e806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.286+0000 7fe40161d640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe3e806c9f0 0x7fe3e806eeb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4044000 tx=0x7fe3f403f000 comp rx=0 tx=0).stop 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.286+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3fc0694b0 msgr2=0x7fe3fc146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.286+0000 7fe40161d640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3fc0694b0 0x7fe3fc146510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c5e0 tx=0x7fe3ec00caa0 comp rx=0 tx=0).stop 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.287+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 shutdown_connections 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.287+0000 7fe40161d640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe3e806c9f0 0x7fe3e806eeb0 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4044000 tx=0x7fe3f403f000 comp rx=0 tx=0).stop 2024-04-26T22:06:08.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.287+0000 7fe40161d640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3fc156e50 0x7fe3fc152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:08.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.287+0000 7fe40161d640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3fc0c3f20 0x7fe3fc1496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:08.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.287+0000 7fe40161d640 1 --2- 172.21.15.135:0/2899013760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3fc0694b0 0x7fe3fc146510 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:08.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.287+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 >> 172.21.15.135:0/2899013760 conn(0x7fe3fc0c89b0 msgr2=0x7fe3fc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:08.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.291+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 shutdown_connections 2024-04-26T22:06:08.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:08.291+0000 7fe40161d640 1 -- 172.21.15.135:0/2899013760 wait complete. 2024-04-26T22:06:08.315 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:06:10.316 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:10.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.531+0000 7f444e166640 1 Processor -- start 2024-04-26T22:06:10.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.531+0000 7f444e166640 1 -- start start 2024-04-26T22:06:10.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f444e166640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44480c2110 0x7f44480aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f444e166640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f444806f7e0 con 0x7f444815a170 2024-04-26T22:06:10.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f444e166640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f444806ea20 con 0x7f4448150b30 2024-04-26T22:06:10.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f444e166640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f444806f350 con 0x7f44480c2110 2024-04-26T22:06:10.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f44477fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4448150b30 0x7f44480ac7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:55420/0 (socket says 172.21.15.135:55420) 2024-04-26T22:06:10.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f44477fe640 1 -- 172.21.15.135:0/4223765751 learned_addr learned my addr 172.21.15.135:0/4223765751 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:10.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.533+0000 7f4446ffd640 1 --2- 172.21.15.135:0/4223765751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44480c2110 0x7f44480aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.534+0000 7f4446ffd640 1 -- 172.21.15.135:0/4223765751 >> v1:172.21.15.139:6789/0 conn(0x7f4448150b30 legacy=0x7f44480ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:10.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.534+0000 7f4446ffd640 1 -- 172.21.15.135:0/4223765751 >> v1:172.21.15.135:6789/0 conn(0x7f444815a170 legacy=0x7f444815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.534+0000 7f4446ffd640 1 -- 172.21.15.135:0/4223765751 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444806ed00 con 0x7f44480c2110 2024-04-26T22:06:10.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.536+0000 7f4446ffd640 1 --2- 172.21.15.135:0/4223765751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44480c2110 0x7f44480aceb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44380097d0 tx=0x7f443802a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5c2d2211be24ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.536+0000 7f44467fc640 1 -- 172.21.15.135:0/4223765751 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438037070 con 0x7f44480c2110 2024-04-26T22:06:10.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.536+0000 7f44467fc640 1 -- 172.21.15.135:0/4223765751 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443802acc0 con 0x7f44480c2110 2024-04-26T22:06:10.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.536+0000 7f44467fc640 1 -- 172.21.15.135:0/4223765751 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438032660 con 0x7f44480c2110 2024-04-26T22:06:10.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.537+0000 7f444e166640 1 -- 172.21.15.135:0/4223765751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44480c2110 msgr2=0x7f44480aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.537+0000 7f444e166640 1 --2- 172.21.15.135:0/4223765751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44480c2110 0x7f44480aceb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44380097d0 tx=0x7f443802a030 comp rx=0 tx=0).stop 2024-04-26T22:06:10.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.538+0000 7f444e166640 1 -- 172.21.15.135:0/4223765751 shutdown_connections 2024-04-26T22:06:10.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.538+0000 7f444e166640 1 --2- 172.21.15.135:0/4223765751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44480c2110 0x7f44480aceb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.538+0000 7f444e166640 1 -- 172.21.15.135:0/4223765751 >> 172.21.15.135:0/4223765751 conn(0x7f44480c82d0 msgr2=0x7f44480c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:10.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.540+0000 7f444e166640 1 -- 172.21.15.135:0/4223765751 shutdown_connections 2024-04-26T22:06:10.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.540+0000 7f444e166640 1 -- 172.21.15.135:0/4223765751 wait complete. 2024-04-26T22:06:10.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.543+0000 7f444e166640 1 Processor -- start 2024-04-26T22:06:10.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.544+0000 7f444e166640 1 -- start start 2024-04-26T22:06:10.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.545+0000 7f444e166640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44480c2110 0x7f4448152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.545+0000 7f444e166640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f444815a170 0x7f44480c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f444e166640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 0x7f44481f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f444e166640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4448150f10 con 0x7f44481576c0 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f4447fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 0x7f44481f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f4447fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 0x7f44481f1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57054/0 (socket says 172.21.15.135:57054) 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f4447fff640 1 -- 172.21.15.135:0/152506426 learned_addr learned my addr 172.21.15.135:0/152506426 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f444e166640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f444806fec0 con 0x7f44480c2110 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f4446ffd640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f444815a170 0x7f44480c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.546+0000 7f444e166640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4448151090 con 0x7f444815a170 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.547+0000 7f4447fff640 1 -- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f444815a170 msgr2=0x7f44480c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:10.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.547+0000 7f4447fff640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f444815a170 0x7f44480c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.547+0000 7f4447fff640 1 -- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44480c2110 msgr2=0x7f4448152660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:10.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.547+0000 7f4447fff640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44480c2110 0x7f4448152660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:10.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.547+0000 7f4447fff640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4448152f80 con 0x7f44481576c0 2024-04-26T22:06:10.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.547+0000 7f4447fff640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 0x7f44481f1180 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f444000c8f0 tx=0x7f444000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.549+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f44481576c0 2024-04-26T22:06:10.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.549+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4440003da0 con 0x7f44481576c0 2024-04-26T22:06:10.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.549+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440005290 con 0x7f44481576c0 2024-04-26T22:06:10.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.550+0000 7f444e166640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44480aaae0 con 0x7f44481576c0 2024-04-26T22:06:10.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.550+0000 7f444e166640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4448151570 con 0x7f44481576c0 2024-04-26T22:06:10.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.553+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f444001a070 con 0x7f44481576c0 2024-04-26T22:06:10.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.554+0000 7f444e166640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4428005290 con 0x7f44481576c0 2024-04-26T22:06:10.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.559+0000 7f442ffff640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f443406c940 0x7f443406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:10.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.559+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 7268+0+0 (secure 0 0 0) 0x7f444008b570 con 0x7f44481576c0 2024-04-26T22:06:10.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.560+0000 7f44477fe640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f443406c940 0x7f443406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:10.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.563+0000 7f44477fe640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f443406c940 0x7f443406ee00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f443c007fd0 tx=0x7f443c007aa0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:10.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:10.566+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4440059400 con 0x7f44481576c0 2024-04-26T22:06:11.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.073+0000 7f444e166640 1 -- 172.21.15.135:0/152506426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4428005850 con 0x7f44481576c0 2024-04-26T22:06:11.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.074+0000 7f442ffff640 1 -- 172.21.15.135:0/152506426 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+6502 (secure 0 0 0) 0x7f4440058d00 con 0x7f44481576c0 2024-04-26T22:06:11.074 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:11.075 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:04:48.111880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6835/3214490783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3214490783},{"type":"v1","addr":"172.21.15.135:6835","nonce":3214490783}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5272,"qdb_cluster":[5272]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:11.075 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 48 2024-04-26T22:06:11.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 -- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f443406c940 msgr2=0x7f443406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:11.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f443406c940 0x7f443406ee00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f443c007fd0 tx=0x7f443c007aa0 comp rx=0 tx=0).stop 2024-04-26T22:06:11.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 -- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 msgr2=0x7f44481f1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 0x7f44481f1180 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f444000c8f0 tx=0x7f444000cdb0 comp rx=0 tx=0).stop 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 -- 172.21.15.135:0/152506426 shutdown_connections 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f443406c940 0x7f443406ee00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44481576c0 0x7f44481f1180 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f444815a170 0x7f44480c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 --2- 172.21.15.135:0/152506426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44480c2110 0x7f4448152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.081+0000 7f442dffb640 1 -- 172.21.15.135:0/152506426 >> 172.21.15.135:0/152506426 conn(0x7f44480c82d0 msgr2=0x7f4448159730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.082+0000 7f442dffb640 1 -- 172.21.15.135:0/152506426 shutdown_connections 2024-04-26T22:06:11.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:11.083+0000 7f442dffb640 1 -- 172.21.15.135:0/152506426 wait complete. 2024-04-26T22:06:11.103 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:06:13.104 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.304+0000 7f5f3a483640 1 Processor -- start 2024-04-26T22:06:13.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.306+0000 7f5f3a483640 1 -- start start 2024-04-26T22:06:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.307+0000 7f5f3a483640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 0x7f5f34156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.307+0000 7f5f3a483640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f3415a1a0 0x7f5f340c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.307+0000 7f5f3a483640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f340c8df0 con 0x7f5f340ca5a0 2024-04-26T22:06:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.307+0000 7f5f3a483640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5f340c9100 con 0x7f5f3415a1a0 2024-04-26T22:06:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.307+0000 7f5f3a483640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f34154540 con 0x7f5f34154160 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.309+0000 7f5f33fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 0x7f5f34156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.309+0000 7f5f33fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 0x7f5f34156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:44156/0 (socket says 172.21.15.135:44156) 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.309+0000 7f5f33fff640 1 -- 172.21.15.135:0/443885248 learned_addr learned my addr 172.21.15.135:0/443885248 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.310+0000 7f5f33fff640 1 -- 172.21.15.135:0/443885248 >> v2:172.21.15.139:3300/0 conn(0x7f5f3415a1a0 msgr2=0x7f5f340c9590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.310+0000 7f5f33fff640 1 --2- 172.21.15.135:0/443885248 >> v2:172.21.15.139:3300/0 conn(0x7f5f3415a1a0 0x7f5f340c9590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.310+0000 7f5f33fff640 1 -- 172.21.15.135:0/443885248 >> v1:172.21.15.139:6789/0 conn(0x7f5f340ca5a0 legacy=0x7f5f340c6e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:13.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.310+0000 7f5f33fff640 1 -- 172.21.15.135:0/443885248 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3415a910 con 0x7f5f34154160 2024-04-26T22:06:13.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.315+0000 7f5f33fff640 1 --2- 172.21.15.135:0/443885248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 0x7f5f34156bb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f24032a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80eecb0576ec8e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.316+0000 7f5f32ffd640 1 -- 172.21.15.135:0/443885248 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2403b070 con 0x7f5f34154160 2024-04-26T22:06:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.317+0000 7f5f32ffd640 1 -- 172.21.15.135:0/443885248 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f24037860 con 0x7f5f34154160 2024-04-26T22:06:13.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.317+0000 7f5f32ffd640 1 -- 172.21.15.135:0/443885248 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24037c30 con 0x7f5f34154160 2024-04-26T22:06:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.320+0000 7f5f3a483640 1 -- 172.21.15.135:0/443885248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 msgr2=0x7f5f34156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.320+0000 7f5f3a483640 1 --2- 172.21.15.135:0/443885248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 0x7f5f34156bb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f24032a40 comp rx=0 tx=0).stop 2024-04-26T22:06:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.321+0000 7f5f3a483640 1 -- 172.21.15.135:0/443885248 shutdown_connections 2024-04-26T22:06:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.321+0000 7f5f3a483640 1 --2- 172.21.15.135:0/443885248 >> v2:172.21.15.139:3300/0 conn(0x7f5f3415a1a0 0x7f5f340c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.321+0000 7f5f3a483640 1 --2- 172.21.15.135:0/443885248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f34154160 0x7f5f34156bb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.321+0000 7f5f3a483640 1 -- 172.21.15.135:0/443885248 >> 172.21.15.135:0/443885248 conn(0x7f5f340ae010 msgr2=0x7f5f340ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:13.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.324+0000 7f5f3a483640 1 -- 172.21.15.135:0/443885248 shutdown_connections 2024-04-26T22:06:13.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.329+0000 7f5f3a483640 1 -- 172.21.15.135:0/443885248 wait complete. 2024-04-26T22:06:13.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.330+0000 7f5f3a483640 1 Processor -- start 2024-04-26T22:06:13.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.334+0000 7f5f3a483640 1 -- start start 2024-04-26T22:06:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f3a483640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f340ca5a0 0x7f5f34157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f3a483640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34154160 0x7f5f34145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:13.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f3a483640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f3415a1a0 0x7f5f341f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f3a483640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415a630 con 0x7f5f3415a1a0 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f3a483640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415aed0 con 0x7f5f34154160 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f3a483640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f340c7510 con 0x7f5f340ca5a0 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f33fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f340ca5a0 0x7f5f34157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f33fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f340ca5a0 0x7f5f34157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:44166/0 (socket says 172.21.15.135:44166) 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.336+0000 7f5f33fff640 1 -- 172.21.15.135:0/165478448 learned_addr learned my addr 172.21.15.135:0/165478448 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:13.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.337+0000 7f5f337fe640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34154160 0x7f5f34145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:13.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.341+0000 7f5f38a20640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f3415a1a0 0x7f5f341f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.342+0000 7f5f337fe640 1 -- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f340ca5a0 msgr2=0x7f5f34157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.342+0000 7f5f337fe640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f340ca5a0 0x7f5f34157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.342+0000 7f5f337fe640 1 -- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f3415a1a0 msgr2=0x7f5f341f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.342+0000 7f5f337fe640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f3415a1a0 0x7f5f341f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.342+0000 7f5f337fe640 1 -- 172.21.15.135:0/165478448 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341f9c00 con 0x7f5f34154160 2024-04-26T22:06:13.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.342+0000 7f5f38a20640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f3415a1a0 0x7f5f341f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:13.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.345+0000 7f5f337fe640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34154160 0x7f5f34145340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fd0 tx=0x7f5f2800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:13.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.349+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28015020 con 0x7f5f34154160 2024-04-26T22:06:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.349+0000 7f5f3a483640 1 -- 172.21.15.135:0/165478448 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341f9ee0 con 0x7f5f34154160 2024-04-26T22:06:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.349+0000 7f5f3a483640 1 -- 172.21.15.135:0/165478448 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341fa4a0 con 0x7f5f34154160 2024-04-26T22:06:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.349+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f28011850 con 0x7f5f34154160 2024-04-26T22:06:13.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.349+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28011c40 con 0x7f5f34154160 2024-04-26T22:06:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.352+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f28018020 con 0x7f5f34154160 2024-04-26T22:06:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.353+0000 7f5f30ff9640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f2006c9c0 0x7f5f2006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.353+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f5f2808bf80 con 0x7f5f34154160 2024-04-26T22:06:13.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.354+0000 7f5f3a483640 1 -- 172.21.15.135:0/165478448 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f14005290 con 0x7f5f34154160 2024-04-26T22:06:13.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.354+0000 7f5f33fff640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f2006c9c0 0x7f5f2006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:13.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.355+0000 7f5f33fff640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f2006c9c0 0x7f5f2006ee80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f24032a80 tx=0x7f5f24008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:13.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.364+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f2801f020 con 0x7f5f34154160 2024-04-26T22:06:13.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.872+0000 7f5f3a483640 1 -- 172.21.15.135:0/165478448 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f14005580 con 0x7f5f34154160 2024-04-26T22:06:13.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.874+0000 7f5f30ff9640 1 -- 172.21.15.135:0/165478448 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f5f280596d0 con 0x7f5f34154160 2024-04-26T22:06:13.875 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:13.875 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:13.876 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 -- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f2006c9c0 msgr2=0x7f5f2006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f2006c9c0 0x7f5f2006ee80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f24032a80 tx=0x7f5f24008030 comp rx=0 tx=0).stop 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 -- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34154160 msgr2=0x7f5f34145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34154160 0x7f5f34145340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fd0 tx=0x7f5f2800b870 comp rx=0 tx=0).stop 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 -- 172.21.15.135:0/165478448 shutdown_connections 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f2006c9c0 0x7f5f2006ee80 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f24032a80 tx=0x7f5f24008030 comp rx=0 tx=0).stop 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f3415a1a0 0x7f5f341f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f34154160 0x7f5f34145340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 --2- 172.21.15.135:0/165478448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f340ca5a0 0x7f5f34157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:13.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.881+0000 7f5f1a7fc640 1 -- 172.21.15.135:0/165478448 >> 172.21.15.135:0/165478448 conn(0x7f5f340ae010 msgr2=0x7f5f3414e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:13.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.882+0000 7f5f1a7fc640 1 -- 172.21.15.135:0/165478448 shutdown_connections 2024-04-26T22:06:13.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:13.883+0000 7f5f1a7fc640 1 -- 172.21.15.135:0/165478448 wait complete. 2024-04-26T22:06:13.907 INFO:tasks.mds_thrash.fs.[a]:mds.c down, removed from mdsmap 2024-04-26T22:06:13.907 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-04-26T22:06:13.908 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:14.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.149+0000 7f0f29338640 1 Processor -- start 2024-04-26T22:06:14.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.149+0000 7f0f29338640 1 -- start start 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.150+0000 7f0f29338640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0f240c3650 0x7f0f2406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.150+0000 7f0f29338640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f240aa8c0 0x7f0f240c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.150+0000 7f0f29338640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2415c0d0 con 0x7f0f240c3f70 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.150+0000 7f0f29338640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0f240ab4b0 con 0x7f0f240c3650 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.150+0000 7f0f29338640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2415bdf0 con 0x7f0f240aa8c0 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.150+0000 7f0f227fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f240aa8c0 0x7f0f240c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f227fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f240aa8c0 0x7f0f240c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:46130/0 (socket says 172.21.15.135:46130) 2024-04-26T22:06:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f227fc640 1 -- 172.21.15.135:0/1960179830 learned_addr learned my addr 172.21.15.135:0/1960179830 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f22ffd640 1 --2- 172.21.15.135:0/1960179830 >> v2:172.21.15.135:3300/0 conn(0x7f0f240c3650 0x7f0f2406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f227fc640 1 -- 172.21.15.135:0/1960179830 >> v2:172.21.15.135:3300/0 conn(0x7f0f240c3650 msgr2=0x7f0f2406e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f227fc640 1 --2- 172.21.15.135:0/1960179830 >> v2:172.21.15.135:3300/0 conn(0x7f0f240c3650 0x7f0f2406e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f227fc640 1 -- 172.21.15.135:0/1960179830 >> v1:172.21.15.135:6789/0 conn(0x7f0f240c3f70 legacy=0x7f0f240abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f21ffb640 1 -- 172.21.15.135:0/1960179830 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830958313 0 0) 0x7f0f2415c0d0 con 0x7f0f240c3f70 2024-04-26T22:06:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f227fc640 1 -- 172.21.15.135:0/1960179830 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f240ab750 con 0x7f0f240aa8c0 2024-04-26T22:06:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.151+0000 7f0f22ffd640 1 --2- 172.21.15.135:0/1960179830 >> v2:172.21.15.135:3300/0 conn(0x7f0f240c3650 0x7f0f2406e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.152+0000 7f0f227fc640 1 --2- 172.21.15.135:0/1960179830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f240c3a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f24071320 tx=0x7f0f1802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a0d3ccc2e28ada server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.153+0000 7f0f21ffb640 1 -- 172.21.15.135:0/1960179830 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18037070 con 0x7f0f240aa8c0 2024-04-26T22:06:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.155+0000 7f0f21ffb640 1 -- 172.21.15.135:0/1960179830 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f18032440 con 0x7f0f240aa8c0 2024-04-26T22:06:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.155+0000 7f0f21ffb640 1 -- 172.21.15.135:0/1960179830 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18032810 con 0x7f0f240aa8c0 2024-04-26T22:06:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.156+0000 7f0f29338640 1 -- 172.21.15.135:0/1960179830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 msgr2=0x7f0f240c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.156+0000 7f0f29338640 1 --2- 172.21.15.135:0/1960179830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f240c3a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f24071320 tx=0x7f0f1802a180 comp rx=0 tx=0).stop 2024-04-26T22:06:14.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.156+0000 7f0f29338640 1 -- 172.21.15.135:0/1960179830 shutdown_connections 2024-04-26T22:06:14.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.156+0000 7f0f29338640 1 --2- 172.21.15.135:0/1960179830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f240c3a30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.156+0000 7f0f29338640 1 --2- 172.21.15.135:0/1960179830 >> v2:172.21.15.135:3300/0 conn(0x7f0f240c3650 0x7f0f2406e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.156+0000 7f0f29338640 1 -- 172.21.15.135:0/1960179830 >> 172.21.15.135:0/1960179830 conn(0x7f0f240c82d0 msgr2=0x7f0f240c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:14.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.161+0000 7f0f29338640 1 -- 172.21.15.135:0/1960179830 shutdown_connections 2024-04-26T22:06:14.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.161+0000 7f0f29338640 1 -- 172.21.15.135:0/1960179830 wait complete. 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.163+0000 7f0f29338640 1 Processor -- start 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.163+0000 7f0f29338640 1 -- start start 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.164+0000 7f0f29338640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f241434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.164+0000 7f0f29338640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 0x7f0f241fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.164+0000 7f0f29338640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 0x7f0f2414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.164+0000 7f0f237fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 0x7f0f2414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.164+0000 7f0f237fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 0x7f0f2414bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57084/0 (socket says 172.21.15.135:57084) 2024-04-26T22:06:14.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f22ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f241434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f227fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 0x7f0f241fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f22ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f241434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:46138/0 (socket says 172.21.15.135:46138) 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f227fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 0x7f0f241fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:44172/0 (socket says 172.21.15.135:44172) 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f29338640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f240ac7e0 con 0x7f0f240c3f70 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f29338640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f240ac660 con 0x7f0f240aa8c0 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f29338640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2415a6a0 con 0x7f0f240c3650 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f237fe640 1 -- 172.21.15.135:0/1542147107 learned_addr learned my addr 172.21.15.135:0/1542147107 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f237fe640 1 -- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 msgr2=0x7f0f241fd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f237fe640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 0x7f0f241fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f237fe640 1 -- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 msgr2=0x7f0f241434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f237fe640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f241434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f237fe640 1 -- 172.21.15.135:0/1542147107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f240c7540 con 0x7f0f240c3f70 2024-04-26T22:06:14.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.165+0000 7f0f22ffd640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f241434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T22:06:14.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.166+0000 7f0f237fe640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 0x7f0f2414bd80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c780 tx=0x7f0f1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:14.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.166+0000 7f0f227fc640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 0x7f0f241fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T22:06:14.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.167+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14017070 con 0x7f0f240c3f70 2024-04-26T22:06:14.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.167+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f241429d0 con 0x7f0f240c3f70 2024-04-26T22:06:14.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.167+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2414c2c0 con 0x7f0f240c3f70 2024-04-26T22:06:14.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.167+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f14005070 con 0x7f0f240c3f70 2024-04-26T22:06:14.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.167+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14005440 con 0x7f0f240c3f70 2024-04-26T22:06:14.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.170+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f04005290 con 0x7f0f240c3f70 2024-04-26T22:06:14.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.170+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f1401a070 con 0x7f0f240c3f70 2024-04-26T22:06:14.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.170+0000 7f0f0b7fe640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0f1006c9c0 0x7f0f1006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:14.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.171+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f0f1408b3b0 con 0x7f0f240c3f70 2024-04-26T22:06:14.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.171+0000 7f0f22ffd640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0f1006c9c0 0x7f0f1006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:14.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.176+0000 7f0f22ffd640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0f1006c9c0 0x7f0f1006ee80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00d430 tx=0x7f0f1c007530 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:14.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.180+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f140591e0 con 0x7f0f240c3f70 2024-04-26T22:06:14.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.653+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f04005580 con 0x7f0f240c3f70 2024-04-26T22:06:14.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.656+0000 7f0f0b7fe640 1 -- 172.21.15.135:0/1542147107 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f0f14058ae0 con 0x7f0f240c3f70 2024-04-26T22:06:14.659 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:14.660 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:14.660 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.661+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0f1006c9c0 msgr2=0x7f0f1006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.661+0000 7f0f29338640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0f1006c9c0 0x7f0f1006ee80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00d430 tx=0x7f0f1c007530 comp rx=0 tx=0).stop 2024-04-26T22:06:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.661+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 msgr2=0x7f0f2414bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:14.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.661+0000 7f0f29338640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 0x7f0f2414bd80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c780 tx=0x7f0f1400cc40 comp rx=0 tx=0).stop 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 shutdown_connections 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f0f1006c9c0 0x7f0f1006ee80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f240c3f70 0x7f0f2414bd80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f240c3650 0x7f0f241fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 --2- 172.21.15.135:0/1542147107 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240aa8c0 0x7f0f241434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 >> 172.21.15.135:0/1542147107 conn(0x7f0f240c82d0 msgr2=0x7f0f240c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 shutdown_connections 2024-04-26T22:06:14.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:14.662+0000 7f0f29338640 1 -- 172.21.15.135:0/1542147107 wait complete. 2024-04-26T22:06:14.687 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 49, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 49, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5105": { "addr": "172.21.15.139:6835/3733666635", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 3733666635, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 3733666635, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5105, "incarnation": 49, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:06:12.822242+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5105 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 47, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5610": { "addr": "172.21.15.135:6837/2165063360", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 2165063360, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 2165063360, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5610, "incarnation": 44, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:05:57.871105+0000", "qdb_cluster": [ 5610 ], "qdb_leader": 5610, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5610 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:06:16.688 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:16.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.862+0000 7f242359e640 1 Processor -- start 2024-04-26T22:06:16.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.866+0000 7f242359e640 1 -- start start 2024-04-26T22:06:16.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f242359e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f241c0dbce0 0x7f241c0474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f242359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0df180 0x7f241c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f242359e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241c00bb50 con 0x7f241c0073e0 2024-04-26T22:06:16.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f242359e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f241c00a420 con 0x7f241c0dbce0 2024-04-26T22:06:16.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f242359e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f241c048060 con 0x7f241c0df180 2024-04-26T22:06:16.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f2422d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f241c0073e0 0x7f241c00a8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37680/0 (socket says 172.21.15.135:37680) 2024-04-26T22:06:16.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.867+0000 7f2422d9d640 1 -- 172.21.15.135:0/4099327010 learned_addr learned my addr 172.21.15.135:0/4099327010 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:16.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.870+0000 7f242259c640 1 --2- 172.21.15.135:0/4099327010 >> v2:172.21.15.139:3300/0 conn(0x7f241c0dbce0 0x7f241c0474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.870+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794076520 0 0) 0x7f241c00bb50 con 0x7f241c0073e0 2024-04-26T22:06:16.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.870+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f240c003520 con 0x7f241c0073e0 2024-04-26T22:06:16.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.871+0000 7f2421d9b640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0df180 0x7f241c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2633893650 0 0) 0x7f240c003520 con 0x7f241c0073e0 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0df180 msgr2=0x7f241c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0df180 0x7f241c006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 >> v2:172.21.15.139:3300/0 conn(0x7f241c0dbce0 msgr2=0x7f241c0474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 --2- 172.21.15.135:0/4099327010 >> v2:172.21.15.139:3300/0 conn(0x7f241c0dbce0 0x7f241c0474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c00a0f0 con 0x7f241c0073e0 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2418001670 con 0x7f241c0073e0 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 >> v1:172.21.15.135:6789/0 conn(0x7f241c0073e0 legacy=0x7f241c00a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:16.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f240c0039b0 0x7f240c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f240c004800 0x7f240c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f240c0051c0 0x7f240c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f241c00a420 con 0x7f240c004800 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f241c048060 con 0x7f240c0039b0 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.874+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f240c00c0e0 con 0x7f240c0051c0 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.875+0000 7f2422d9d640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f240c0051c0 0x7f240c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.875+0000 7f242259c640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f240c0039b0 0x7f240c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.875+0000 7f2421d9b640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f240c004800 0x7f240c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.876+0000 7f2421d9b640 1 -- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f240c0051c0 msgr2=0x7f240c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:16.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.876+0000 7f2421d9b640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f240c0051c0 0x7f240c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:16.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.876+0000 7f2421d9b640 1 -- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f240c0039b0 msgr2=0x7f240c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:16.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.876+0000 7f2421d9b640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f240c0039b0 0x7f240c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:16.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.876+0000 7f2421d9b640 1 -- 172.21.15.135:0/4099327010 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c003520 con 0x7f240c004800 2024-04-26T22:06:16.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.877+0000 7f2422d9d640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f240c0051c0 0x7f240c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:16.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.877+0000 7f242259c640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f240c0039b0 0x7f240c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:16.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.877+0000 7f2421d9b640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f240c004800 0x7f240c004c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f241c046be0 tx=0x7f240c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23facab83c3b6964 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:16.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.878+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414030070 con 0x7f240c004800 2024-04-26T22:06:16.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.878+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241402acc0 con 0x7f240c004800 2024-04-26T22:06:16.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.878+0000 7f242159a640 1 -- 172.21.15.135:0/4099327010 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400b530 con 0x7f240c004800 2024-04-26T22:06:16.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.879+0000 7f242359e640 1 -- 172.21.15.135:0/4099327010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f240c004800 msgr2=0x7f240c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:16.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.879+0000 7f242359e640 1 --2- 172.21.15.135:0/4099327010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f240c004800 0x7f240c004c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f241c046be0 tx=0x7f240c005830 comp rx=0 tx=0).stop 2024-04-26T22:06:16.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.881+0000 7f2422d9d640 1 -- 172.21.15.135:0/4099327010 reap_dead start 2024-04-26T22:06:16.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.881+0000 7f242359e640 1 -- 172.21.15.135:0/4099327010 shutdown_connections 2024-04-26T22:06:16.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.881+0000 7f242359e640 1 -- 172.21.15.135:0/4099327010 >> 172.21.15.135:0/4099327010 conn(0x7f241c048250 msgr2=0x7f241c048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:16.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.882+0000 7f242359e640 1 -- 172.21.15.135:0/4099327010 shutdown_connections 2024-04-26T22:06:16.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.882+0000 7f242359e640 1 -- 172.21.15.135:0/4099327010 wait complete. 2024-04-26T22:06:16.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.884+0000 7f242359e640 1 Processor -- start 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.884+0000 7f242359e640 1 -- start start 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.885+0000 7f242359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 0x7f241c0cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.885+0000 7f242359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 0x7f241c044010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.885+0000 7f242359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0493a0 0x7f241c02c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.885+0000 7f242359e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f241c0dc0c0 con 0x7f241c042910 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.885+0000 7f242359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f241c047d80 con 0x7f241c04a150 2024-04-26T22:06:16.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.885+0000 7f242359e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f241c0df5c0 con 0x7f241c0493a0 2024-04-26T22:06:16.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.886+0000 7f2421d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 0x7f241c044010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.886+0000 7f242259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 0x7f241c0cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.886+0000 7f242259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 0x7f241c0cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:46156/0 (socket says 172.21.15.135:46156) 2024-04-26T22:06:16.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.886+0000 7f2421d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 0x7f241c044010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57110/0 (socket says 172.21.15.135:57110) 2024-04-26T22:06:16.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.886+0000 7f242259c640 1 -- 172.21.15.135:0/180747354 learned_addr learned my addr 172.21.15.135:0/180747354 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.887+0000 7f2421d9b640 1 -- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0493a0 msgr2=0x7f241c02c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.887+0000 7f2421d9b640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0493a0 0x7f241c02c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.887+0000 7f2421d9b640 1 -- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 msgr2=0x7f241c0cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.887+0000 7f2421d9b640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 0x7f241c0cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.888+0000 7f2421d9b640 1 -- 172.21.15.135:0/180747354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c044550 con 0x7f241c042910 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.888+0000 7f242259c640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 0x7f241c0cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:16.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.888+0000 7f2421d9b640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 0x7f241c044010 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f240c0057b0 tx=0x7f2414004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:16.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.889+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414043020 con 0x7f241c042910 2024-04-26T22:06:16.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.889+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241403f470 con 0x7f241c042910 2024-04-26T22:06:16.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.889+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c043550 con 0x7f241c042910 2024-04-26T22:06:16.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.889+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c04b3c0 con 0x7f241c042910 2024-04-26T22:06:16.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.889+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241403f840 con 0x7f241c042910 2024-04-26T22:06:16.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.893+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2408005290 con 0x7f241c042910 2024-04-26T22:06:16.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.894+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2414046020 con 0x7f241c042910 2024-04-26T22:06:16.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.894+0000 7f2412ffd640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f240406c9f0 0x7f240406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:16.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.894+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f24140acba0 con 0x7f241c042910 2024-04-26T22:06:16.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.895+0000 7f242259c640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f240406c9f0 0x7f240406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:16.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.904+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f241404d030 con 0x7f241c042910 2024-04-26T22:06:16.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:16.906+0000 7f242259c640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f240406c9f0 0x7f240406eeb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2424076900 tx=0x7f2424090040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:17.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.354+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2408005580 con 0x7f241c042910 2024-04-26T22:06:17.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.355+0000 7f2412ffd640 1 -- 172.21.15.135:0/180747354 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f241c044550 con 0x7f241c042910 2024-04-26T22:06:17.358 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:17.358 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:17.359 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:17.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f240406c9f0 msgr2=0x7f240406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f240406c9f0 0x7f240406eeb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2424076900 tx=0x7f2424090040 comp rx=0 tx=0).stop 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 msgr2=0x7f241c044010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 0x7f241c044010 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f240c0057b0 tx=0x7f2414004730 comp rx=0 tx=0).stop 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 shutdown_connections 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f240406c9f0 0x7f240406eeb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f241c0493a0 0x7f241c02c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c042910 0x7f241c044010 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 --2- 172.21.15.135:0/180747354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241c04a150 0x7f241c0cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.362+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 >> 172.21.15.135:0/180747354 conn(0x7f241c048250 msgr2=0x7f241c0e0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:17.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.364+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 shutdown_connections 2024-04-26T22:06:17.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:17.364+0000 7f242359e640 1 -- 172.21.15.135:0/180747354 wait complete. 2024-04-26T22:06:17.385 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:06:19.385 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:19.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.590+0000 7f8d7cb91640 1 Processor -- start 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.591+0000 7f8d7cb91640 1 -- start start 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d7cb91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d78072600 0x7f8d7815f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d7cb91640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d780c4b50 con 0x7f8d780c4160 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d75d9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8d780c4160 0x7f8d7806d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:50702/0 (socket says 172.21.15.135:50702) 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d75d9b640 1 -- 172.21.15.135:0/3916266120 learned_addr learned my addr 172.21.15.135:0/3916266120 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d781514a0 con 0x7f8d781510c0 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8d780c50d0 con 0x7f8d78072600 2024-04-26T22:06:19.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.592+0000 7f8d76d9d640 1 --2- 172.21.15.135:0/3916266120 >> v2:172.21.15.139:3300/0 conn(0x7f8d78072600 0x7f8d7815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.593+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030545303 0 0) 0x7f8d780c4b50 con 0x7f8d780c4160 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.593+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d60003520 con 0x7f8d780c4160 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.593+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1367892144 0 0) 0x7f8d60003520 con 0x7f8d780c4160 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.593+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 >> v2:172.21.15.139:3300/0 conn(0x7f8d78072600 msgr2=0x7f8d7815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.593+0000 7f8d7559a640 1 --2- 172.21.15.135:0/3916266120 >> v2:172.21.15.139:3300/0 conn(0x7f8d78072600 0x7f8d7815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 >> v1:172.21.15.139:6789/0 conn(0x7f8d781510c0 legacy=0x7f8d78072f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d780736a0 con 0x7f8d780c4160 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f8d6c001690 con 0x7f8d780c4160 2024-04-26T22:06:19.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 >> v1:172.21.15.135:6789/0 conn(0x7f8d780c4160 legacy=0x7f8d7806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60003a00 0x7f8d60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d60004800 0x7f8d60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d600052e0 0x7f8d6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780c50d0 con 0x7f8d60003a00 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6000c1e0 con 0x7f8d600052e0 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6000c3e0 con 0x7f8d60004800 2024-04-26T22:06:19.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.594+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d60004800 0x7f8d60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:19.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.597+0000 7f8d75d9b640 1 -- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d600052e0 msgr2=0x7f8d6000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:19.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.597+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d600052e0 0x7f8d6000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:19.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.597+0000 7f8d75d9b640 1 -- 172.21.15.135:0/3916266120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60003a00 msgr2=0x7f8d60003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:19.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.597+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60003a00 0x7f8d60003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:19.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.597+0000 7f8d75d9b640 1 -- 172.21.15.135:0/3916266120 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d60003520 con 0x7f8d60004800 2024-04-26T22:06:19.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.597+0000 7f8d76d9d640 1 -- 172.21.15.135:0/3916266120 reap_dead start 2024-04-26T22:06:19.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.599+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d60004800 0x7f8d60004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001eb0 tx=0x7f8d60005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa57852e068a492 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:19.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.599+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007ca0 con 0x7f8d60004800 2024-04-26T22:06:19.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.599+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c005af0 con 0x7f8d60004800 2024-04-26T22:06:19.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.600+0000 7f8d7559a640 1 -- 172.21.15.135:0/3916266120 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007070 con 0x7f8d60004800 2024-04-26T22:06:19.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.601+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d60004800 msgr2=0x7f8d60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:19.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.601+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d60004800 0x7f8d60004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001eb0 tx=0x7f8d60005830 comp rx=0 tx=0).stop 2024-04-26T22:06:19.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.602+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 shutdown_connections 2024-04-26T22:06:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.602+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/3916266120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d60004800 0x7f8d60004c80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.602+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 >> 172.21.15.135:0/3916266120 conn(0x7f8d780c5ce0 msgr2=0x7f8d780c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.607+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 shutdown_connections 2024-04-26T22:06:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.607+0000 7f8d7cb91640 1 -- 172.21.15.135:0/3916266120 wait complete. 2024-04-26T22:06:19.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.609+0000 7f8d7cb91640 1 Processor -- start 2024-04-26T22:06:19.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.610+0000 7f8d7cb91640 1 -- start start 2024-04-26T22:06:19.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d7cb91640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60004800 0x7f8d780c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d7cb91640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 0x7f8d781de9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d7cb91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d78156e90 0x7f8d78157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d7cb91640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780c5400 con 0x7f8d60004800 2024-04-26T22:06:19.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d7cb91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780c4df0 con 0x7f8d78156e90 2024-04-26T22:06:19.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d75d9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 0x7f8d781de9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:19.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d75d9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 0x7f8d781de9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:37324/0 (socket says 172.21.15.135:37324) 2024-04-26T22:06:19.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d75d9b640 1 -- 172.21.15.135:0/771483679 learned_addr learned my addr 172.21.15.135:0/771483679 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:19.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.611+0000 7f8d76d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d78156e90 0x7f8d78157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:19.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8d780c5920 con 0x7f8d781de570 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d7659c640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60004800 0x7f8d780c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d75d9b640 1 -- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d78156e90 msgr2=0x7f8d78157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d78156e90 0x7f8d78157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d75d9b640 1 -- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60004800 msgr2=0x7f8d780c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60004800 0x7f8d780c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d75d9b640 1 -- 172.21.15.135:0/771483679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d781dd9e0 con 0x7f8d781de570 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.612+0000 7f8d75d9b640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 0x7f8d781de9d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d780c4560 tx=0x7f8d6c001990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.614+0000 7f8d76d9d640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d78156e90 0x7f8d78157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.614+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c001590 con 0x7f8d781de570 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.614+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d781ddcc0 con 0x7f8d781de570 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.615+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d78073a80 con 0x7f8d781de570 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.615+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c046030 con 0x7f8d781de570 2024-04-26T22:06:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.615+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c042cc0 con 0x7f8d781de570 2024-04-26T22:06:19.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.618+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c005290 con 0x7f8d781de570 2024-04-26T22:06:19.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.619+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d6c007070 con 0x7f8d781de570 2024-04-26T22:06:19.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.620+0000 7f8d66ffd640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8d5806c9c0 0x7f8d5806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:19.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.620+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f8d6c0b4660 con 0x7f8d781de570 2024-04-26T22:06:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.620+0000 7f8d7659c640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8d5806c9c0 0x7f8d5806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.621+0000 7f8d7659c640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8d5806c9c0 0x7f8d5806ee80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8d68006fd0 tx=0x7f8d68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:19.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:19.629+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d6c081db0 con 0x7f8d781de570 2024-04-26T22:06:20.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.093+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d5c005580 con 0x7f8d781de570 2024-04-26T22:06:20.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.098+0000 7f8d66ffd640 1 -- 172.21.15.135:0/771483679 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f8d6c0860b0 con 0x7f8d781de570 2024-04-26T22:06:20.098 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:20.098 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:20.099 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8d5806c9c0 msgr2=0x7f8d5806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8d5806c9c0 0x7f8d5806ee80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8d68006fd0 tx=0x7f8d68008040 comp rx=0 tx=0).stop 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 msgr2=0x7f8d781de9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 0x7f8d781de9d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d780c4560 tx=0x7f8d6c001990 comp rx=0 tx=0).stop 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 shutdown_connections 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f8d5806c9c0 0x7f8d5806ee80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d78156e90 0x7f8d78157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d781de570 0x7f8d781de9d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 --2- 172.21.15.135:0/771483679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d60004800 0x7f8d780c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:20.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 >> 172.21.15.135:0/771483679 conn(0x7f8d780c5ce0 msgr2=0x7f8d7806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:20.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 shutdown_connections 2024-04-26T22:06:20.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:20.102+0000 7f8d7cb91640 1 -- 172.21.15.135:0/771483679 wait complete. 2024-04-26T22:06:20.127 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:06:22.128 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:22.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.320+0000 7fc324a47640 1 Processor -- start 2024-04-26T22:06:22.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.320+0000 7fc324a47640 1 -- start start 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc324a47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3201540f0 0x7fc32015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc324a47640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc320156bc0 con 0x7fc320157680 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc324a47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3200c5350 con 0x7fc3200ac610 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc324a47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc3201571f0 con 0x7fc3201540f0 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc31e59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3201540f0 0x7fc32015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc31dd9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc320157680 0x7fc3200c4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:50716/0 (socket says 172.21.15.135:50716) 2024-04-26T22:06:22.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.322+0000 7fc31dd9b640 1 -- 172.21.15.135:0/300004465 learned_addr learned my addr 172.21.15.135:0/300004465 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:22.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.324+0000 7fc31e59c640 1 -- 172.21.15.135:0/300004465 >> v1:172.21.15.139:6789/0 conn(0x7fc3200ac610 legacy=0x7fc32006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:22.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.324+0000 7fc31e59c640 1 -- 172.21.15.135:0/300004465 >> v1:172.21.15.135:6789/0 conn(0x7fc320157680 legacy=0x7fc3200c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:22.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.324+0000 7fc31e59c640 1 -- 172.21.15.135:0/300004465 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3200c5630 con 0x7fc3201540f0 2024-04-26T22:06:22.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.325+0000 7fc31e59c640 1 --2- 172.21.15.135:0/300004465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3201540f0 0x7fc32015ac70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc320157370 tx=0x7fc31402a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b2cc040c16b9d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:22.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.326+0000 7fc31d59a640 1 -- 172.21.15.135:0/300004465 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314037070 con 0x7fc3201540f0 2024-04-26T22:06:22.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.326+0000 7fc31d59a640 1 -- 172.21.15.135:0/300004465 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31402aca0 con 0x7fc3201540f0 2024-04-26T22:06:22.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.326+0000 7fc31d59a640 1 -- 172.21.15.135:0/300004465 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314032660 con 0x7fc3201540f0 2024-04-26T22:06:22.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.328+0000 7fc324a47640 1 -- 172.21.15.135:0/300004465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3201540f0 msgr2=0x7fc32015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:22.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.328+0000 7fc324a47640 1 --2- 172.21.15.135:0/300004465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3201540f0 0x7fc32015ac70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc320157370 tx=0x7fc31402a010 comp rx=0 tx=0).stop 2024-04-26T22:06:22.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.330+0000 7fc324a47640 1 -- 172.21.15.135:0/300004465 shutdown_connections 2024-04-26T22:06:22.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.330+0000 7fc324a47640 1 --2- 172.21.15.135:0/300004465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3201540f0 0x7fc32015ac70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.330+0000 7fc324a47640 1 -- 172.21.15.135:0/300004465 >> 172.21.15.135:0/300004465 conn(0x7fc32006c310 msgr2=0x7fc32006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:22.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.332+0000 7fc324a47640 1 -- 172.21.15.135:0/300004465 shutdown_connections 2024-04-26T22:06:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.337+0000 7fc324a47640 1 -- 172.21.15.135:0/300004465 wait complete. 2024-04-26T22:06:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.339+0000 7fc324a47640 1 Processor -- start 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.343+0000 7fc324a47640 1 -- start start 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc324a47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3200ac610 0x7fc3201e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc324a47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 0x7fc3201e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc324a47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc320157680 0x7fc320150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc324a47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc32006f790 con 0x7fc320157680 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc324a47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc320157ac0 con 0x7fc3201540f0 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc324a47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc32006f910 con 0x7fc3200ac610 2024-04-26T22:06:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc31dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 0x7fc3201e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc31dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 0x7fc3201e1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40728/0 (socket says 172.21.15.135:40728) 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc31dd9b640 1 -- 172.21.15.135:0/3512908885 learned_addr learned my addr 172.21.15.135:0/3512908885 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.344+0000 7fc31ed9d640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc320157680 0x7fc320150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.345+0000 7fc31dd9b640 1 -- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3200ac610 msgr2=0x7fc3201e2a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.345+0000 7fc31dd9b640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3200ac610 0x7fc3201e2a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.345+0000 7fc31dd9b640 1 -- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc320157680 msgr2=0x7fc320150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.345+0000 7fc31dd9b640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc320157680 0x7fc320150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.345+0000 7fc31dd9b640 1 -- 172.21.15.135:0/3512908885 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3200c6ba0 con 0x7fc3201540f0 2024-04-26T22:06:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.345+0000 7fc31dd9b640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 0x7fc3201e1ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc31800caf0 tx=0x7fc318002090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:22.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.346+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318018070 con 0x7fc3201540f0 2024-04-26T22:06:22.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.346+0000 7fc324a47640 1 -- 172.21.15.135:0/3512908885 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3201e0b40 con 0x7fc3201540f0 2024-04-26T22:06:22.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.346+0000 7fc324a47640 1 -- 172.21.15.135:0/3512908885 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3201e0220 con 0x7fc3201540f0 2024-04-26T22:06:22.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.349+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31800a700 con 0x7fc3201540f0 2024-04-26T22:06:22.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.349+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31800ab30 con 0x7fc3201540f0 2024-04-26T22:06:22.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.349+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc31801b070 con 0x7fc3201540f0 2024-04-26T22:06:22.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.350+0000 7fc30effd640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc30006ca20 0x7fc30006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:22.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.350+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7fc31808ba70 con 0x7fc3201540f0 2024-04-26T22:06:22.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.350+0000 7fc31e59c640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc30006ca20 0x7fc30006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:22.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.351+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc304005290 con 0x7fc3201540f0 2024-04-26T22:06:22.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.351+0000 7fc31e59c640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc30006ca20 0x7fc30006eee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc31402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:22.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.366+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc318059250 con 0x7fc3201540f0 2024-04-26T22:06:22.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.812+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc304005580 con 0x7fc3201540f0 2024-04-26T22:06:22.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.815+0000 7fc30effd640 1 -- 172.21.15.135:0/3512908885 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7fc31805d550 con 0x7fc3201540f0 2024-04-26T22:06:22.818 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:22.818 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:22.819 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:22.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc30006ca20 msgr2=0x7fc30006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:22.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc30006ca20 0x7fc30006eee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc31402e000 comp rx=0 tx=0).stop 2024-04-26T22:06:22.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 msgr2=0x7fc3201e1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:22.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 0x7fc3201e1ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc31800caf0 tx=0x7fc318002090 comp rx=0 tx=0).stop 2024-04-26T22:06:22.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 shutdown_connections 2024-04-26T22:06:22.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc30006ca20 0x7fc30006eee0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc320157680 0x7fc320150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3201540f0 0x7fc3201e1ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 --2- 172.21.15.135:0/3512908885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3200ac610 0x7fc3201e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:22.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 >> 172.21.15.135:0/3512908885 conn(0x7fc32006c310 msgr2=0x7fc3200c2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:22.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.820+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 shutdown_connections 2024-04-26T22:06:22.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:22.821+0000 7fc30cff9640 1 -- 172.21.15.135:0/3512908885 wait complete. 2024-04-26T22:06:22.840 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:06:24.841 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.035+0000 7f9b25f5f640 1 Processor -- start 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.035+0000 7f9b25f5f640 1 -- start start 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.036+0000 7f9b25f5f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b180474a0 0x7f9b18006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.036+0000 7f9b25f5f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1800bb50 con 0x7f9b180073e0 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.036+0000 7f9b25f5f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b18048060 con 0x7f9b180dbce0 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.036+0000 7f9b25f5f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1800a420 con 0x7f9b180474a0 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.036+0000 7f9b1ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9b180073e0 0x7f9b1800a8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:50720/0 (socket says 172.21.15.135:50720) 2024-04-26T22:06:25.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.036+0000 7f9b1ffff640 1 -- 172.21.15.135:0/2524198841 learned_addr learned my addr 172.21.15.135:0/2524198841 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:25.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.038+0000 7f9b1effd640 1 --2- 172.21.15.135:0/2524198841 >> v2:172.21.15.139:3300/0 conn(0x7f9b180474a0 0x7f9b18006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:25.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.038+0000 7f9b1e7fc640 1 -- 172.21.15.135:0/2524198841 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370039516 0 0) 0x7f9b1800bb50 con 0x7f9b180073e0 2024-04-26T22:06:25.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.038+0000 7f9b1e7fc640 1 -- 172.21.15.135:0/2524198841 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b10003520 con 0x7f9b180073e0 2024-04-26T22:06:25.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.038+0000 7f9b1effd640 1 -- 172.21.15.135:0/2524198841 >> v1:172.21.15.139:6789/0 conn(0x7f9b180dbce0 legacy=0x7f9b180df180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:25.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.038+0000 7f9b1effd640 1 -- 172.21.15.135:0/2524198841 >> v1:172.21.15.135:6789/0 conn(0x7f9b180073e0 legacy=0x7f9b1800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.038+0000 7f9b1effd640 1 -- 172.21.15.135:0/2524198841 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1800a0f0 con 0x7f9b180474a0 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.040+0000 7f9b1effd640 1 --2- 172.21.15.135:0/2524198841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18006ea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b2006acf0 tx=0x7f9b200a7150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf94ed8ff6c924b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.042+0000 7f9b1e7fc640 1 -- 172.21.15.135:0/2524198841 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b200b4070 con 0x7f9b180474a0 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.042+0000 7f9b1e7fc640 1 -- 172.21.15.135:0/2524198841 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b200a7ca0 con 0x7f9b180474a0 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.043+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2524198841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 msgr2=0x7f9b18006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.043+0000 7f9b25f5f640 1 --2- 172.21.15.135:0/2524198841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18006ea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b2006acf0 tx=0x7f9b200a7150 comp rx=0 tx=0).stop 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.044+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2524198841 shutdown_connections 2024-04-26T22:06:25.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.044+0000 7f9b25f5f640 1 --2- 172.21.15.135:0/2524198841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18006ea0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.044+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2524198841 >> 172.21.15.135:0/2524198841 conn(0x7f9b18048250 msgr2=0x7f9b18048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:25.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.044+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2524198841 shutdown_connections 2024-04-26T22:06:25.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.044+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2524198841 wait complete. 2024-04-26T22:06:25.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.046+0000 7f9b25f5f640 1 Processor -- start 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.046+0000 7f9b25f5f640 1 -- start start 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.047+0000 7f9b25f5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b180073e0 0x7f9b18042210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.047+0000 7f9b25f5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18044190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.047+0000 7f9b25f5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b180dbce0 0x7f9b180de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.047+0000 7f9b25f5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9b180dc0c0 con 0x7f9b180dbce0 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.047+0000 7f9b25f5f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b18047d80 con 0x7f9b180474a0 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.047+0000 7f9b25f5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9b180dfbf0 con 0x7f9b180073e0 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.048+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18044190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.048+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18044190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40756/0 (socket says 172.21.15.135:40756) 2024-04-26T22:06:25.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.048+0000 7f9b1effd640 1 -- 172.21.15.135:0/2157152738 learned_addr learned my addr 172.21.15.135:0/2157152738 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.048+0000 7f9b1ffff640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b180dbce0 0x7f9b180de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.049+0000 7f9b1effd640 1 -- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b180073e0 msgr2=0x7f9b18042210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.048+0000 7f9b1f7fe640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b180073e0 0x7f9b18042210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.049+0000 7f9b1effd640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b180073e0 0x7f9b18042210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.049+0000 7f9b1effd640 1 -- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b180dbce0 msgr2=0x7f9b180de9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.049+0000 7f9b1effd640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b180dbce0 0x7f9b180de9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.049+0000 7f9b1effd640 1 -- 172.21.15.135:0/2157152738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b18041810 con 0x7f9b180474a0 2024-04-26T22:06:25.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.049+0000 7f9b1ffff640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b180dbce0 0x7f9b180de9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:25.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.050+0000 7f9b1f7fe640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b180073e0 0x7f9b18042210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:25.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.050+0000 7f9b1effd640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18044190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9b20051fc0 tx=0x7f9b2006b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:25.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.050+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b200bf020 con 0x7f9b180474a0 2024-04-26T22:06:25.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.051+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2157152738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b180446d0 con 0x7f9b180474a0 2024-04-26T22:06:25.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.051+0000 7f9b25f5f640 1 -- 172.21.15.135:0/2157152738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b18040310 con 0x7f9b180474a0 2024-04-26T22:06:25.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.052+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b200b5040 con 0x7f9b180474a0 2024-04-26T22:06:25.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.053+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b200b4070 con 0x7f9b180474a0 2024-04-26T22:06:25.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.055+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b18005390 con 0x7f9b180474a0 2024-04-26T22:06:25.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.055+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b200c2020 con 0x7f9b180474a0 2024-04-26T22:06:25.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.056+0000 7f9b0bfff640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9b0406c9c0 0x7f9b0406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:25.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.056+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f9b20129c30 con 0x7f9b180474a0 2024-04-26T22:06:25.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.056+0000 7f9b1f7fe640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9b0406c9c0 0x7f9b0406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:25.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.057+0000 7f9b1f7fe640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9b0406c9c0 0x7f9b0406ee80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b1802cbd0 tx=0x7f9b14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:25.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.065+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b180446d0 con 0x7f9b180474a0 2024-04-26T22:06:25.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.520+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b180df6a0 con 0x7f9b180474a0 2024-04-26T22:06:25.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.521+0000 7f9b0bfff640 1 -- 172.21.15.135:0/2157152738 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f9b200f7a60 con 0x7f9b180474a0 2024-04-26T22:06:25.525 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:25.525 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:25.526 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9b0406c9c0 msgr2=0x7f9b0406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9b0406c9c0 0x7f9b0406ee80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b1802cbd0 tx=0x7f9b14008040 comp rx=0 tx=0).stop 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 msgr2=0x7f9b18044190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18044190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9b20051fc0 tx=0x7f9b2006b850 comp rx=0 tx=0).stop 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 shutdown_connections 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9b0406c9c0 0x7f9b0406ee80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b180dbce0 0x7f9b180de9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b180474a0 0x7f9b18044190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 --2- 172.21.15.135:0/2157152738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b180073e0 0x7f9b18042210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:25.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.528+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 >> 172.21.15.135:0/2157152738 conn(0x7f9b18048250 msgr2=0x7f9b180dfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:25.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.533+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 shutdown_connections 2024-04-26T22:06:25.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:25.533+0000 7f9b09ffb640 1 -- 172.21.15.135:0/2157152738 wait complete. 2024-04-26T22:06:25.555 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:06:27.556 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:27.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.744+0000 7f4885f0f640 1 Processor -- start 2024-04-26T22:06:27.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.744+0000 7f4885f0f640 1 -- start start 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f4885f0f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f48800b0f20 0x7f488015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f4885f0f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48800bf450 0x7f48800af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f4885f0f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 0x7f48800c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f4885f0f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f48800af700 con 0x7f48800bf450 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f4885f0f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f48800b1e10 con 0x7f48800b0f20 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f4885f0f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f48800b20f0 con 0x7f48800cbaf0 2024-04-26T22:06:27.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f488570e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 0x7f48800c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f488570e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 0x7f48800c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:37356/0 (socket says 172.21.15.135:37356) 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.745+0000 7f488570e640 1 -- 172.21.15.135:0/2564867989 learned_addr learned my addr 172.21.15.135:0/2564867989 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.746+0000 7f488570e640 1 -- 172.21.15.135:0/2564867989 >> v2:172.21.15.139:3300/0 conn(0x7f48800b0f20 msgr2=0x7f488015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.746+0000 7f488570e640 1 --2- 172.21.15.135:0/2564867989 >> v2:172.21.15.139:3300/0 conn(0x7f48800b0f20 0x7f488015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.746+0000 7f488570e640 1 -- 172.21.15.135:0/2564867989 >> v2:172.21.15.135:3300/0 conn(0x7f48800bf450 msgr2=0x7f48800af1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.746+0000 7f488570e640 1 --2- 172.21.15.135:0/2564867989 >> v2:172.21.15.135:3300/0 conn(0x7f48800bf450 0x7f48800af1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.746+0000 7f488570e640 1 -- 172.21.15.135:0/2564867989 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48800cae20 con 0x7f48800cbaf0 2024-04-26T22:06:27.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.749+0000 7f488570e640 1 --2- 172.21.15.135:0/2564867989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 0x7f48800c9270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48800abcb0 tx=0x7f487002a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c22c74c8e10db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:27.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.749+0000 7f487f7fe640 1 -- 172.21.15.135:0/2564867989 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870037070 con 0x7f48800cbaf0 2024-04-26T22:06:27.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.749+0000 7f487f7fe640 1 -- 172.21.15.135:0/2564867989 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487002ae30 con 0x7f48800cbaf0 2024-04-26T22:06:27.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.750+0000 7f4885f0f640 1 -- 172.21.15.135:0/2564867989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 msgr2=0x7f48800c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:27.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.750+0000 7f4885f0f640 1 --2- 172.21.15.135:0/2564867989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 0x7f48800c9270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48800abcb0 tx=0x7f487002a0e0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.751+0000 7f4885f0f640 1 -- 172.21.15.135:0/2564867989 shutdown_connections 2024-04-26T22:06:27.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.751+0000 7f4885f0f640 1 --2- 172.21.15.135:0/2564867989 >> v2:172.21.15.135:3300/0 conn(0x7f48800bf450 0x7f48800af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.751+0000 7f4885f0f640 1 --2- 172.21.15.135:0/2564867989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800cbaf0 0x7f48800c9270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.751+0000 7f4885f0f640 1 --2- 172.21.15.135:0/2564867989 >> v2:172.21.15.139:3300/0 conn(0x7f48800b0f20 0x7f488015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.751+0000 7f4885f0f640 1 -- 172.21.15.135:0/2564867989 >> 172.21.15.135:0/2564867989 conn(0x7f48800adea0 msgr2=0x7f48800ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:27.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.752+0000 7f4885f0f640 1 -- 172.21.15.135:0/2564867989 shutdown_connections 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.752+0000 7f4885f0f640 1 -- 172.21.15.135:0/2564867989 wait complete. 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.754+0000 7f4885f0f640 1 Processor -- start 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.754+0000 7f4885f0f640 1 -- start start 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.755+0000 7f4885f0f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800b0f20 0x7f48802050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.755+0000 7f4885f0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 0x7f4880158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.755+0000 7f4885f0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48800cbaf0 0x7f48801fea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.755+0000 7f4885f0f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48800b1480 con 0x7f48800bf450 2024-04-26T22:06:27.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.755+0000 7f4885f0f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48800b1300 con 0x7f48800cbaf0 2024-04-26T22:06:27.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.755+0000 7f4885f0f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f48800c97b0 con 0x7f48800b0f20 2024-04-26T22:06:27.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.756+0000 7f487ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 0x7f4880158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.756+0000 7f487ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 0x7f4880158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44316/0 (socket says 172.21.15.135:44316) 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.756+0000 7f487ffff640 1 -- 172.21.15.135:0/4137845984 learned_addr learned my addr 172.21.15.135:0/4137845984 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.756+0000 7f4884f0d640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800b0f20 0x7f48802050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.757+0000 7f487ffff640 1 -- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800b0f20 msgr2=0x7f48802050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.757+0000 7f487ffff640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800b0f20 0x7f48802050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.757+0000 7f487ffff640 1 -- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48800cbaf0 msgr2=0x7f48801fea90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.757+0000 7f487ffff640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48800cbaf0 0x7f48801fea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.757+0000 7f487ffff640 1 -- 172.21.15.135:0/4137845984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48800c0b90 con 0x7f48800bf450 2024-04-26T22:06:27.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.757+0000 7f487ffff640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 0x7f4880158110 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f487400b480 tx=0x7f487400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:27.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.759+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874014070 con 0x7f48800bf450 2024-04-26T22:06:27.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.759+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4874010870 con 0x7f48800bf450 2024-04-26T22:06:27.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.759+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874010c60 con 0x7f48800bf450 2024-04-26T22:06:27.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.759+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4880205ad0 con 0x7f48800bf450 2024-04-26T22:06:27.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.759+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4880159290 con 0x7f48800bf450 2024-04-26T22:06:27.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.764+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4874017070 con 0x7f48800bf450 2024-04-26T22:06:27.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.765+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48800506c0 con 0x7f48800bf450 2024-04-26T22:06:27.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.765+0000 7f487d7fa640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f486c06c9f0 0x7f486c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.765+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f487408bfc0 con 0x7f48800bf450 2024-04-26T22:06:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.766+0000 7f4884f0d640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f486c06c9f0 0x7f486c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:27.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.767+0000 7f4884f0d640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f486c06c9f0 0x7f486c06eeb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4878009f20 tx=0x7f48780094f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:27.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:27.777+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f487401e030 con 0x7f48800bf450 2024-04-26T22:06:28.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.224+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48800cb790 con 0x7f48800bf450 2024-04-26T22:06:28.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.226+0000 7f487d7fa640 1 -- 172.21.15.135:0/4137845984 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f4874020300 con 0x7f48800bf450 2024-04-26T22:06:28.231 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:28.231 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:28.232 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:28.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.232+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f486c06c9f0 msgr2=0x7f486c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:28.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.232+0000 7f4885f0f640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f486c06c9f0 0x7f486c06eeb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4878009f20 tx=0x7f48780094f0 comp rx=0 tx=0).stop 2024-04-26T22:06:28.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.232+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 msgr2=0x7f4880158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:28.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.232+0000 7f4885f0f640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 0x7f4880158110 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f487400b480 tx=0x7f487400b940 comp rx=0 tx=0).stop 2024-04-26T22:06:28.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.233+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 shutdown_connections 2024-04-26T22:06:28.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.233+0000 7f4885f0f640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f486c06c9f0 0x7f486c06eeb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:28.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.233+0000 7f4885f0f640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48800cbaf0 0x7f48801fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:28.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.233+0000 7f4885f0f640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48800bf450 0x7f4880158110 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:28.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.233+0000 7f4885f0f640 1 --2- 172.21.15.135:0/4137845984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48800b0f20 0x7f48802050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:28.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.233+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 >> 172.21.15.135:0/4137845984 conn(0x7f48800adea0 msgr2=0x7f488015bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:28.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.239+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 shutdown_connections 2024-04-26T22:06:28.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:28.239+0000 7f4885f0f640 1 -- 172.21.15.135:0/4137845984 wait complete. 2024-04-26T22:06:28.259 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 49, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 49, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5105": { "addr": "172.21.15.139:6835/3733666635", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 3733666635, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 3733666635, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5105, "incarnation": 49, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:06:12.822242+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5105 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 47, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5610": { "addr": "172.21.15.135:6837/2165063360", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 2165063360, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 2165063360, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5610, "incarnation": 44, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:05:57.871105+0000", "qdb_cluster": [ 5610 ], "qdb_leader": 5610, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5610 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:06:30.133 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:06:30.135 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:06:30.260 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.431+0000 7f4d79746640 1 Processor -- start 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.431+0000 7f4d79746640 1 -- start start 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.432+0000 7f4d79746640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 0x7f4d7406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.432+0000 7f4d79746640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d7415c0d0 con 0x7f4d740aa8c0 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.432+0000 7f4d79746640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d740ab4b0 con 0x7f4d740abf50 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.432+0000 7f4d79746640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7415bdf0 con 0x7f4d740c3650 2024-04-26T22:06:30.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.433+0000 7f4d72ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 0x7f4d7406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:30.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.433+0000 7f4d72ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 0x7f4d7406e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:32812/0 (socket says 172.21.15.135:32812) 2024-04-26T22:06:30.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.433+0000 7f4d72ffd640 1 -- 172.21.15.135:0/457781712 learned_addr learned my addr 172.21.15.135:0/457781712 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:30.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.433+0000 7f4d72ffd640 1 -- 172.21.15.135:0/457781712 >> v1:172.21.15.139:6789/0 conn(0x7f4d740abf50 legacy=0x7f4d7415a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.433+0000 7f4d72ffd640 1 -- 172.21.15.135:0/457781712 >> v1:172.21.15.135:6789/0 conn(0x7f4d740aa8c0 legacy=0x7f4d740c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.433+0000 7f4d72ffd640 1 -- 172.21.15.135:0/457781712 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d740ab750 con 0x7f4d740c3650 2024-04-26T22:06:30.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.435+0000 7f4d72ffd640 1 --2- 172.21.15.135:0/457781712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 0x7f4d7406e760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e80 tx=0x7f4d6c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86aa1cedfa383c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:30.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.435+0000 7f4d71ffb640 1 -- 172.21.15.135:0/457781712 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c036070 con 0x7f4d740c3650 2024-04-26T22:06:30.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.436+0000 7f4d71ffb640 1 -- 172.21.15.135:0/457781712 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c031600 con 0x7f4d740c3650 2024-04-26T22:06:30.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.436+0000 7f4d71ffb640 1 -- 172.21.15.135:0/457781712 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0319d0 con 0x7f4d740c3650 2024-04-26T22:06:30.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.440+0000 7f4d79746640 1 -- 172.21.15.135:0/457781712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 msgr2=0x7f4d7406e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.440+0000 7f4d79746640 1 --2- 172.21.15.135:0/457781712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 0x7f4d7406e760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e80 tx=0x7f4d6c02a6c0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.442+0000 7f4d79746640 1 -- 172.21.15.135:0/457781712 shutdown_connections 2024-04-26T22:06:30.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.442+0000 7f4d79746640 1 --2- 172.21.15.135:0/457781712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740c3650 0x7f4d7406e760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.442+0000 7f4d79746640 1 -- 172.21.15.135:0/457781712 >> 172.21.15.135:0/457781712 conn(0x7f4d740c82d0 msgr2=0x7f4d740c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:30.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.443+0000 7f4d79746640 1 -- 172.21.15.135:0/457781712 shutdown_connections 2024-04-26T22:06:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.444+0000 7f4d79746640 1 -- 172.21.15.135:0/457781712 wait complete. 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.446+0000 7f4d79746640 1 Processor -- start 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.447+0000 7f4d79746640 1 -- start start 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d79746640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740aa8c0 0x7f4d74157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d79746640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d740abf50 0x7f4d740bd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d79746640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d740c3650 0x7f4d740729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d79746640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d740c42c0 con 0x7f4d740c3650 2024-04-26T22:06:30.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d79746640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d740c4140 con 0x7f4d740abf50 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d79746640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7415adb0 con 0x7f4d740aa8c0 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d740c3650 0x7f4d740729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d740c3650 0x7f4d740729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38246/0 (socket says 172.21.15.135:38246) 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d737fe640 1 -- 172.21.15.135:0/3236872130 learned_addr learned my addr 172.21.15.135:0/3236872130 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d72ffd640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740aa8c0 0x7f4d74157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.448+0000 7f4d727fc640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d740abf50 0x7f4d740bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.449+0000 7f4d72ffd640 1 -- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d740abf50 msgr2=0x7f4d740bd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.449+0000 7f4d72ffd640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d740abf50 0x7f4d740bd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.449+0000 7f4d72ffd640 1 -- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d740c3650 msgr2=0x7f4d740729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.449+0000 7f4d72ffd640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d740c3650 0x7f4d740729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.449+0000 7f4d72ffd640 1 -- 172.21.15.135:0/3236872130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d740c1720 con 0x7f4d740aa8c0 2024-04-26T22:06:30.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.449+0000 7f4d72ffd640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740aa8c0 0x7f4d74157cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c030000 tx=0x7f4d6c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:30.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.452+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c03f020 con 0x7f4d740aa8c0 2024-04-26T22:06:30.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.452+0000 7f4d79746640 1 -- 172.21.15.135:0/3236872130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d741533c0 con 0x7f4d740aa8c0 2024-04-26T22:06:30.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.452+0000 7f4d79746640 1 -- 172.21.15.135:0/3236872130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d74153960 con 0x7f4d740aa8c0 2024-04-26T22:06:30.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.452+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c037040 con 0x7f4d740aa8c0 2024-04-26T22:06:30.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.452+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c036070 con 0x7f4d740aa8c0 2024-04-26T22:06:30.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.455+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d6c042020 con 0x7f4d740aa8c0 2024-04-26T22:06:30.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.455+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d54005290 con 0x7f4d740aa8c0 2024-04-26T22:06:30.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.456+0000 7f4d5b7fe640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4d6006c9c0 0x7f4d6006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:30.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.456+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f4d6c0ac560 con 0x7f4d740aa8c0 2024-04-26T22:06:30.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.457+0000 7f4d727fc640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4d6006c9c0 0x7f4d6006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:30.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.458+0000 7f4d727fc640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4d6006c9c0 0x7f4d6006ee80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d74074770 tx=0x7f4d64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:30.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.465+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d6c049030 con 0x7f4d740aa8c0 2024-04-26T22:06:30.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.867+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d54005580 con 0x7f4d740aa8c0 2024-04-26T22:06:30.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.868+0000 7f4d5b7fe640 1 -- 172.21.15.135:0/3236872130 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5667 (secure 0 0 0) 0x7f4d6c079d80 con 0x7f4d740aa8c0 2024-04-26T22:06:30.868 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:30.868 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:12.822242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:30.869 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 49 2024-04-26T22:06:30.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4d6006c9c0 msgr2=0x7f4d6006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4d6006c9c0 0x7f4d6006ee80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d74074770 tx=0x7f4d64009040 comp rx=0 tx=0).stop 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740aa8c0 msgr2=0x7f4d74157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740aa8c0 0x7f4d74157cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c030000 tx=0x7f4d6c004a40 comp rx=0 tx=0).stop 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 shutdown_connections 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4d6006c9c0 0x7f4d6006ee80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d740c3650 0x7f4d740729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d740abf50 0x7f4d740bd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 --2- 172.21.15.135:0/3236872130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d740aa8c0 0x7f4d74157cf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 >> 172.21.15.135:0/3236872130 conn(0x7f4d740c82d0 msgr2=0x7f4d74158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 shutdown_connections 2024-04-26T22:06:30.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:30.871+0000 7f4d597fa640 1 -- 172.21.15.135:0/3236872130 wait complete. 2024-04-26T22:06:30.885 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:06:32.886 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:33.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.067+0000 7f73d25da640 1 Processor -- start 2024-04-26T22:06:33.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.067+0000 7f73d25da640 1 -- start start 2024-04-26T22:06:33.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73d25da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73cc150200 0x7f73cc0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:33.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73d25da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc0ac680 0x7f73cc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:33.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73d25da640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73cc06ff20 con 0x7f73cc0c2cd0 2024-04-26T22:06:33.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73d25da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f73cc0c4bd0 con 0x7f73cc150200 2024-04-26T22:06:33.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73d25da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc06f4e0 con 0x7f73cc0ac680 2024-04-26T22:06:33.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73cbfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73cc150200 0x7f73cc0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:33.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73cbfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73cc150200 0x7f73cc0ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38252/0 (socket says 172.21.15.135:38252) 2024-04-26T22:06:33.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73cb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f73cc0c2cd0 0x7f73cc1533b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:47324/0 (socket says 172.21.15.135:47324) 2024-04-26T22:06:33.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.069+0000 7f73cb7fe640 1 -- 172.21.15.135:0/479854754 learned_addr learned my addr 172.21.15.135:0/479854754 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:33.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.070+0000 7f73cbfff640 1 -- 172.21.15.135:0/479854754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc0ac680 msgr2=0x7f73cc15d590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:33.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.070+0000 7f73cbfff640 1 --2- 172.21.15.135:0/479854754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc0ac680 0x7f73cc15d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.070+0000 7f73cbfff640 1 -- 172.21.15.135:0/479854754 >> v1:172.21.15.139:6789/0 conn(0x7f73cc0c2cd0 legacy=0x7f73cc1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:33.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.070+0000 7f73cbfff640 1 -- 172.21.15.135:0/479854754 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc0c48f0 con 0x7f73cc150200 2024-04-26T22:06:33.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.071+0000 7f73cbfff640 1 --2- 172.21.15.135:0/479854754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc150200 0x7f73cc0ac140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f73c00097a0 tx=0x7f73c002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db109702d02b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:33.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.072+0000 7f73caffd640 1 -- 172.21.15.135:0/479854754 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0037070 con 0x7f73cc150200 2024-04-26T22:06:33.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.072+0000 7f73caffd640 1 -- 172.21.15.135:0/479854754 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c002ade0 con 0x7f73cc150200 2024-04-26T22:06:33.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.072+0000 7f73caffd640 1 -- 172.21.15.135:0/479854754 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0032660 con 0x7f73cc150200 2024-04-26T22:06:33.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.075+0000 7f73d25da640 1 -- 172.21.15.135:0/479854754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc150200 msgr2=0x7f73cc0ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:33.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.075+0000 7f73d25da640 1 --2- 172.21.15.135:0/479854754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc150200 0x7f73cc0ac140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f73c00097a0 tx=0x7f73c002a0d0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.078+0000 7f73d25da640 1 -- 172.21.15.135:0/479854754 shutdown_connections 2024-04-26T22:06:33.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.078+0000 7f73d25da640 1 --2- 172.21.15.135:0/479854754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc0ac680 0x7f73cc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.078+0000 7f73d25da640 1 --2- 172.21.15.135:0/479854754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc150200 0x7f73cc0ac140 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.078+0000 7f73d25da640 1 -- 172.21.15.135:0/479854754 >> 172.21.15.135:0/479854754 conn(0x7f73cc0c82d0 msgr2=0x7f73cc0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:33.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.079+0000 7f73d25da640 1 -- 172.21.15.135:0/479854754 shutdown_connections 2024-04-26T22:06:33.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.082+0000 7f73d25da640 1 -- 172.21.15.135:0/479854754 wait complete. 2024-04-26T22:06:33.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.083+0000 7f73d25da640 1 Processor -- start 2024-04-26T22:06:33.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.085+0000 7f73d25da640 1 -- start start 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.086+0000 7f73d25da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 0x7f73cc13fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.086+0000 7f73d25da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc0c2cd0 0x7f73cc144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.086+0000 7f73d25da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc150200 0x7f73cc13d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.086+0000 7f73d25da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc150760 con 0x7f73cc0c2cd0 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.086+0000 7f73d25da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1505e0 con 0x7f73cc0ac680 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.086+0000 7f73d25da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc153ac0 con 0x7f73cc150200 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.088+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 0x7f73cc13fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.090+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 0x7f73cc13fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:41558/0 (socket says 172.21.15.135:41558) 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.090+0000 7f73cbfff640 1 -- 172.21.15.135:0/3203568359 learned_addr learned my addr 172.21.15.135:0/3203568359 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73d0b77640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc150200 0x7f73cc13d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73cb7fe640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc0c2cd0 0x7f73cc144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73cbfff640 1 -- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc150200 msgr2=0x7f73cc13d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73cbfff640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc150200 0x7f73cc13d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73cbfff640 1 -- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc0c2cd0 msgr2=0x7f73cc144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:33.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73cbfff640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc0c2cd0 0x7f73cc144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.092+0000 7f73cbfff640 1 -- 172.21.15.135:0/3203568359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc14dae0 con 0x7f73cc0ac680 2024-04-26T22:06:33.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.093+0000 7f73cbfff640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 0x7f73cc13fc30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73c002a6c0 tx=0x7f73c0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:33.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.102+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0042020 con 0x7f73cc0ac680 2024-04-26T22:06:33.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.102+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c003d670 con 0x7f73cc0ac680 2024-04-26T22:06:33.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.103+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc13f330 con 0x7f73cc0ac680 2024-04-26T22:06:33.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.103+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc14d800 con 0x7f73cc0ac680 2024-04-26T22:06:33.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.103+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0037040 con 0x7f73cc0ac680 2024-04-26T22:06:33.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.106+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac005290 con 0x7f73cc0ac680 2024-04-26T22:06:33.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.112+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73c0045020 con 0x7f73cc0ac680 2024-04-26T22:06:33.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.112+0000 7f73c8ff9640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73b806c9c0 0x7f73b806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:33.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.113+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f73c00acd10 con 0x7f73cc0ac680 2024-04-26T22:06:33.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.114+0000 7f73cb7fe640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73b806c9c0 0x7f73b806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:33.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.116+0000 7f73cb7fe640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73b806c9c0 0x7f73b806ee80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005620 tx=0x7f73c40055b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:33.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.118+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73c004c070 con 0x7f73cc0ac680 2024-04-26T22:06:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.605+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73ac005580 con 0x7f73cc0ac680 2024-04-26T22:06:33.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.608+0000 7f73c8ff9640 1 -- 172.21.15.135:0/3203568359 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5668 (secure 0 0 0) 0x7f73cc06ff20 con 0x7f73cc0ac680 2024-04-26T22:06:33.609 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:33.609 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:32.115512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 51 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73b806c9c0 msgr2=0x7f73b806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73b806c9c0 0x7f73b806ee80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005620 tx=0x7f73c40055b0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 msgr2=0x7f73cc13fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 0x7f73cc13fc30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73c002a6c0 tx=0x7f73c0004730 comp rx=0 tx=0).stop 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 shutdown_connections 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73b806c9c0 0x7f73b806ee80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73cc150200 0x7f73cc13d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73cc0c2cd0 0x7f73cc144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 --2- 172.21.15.135:0/3203568359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73cc0ac680 0x7f73cc13fc30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:33.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 >> 172.21.15.135:0/3203568359 conn(0x7f73cc0c82d0 msgr2=0x7f73cc0ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:33.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 shutdown_connections 2024-04-26T22:06:33.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:33.615+0000 7f73d25da640 1 -- 172.21.15.135:0/3203568359 wait complete. 2024-04-26T22:06:33.637 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:06:34.909 INFO:tasks.ceph.mds.b.smithi139.stderr:2024-04-26T22:06:34.909+0000 7f084bf8d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:06:35.637 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.809+0000 7fb733dac640 1 Processor -- start 2024-04-26T22:06:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.810+0000 7fb733dac640 1 -- start start 2024-04-26T22:06:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb733dac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb72c150d20 0x7fb72c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb733dac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb733dac640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72c0c3d10 con 0x7fb72c06f790 2024-04-26T22:06:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb733dac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb72c0c4290 con 0x7fb72c150d20 2024-04-26T22:06:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb733dac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0c3fb0 con 0x7fb72c157680 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb731347640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb72c06f790 0x7fb72c0c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:45366/0 (socket says 172.21.15.135:45366) 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb731347640 1 -- 172.21.15.135:0/936158462 learned_addr learned my addr 172.21.15.135:0/936158462 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.811+0000 7fb731b48640 1 --2- 172.21.15.135:0/936158462 >> v2:172.21.15.139:3300/0 conn(0x7fb72c150d20 0x7fb72c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.812+0000 7fb731b48640 1 -- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 msgr2=0x7fb72c06d320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.812+0000 7fb731b48640 1 --2- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c06d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.812+0000 7fb731b48640 1 -- 172.21.15.135:0/936158462 >> v1:172.21.15.135:6789/0 conn(0x7fb72c06f790 legacy=0x7fb72c0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.812+0000 7fb731b48640 1 -- 172.21.15.135:0/936158462 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c156e80 con 0x7fb72c150d20 2024-04-26T22:06:35.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.814+0000 7fb731b48640 1 --2- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c150d20 0x7fb72c15a360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7280097a0 tx=0x7fb72802a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec2d58317977780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.816+0000 7fb730b46640 1 -- 172.21.15.135:0/936158462 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728037070 con 0x7fb72c150d20 2024-04-26T22:06:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.816+0000 7fb730b46640 1 -- 172.21.15.135:0/936158462 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72802aca0 con 0x7fb72c150d20 2024-04-26T22:06:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.816+0000 7fb730b46640 1 -- 172.21.15.135:0/936158462 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728032660 con 0x7fb72c150d20 2024-04-26T22:06:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.817+0000 7fb733dac640 1 -- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c150d20 msgr2=0x7fb72c15a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:35.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.817+0000 7fb733dac640 1 --2- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c150d20 0x7fb72c15a360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7280097a0 tx=0x7fb72802a010 comp rx=0 tx=0).stop 2024-04-26T22:06:35.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.820+0000 7fb733dac640 1 -- 172.21.15.135:0/936158462 shutdown_connections 2024-04-26T22:06:35.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.820+0000 7fb733dac640 1 --2- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:35.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.820+0000 7fb733dac640 1 --2- 172.21.15.135:0/936158462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c150d20 0x7fb72c15a360 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:35.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.820+0000 7fb733dac640 1 -- 172.21.15.135:0/936158462 >> 172.21.15.135:0/936158462 conn(0x7fb72c0c5ce0 msgr2=0x7fb72c0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:35.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.822+0000 7fb733dac640 1 -- 172.21.15.135:0/936158462 shutdown_connections 2024-04-26T22:06:35.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.823+0000 7fb733dac640 1 -- 172.21.15.135:0/936158462 wait complete. 2024-04-26T22:06:35.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.825+0000 7fb733dac640 1 Processor -- start 2024-04-26T22:06:35.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.825+0000 7fb733dac640 1 -- start start 2024-04-26T22:06:35.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.826+0000 7fb733dac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c06f790 0x7fb72c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:35.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.826+0000 7fb733dac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 0x7fb72c14ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:35.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.826+0000 7fb733dac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:35.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.826+0000 7fb733dac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c151280 con 0x7fb72c150d20 2024-04-26T22:06:35.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.826+0000 7fb733dac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c151100 con 0x7fb72c06f790 2024-04-26T22:06:35.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.826+0000 7fb733dac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c157af0 con 0x7fb72c157680 2024-04-26T22:06:35.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.827+0000 7fb731347640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 0x7fb72c14ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:35.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.827+0000 7fb731347640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 0x7fb72c14ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:38276/0 (socket says 172.21.15.135:38276) 2024-04-26T22:06:35.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.827+0000 7fb731347640 1 -- 172.21.15.135:0/3707936032 learned_addr learned my addr 172.21.15.135:0/3707936032 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.827+0000 7fb731b48640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c06f790 0x7fb72c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.827+0000 7fb732349640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.827+0000 7fb731347640 1 -- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 msgr2=0x7fb72c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.828+0000 7fb731347640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.828+0000 7fb731347640 1 -- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c06f790 msgr2=0x7fb72c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.828+0000 7fb731347640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c06f790 0x7fb72c06d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.828+0000 7fb731347640 1 -- 172.21.15.135:0/3707936032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c154140 con 0x7fb72c150d20 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.828+0000 7fb731347640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 0x7fb72c14ed30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c750 tx=0x7fb71c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.828+0000 7fb731b48640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c06f790 0x7fb72c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.829+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c017070 con 0x7fb72c150d20 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.829+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c005ae0 con 0x7fb72c150d20 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.829+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1ed550 con 0x7fb72c150d20 2024-04-26T22:06:35.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.829+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c06d7b0 con 0x7fb72c150d20 2024-04-26T22:06:35.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.829+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c007070 con 0x7fb72c150d20 2024-04-26T22:06:35.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.831+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c068dc0 con 0x7fb72c150d20 2024-04-26T22:06:35.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.842+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb71c01a070 con 0x7fb72c150d20 2024-04-26T22:06:35.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.842+0000 7fb7227fc640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb71006c9c0 0x7fb71006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:35.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.843+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7fb71c08b5f0 con 0x7fb72c150d20 2024-04-26T22:06:35.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.843+0000 7fb731b48640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb71006c9c0 0x7fb71006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:35.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.843+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb71c08daf0 con 0x7fb72c150d20 2024-04-26T22:06:35.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:35.844+0000 7fb731b48640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb71006c9c0 0x7fb71006ee80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb728034000 tx=0x7fb72802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:36.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.252+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72c0507a0 con 0x7fb72c150d20 2024-04-26T22:06:36.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.253+0000 7fb7227fc640 1 -- 172.21.15.135:0/3707936032 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5675 (secure 0 0 0) 0x7fb71c058dd0 con 0x7fb72c150d20 2024-04-26T22:06:36.253 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:36.254 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:36.254 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 52 2024-04-26T22:06:36.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb71006c9c0 msgr2=0x7fb71006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:36.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb71006c9c0 0x7fb71006ee80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb728034000 tx=0x7fb72802e000 comp rx=0 tx=0).stop 2024-04-26T22:06:36.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 msgr2=0x7fb72c14ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:36.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 0x7fb72c14ed30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c750 tx=0x7fb71c00cc10 comp rx=0 tx=0).stop 2024-04-26T22:06:36.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 shutdown_connections 2024-04-26T22:06:36.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb71006c9c0 0x7fb71006ee80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:36.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb72c157680 0x7fb72c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:36.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c150d20 0x7fb72c14ed30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:36.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 --2- 172.21.15.135:0/3707936032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72c06f790 0x7fb72c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:36.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 >> 172.21.15.135:0/3707936032 conn(0x7fb72c0c5ce0 msgr2=0x7fb72c0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:36.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.258+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 shutdown_connections 2024-04-26T22:06:36.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:36.260+0000 7fb733dac640 1 -- 172.21.15.135:0/3707936032 wait complete. 2024-04-26T22:06:36.283 INFO:tasks.mds_thrash.fs.[a]:mds.b has gained rank=0, replacing gid=5272 2024-04-26T22:06:36.283 INFO:tasks.mds_thrash.fs.[a]:waiting for 6 secs before reviving mds.c 2024-04-26T22:06:42.284 INFO:tasks.mds_thrash.fs.[a]:reviving mds.c 2024-04-26T22:06:42.284 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:06:42.284 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:06:42.287 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:06:42.287 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:42.431 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.465+0000 7fe0bf7f0640 1 Processor -- start 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.466+0000 7fe0bf7f0640 1 -- start start 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bf7f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe0b80c3ee0 0x7fe0b8167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bf7f0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b806d530 con 0x7fe0b80c4730 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bf7f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b80748c0 con 0x7fe0b80c5880 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bf7f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe0b806db60 con 0x7fe0b80c3ee0 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bcd8b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe0b80c5880 0x7fe0b80ac7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54038/0 (socket says 172.21.15.135:54038) 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bcd8b640 1 -- 172.21.15.135:0/3096516410 learned_addr learned my addr 172.21.15.135:0/3096516410 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:42.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.467+0000 7fe0bd58c640 1 --2- 172.21.15.135:0/3096516410 >> v2:172.21.15.135:3300/0 conn(0x7fe0b80c3ee0 0x7fe0b8167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.469+0000 7fe0bd58c640 1 -- 172.21.15.135:0/3096516410 >> v1:172.21.15.139:6789/0 conn(0x7fe0b80c5880 legacy=0x7fe0b80ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.469+0000 7fe0bd58c640 1 -- 172.21.15.135:0/3096516410 >> v1:172.21.15.135:6789/0 conn(0x7fe0b80c4730 legacy=0x7fe0b80acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.469+0000 7fe0bd58c640 1 -- 172.21.15.135:0/3096516410 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b8074ba0 con 0x7fe0b80c3ee0 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.470+0000 7fe0bd58c640 1 --2- 172.21.15.135:0/3096516410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8167480 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe0b806dce0 tx=0x7fe0b402e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2b10432dc6a327 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.470+0000 7fe0affff640 1 -- 172.21.15.135:0/3096516410 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40090d0 con 0x7fe0b80c3ee0 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.471+0000 7fe0affff640 1 -- 172.21.15.135:0/3096516410 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b4009270 con 0x7fe0b80c3ee0 2024-04-26T22:06:42.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.471+0000 7fe0affff640 1 -- 172.21.15.135:0/3096516410 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40076b0 con 0x7fe0b80c3ee0 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/3096516410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 msgr2=0x7fe0b8167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/3096516410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8167480 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe0b806dce0 tx=0x7fe0b402e480 comp rx=0 tx=0).stop 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/3096516410 shutdown_connections 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/3096516410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8167480 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/3096516410 >> 172.21.15.135:0/3096516410 conn(0x7fe0b80c4d60 msgr2=0x7fe0b80c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/3096516410 shutdown_connections 2024-04-26T22:06:42.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.472+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/3096516410 wait complete. 2024-04-26T22:06:42.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.474+0000 7fe0bf7f0640 1 Processor -- start 2024-04-26T22:06:42.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.474+0000 7fe0bf7f0640 1 -- start start 2024-04-26T22:06:42.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.475+0000 7fe0bf7f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8204eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bd58c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8204eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bd58c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8204eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58264/0 (socket says 172.21.15.135:58264) 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bd58c640 1 -- 172.21.15.135:0/1374093639 learned_addr learned my addr 172.21.15.135:0/1374093639 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0b80c4730 0x7fe0b8072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0b80c5880 0x7fe0b8201f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b80c42c0 con 0x7fe0b80c3ee0 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8074570 con 0x7fe0b80c4730 2024-04-26T22:06:42.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.476+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b80c4440 con 0x7fe0b80c5880 2024-04-26T22:06:42.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.477+0000 7fe0bcd8b640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0b80c4730 0x7fe0b8072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:42.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.477+0000 7fe0bdd8d640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0b80c5880 0x7fe0b8201f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:42.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bdd8d640 1 -- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0b80c4730 msgr2=0x7fe0b8072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bdd8d640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0b80c4730 0x7fe0b8072920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bdd8d640 1 -- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 msgr2=0x7fe0b8204eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bdd8d640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8204eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bdd8d640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b8205490 con 0x7fe0b80c5880 2024-04-26T22:06:42.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bdd8d640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0b80c5880 0x7fe0b8201f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c4b0 tx=0x7fe0a800c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:42.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.478+0000 7fe0bd58c640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8204eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:42.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.481+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8017070 con 0x7fe0b80c5880 2024-04-26T22:06:42.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.481+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b8205770 con 0x7fe0b80c5880 2024-04-26T22:06:42.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.481+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b8205d30 con 0x7fe0b80c5880 2024-04-26T22:06:42.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.482+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a80035f0 con 0x7fe0b80c5880 2024-04-26T22:06:42.483 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.482+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80039c0 con 0x7fe0b80c5880 2024-04-26T22:06:42.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.484+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe098005290 con 0x7fe0b80c5880 2024-04-26T22:06:42.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.488+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0a801a070 con 0x7fe0b80c5880 2024-04-26T22:06:42.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.488+0000 7fe0adffb640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe09c06c9c0 0x7fe09c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:42.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.488+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7fe0a808b3c0 con 0x7fe0b80c5880 2024-04-26T22:06:42.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.491+0000 7fe0bd58c640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe09c06c9c0 0x7fe09c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:42.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.492+0000 7fe0bd58c640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe09c06c9c0 0x7fe09c06ee80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400d1a0 tx=0x7fe0b4032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:42.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.499+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0a8058ba0 con 0x7fe0b80c5880 2024-04-26T22:06:42.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.984+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe098005580 con 0x7fe0b80c5880 2024-04-26T22:06:42.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.988+0000 7fe0adffb640 1 -- 172.21.15.135:0/1374093639 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5675 (secure 0 0 0) 0x7fe0a805cea0 con 0x7fe0b80c5880 2024-04-26T22:06:42.989 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:42.989 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:42.989 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 52 2024-04-26T22:06:42.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe09c06c9c0 msgr2=0x7fe09c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe09c06c9c0 0x7fe09c06ee80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400d1a0 tx=0x7fe0b4032000 comp rx=0 tx=0).stop 2024-04-26T22:06:42.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0b80c5880 msgr2=0x7fe0b8201f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:42.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0b80c5880 0x7fe0b8201f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c4b0 tx=0x7fe0a800c970 comp rx=0 tx=0).stop 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 shutdown_connections 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe09c06c9c0 0x7fe09c06ee80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0b80c5880 0x7fe0b8201f70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0b80c4730 0x7fe0b8072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 --2- 172.21.15.135:0/1374093639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0b80c3ee0 0x7fe0b8204eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.991+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 >> 172.21.15.135:0/1374093639 conn(0x7fe0b80c4d60 msgr2=0x7fe0b8166820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.992+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 shutdown_connections 2024-04-26T22:06:42.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:42.992+0000 7fe0bf7f0640 1 -- 172.21.15.135:0/1374093639 wait complete. 2024-04-26T22:06:43.014 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-26T22:06:45.014 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:45.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.190+0000 7f81a5d35640 1 Processor -- start 2024-04-26T22:06:45.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.190+0000 7f81a5d35640 1 -- start start 2024-04-26T22:06:45.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.192+0000 7f81a5d35640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8198006ea0 0x7f8198007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.192+0000 7f81a5d35640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81980d9f40 con 0x7f819804b110 2024-04-26T22:06:45.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.192+0000 7f81a5d35640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81980da220 con 0x7f819800bd50 2024-04-26T22:06:45.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.192+0000 7f81a5d35640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81980da500 con 0x7f8198006ea0 2024-04-26T22:06:45.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.192+0000 7f81a4d33640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f819800bd50 0x7f8198046b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:54048/0 (socket says 172.21.15.135:54048) 2024-04-26T22:06:45.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.192+0000 7f81a4d33640 1 -- 172.21.15.135:0/516490014 learned_addr learned my addr 172.21.15.135:0/516490014 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:45.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.194+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839088533 0 0) 0x7f81980d9f40 con 0x7f819804b110 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8190003520 con 0x7f819804b110 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942395828 0 0) 0x7f81980da220 con 0x7f819800bd50 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81980d9f40 con 0x7f819800bd50 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1884764083 0 0) 0x7f81980d9f40 con 0x7f819800bd50 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8198006ea0 msgr2=0x7f8198007350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8198006ea0 0x7f8198007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.195+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 >> v1:172.21.15.135:6789/0 conn(0x7f819804b110 legacy=0x7f81980d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81980da7e0 con 0x7f819800bd50 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f8194001670 con 0x7f819800bd50 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 >> v1:172.21.15.139:6789/0 conn(0x7f819800bd50 legacy=0x7f8198046b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8190004bd0 0x7f8190004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f8190003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8198006ea0 0x7f8190004210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81980da500 con 0x7f81900037a0 2024-04-26T22:06:45.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81900143c0 con 0x7f8190004bd0 2024-04-26T22:06:45.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81900145c0 con 0x7f8198006ea0 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.196+0000 7f819ffff640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f8190003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.197+0000 7f819ffff640 1 -- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8198006ea0 msgr2=0x7f8190004210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.197+0000 7f819ffff640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8198006ea0 0x7f8190004210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.197+0000 7f819ffff640 1 -- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8190004bd0 msgr2=0x7f8190004fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.197+0000 7f819ffff640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8190004bd0 0x7f8190004fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.197+0000 7f819ffff640 1 -- 172.21.15.135:0/516490014 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81980da7e0 con 0x7f81900037a0 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.198+0000 7f819ffff640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f8190003c20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f818c007f50 tx=0x7f8190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa41f602f80d6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.198+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c007b50 con 0x7f81900037a0 2024-04-26T22:06:45.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.198+0000 7f819f7fe640 1 -- 172.21.15.135:0/516490014 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c02ae50 con 0x7f81900037a0 2024-04-26T22:06:45.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.199+0000 7f81a5d35640 1 -- 172.21.15.135:0/516490014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 msgr2=0x7f8190003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.200+0000 7f81a5d35640 1 --2- 172.21.15.135:0/516490014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f8190003c20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f818c007f50 tx=0x7f8190005830 comp rx=0 tx=0).stop 2024-04-26T22:06:45.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.200+0000 7f81a5534640 1 -- 172.21.15.135:0/516490014 reap_dead start 2024-04-26T22:06:45.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.200+0000 7f81a5d35640 1 -- 172.21.15.135:0/516490014 shutdown_connections 2024-04-26T22:06:45.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.200+0000 7f81a5d35640 1 -- 172.21.15.135:0/516490014 >> 172.21.15.135:0/516490014 conn(0x7f8198048250 msgr2=0x7f8198048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:45.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.202+0000 7f81a5d35640 1 -- 172.21.15.135:0/516490014 shutdown_connections 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.203+0000 7f81a5d35640 1 -- 172.21.15.135:0/516490014 wait complete. 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.204+0000 7f81a5d35640 1 Processor -- start 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.206+0000 7f81a5d35640 1 -- start start 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5d35640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f81980d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5d35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198041d80 0x7f819817c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5d35640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81980d5e10 0x7f819817cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5d35640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8198007350 con 0x7f81900037a0 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5d35640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81980d9930 con 0x7f8198041d80 2024-04-26T22:06:45.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5d35640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81980074d0 con 0x7f81980d5e10 2024-04-26T22:06:45.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a4d33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f81980d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:45.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a4d33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f81980d89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58274/0 (socket says 172.21.15.135:58274) 2024-04-26T22:06:45.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a4d33640 1 -- 172.21.15.135:0/4139868691 learned_addr learned my addr 172.21.15.135:0/4139868691 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:45.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.207+0000 7f81a5534640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81980d5e10 0x7f819817cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:45.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f819ffff640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198041d80 0x7f819817c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:45.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f81a5534640 1 -- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198041d80 msgr2=0x7f819817c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f81a5534640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198041d80 0x7f819817c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f81a5534640 1 -- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 msgr2=0x7f81980d89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f81a5534640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f81980d89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f81a5534640 1 -- 172.21.15.135:0/4139868691 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8198042c20 con 0x7f81980d5e10 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.208+0000 7f81a5534640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81980d5e10 0x7f819817cd60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f81a006acf0 tx=0x7f81a0069f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.209+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a006f640 con 0x7f81980d5e10 2024-04-26T22:06:45.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.209+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a006f7e0 con 0x7f81980d5e10 2024-04-26T22:06:45.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.209+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8198042940 con 0x7f81980d5e10 2024-04-26T22:06:45.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.209+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a006fbd0 con 0x7f81980d5e10 2024-04-26T22:06:45.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.209+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8198180b20 con 0x7f81980d5e10 2024-04-26T22:06:45.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.210+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8198005390 con 0x7f81980d5e10 2024-04-26T22:06:45.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.212+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81a00ad730 con 0x7f81980d5e10 2024-04-26T22:06:45.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.213+0000 7f819d7fa640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f818406c940 0x7f818406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:45.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.213+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f81a01077b0 con 0x7f81980d5e10 2024-04-26T22:06:45.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.213+0000 7f81a4d33640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f818406c940 0x7f818406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:45.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.214+0000 7f81a4d33640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f818406c940 0x7f818406ee00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8194001050 tx=0x7f8194001b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:45.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.223+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81a00d55e0 con 0x7f81980d5e10 2024-04-26T22:06:45.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.688+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8198048030 con 0x7f81980d5e10 2024-04-26T22:06:45.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.691+0000 7f819d7fa640 1 -- 172.21.15.135:0/4139868691 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7f81a00d4ee0 con 0x7f81980d5e10 2024-04-26T22:06:45.695 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:45.695 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:45.696 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:06:45.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.696+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f818406c940 msgr2=0x7f818406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.696+0000 7f81a5d35640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f818406c940 0x7f818406ee00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8194001050 tx=0x7f8194001b20 comp rx=0 tx=0).stop 2024-04-26T22:06:45.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.696+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81980d5e10 msgr2=0x7f819817cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:45.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.696+0000 7f81a5d35640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81980d5e10 0x7f819817cd60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f81a006acf0 tx=0x7f81a0069f90 comp rx=0 tx=0).stop 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 shutdown_connections 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f818406c940 0x7f818406ee00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81980d5e10 0x7f819817cd60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198041d80 0x7f819817c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 --2- 172.21.15.135:0/4139868691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81900037a0 0x7f81980d89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 >> 172.21.15.135:0/4139868691 conn(0x7f8198048250 msgr2=0x7f81980d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 shutdown_connections 2024-04-26T22:06:45.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:45.697+0000 7f81a5d35640 1 -- 172.21.15.135:0/4139868691 wait complete. 2024-04-26T22:06:45.722 INFO:tasks.mds_thrash.fs.[a]:mds.c reported in up:standby state 2024-04-26T22:06:45.723 INFO:tasks.mds_thrash.fs.[a]:waiting for 30 secs before thrashing 2024-04-26T22:06:52.336 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:52.483 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 Processor -- start 2024-04-26T22:06:52.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 -- start start 2024-04-26T22:06:52.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae300ca5a0 0x7fae30156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:52.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae3015a1a0 0x7fae300c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae300c8df0 con 0x7fae30154160 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fae300c9100 con 0x7fae3015a1a0 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.484+0000 7fae35739640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae30154540 con 0x7fae300ca5a0 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae3015a1a0 0x7fae300c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae3015a1a0 0x7fae300c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:52708/0 (socket says 172.21.15.135:52708) 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 -- 172.21.15.135:0/1887742215 learned_addr learned my addr 172.21.15.135:0/1887742215 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2e7fc640 1 --2- 172.21.15.135:0/1887742215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae300ca5a0 0x7fae30156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 -- 172.21.15.135:0/1887742215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae300ca5a0 msgr2=0x7fae30156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 --2- 172.21.15.135:0/1887742215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae300ca5a0 0x7fae30156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 -- 172.21.15.135:0/1887742215 >> v1:172.21.15.139:6789/0 conn(0x7fae30154160 legacy=0x7fae300c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.485+0000 7fae2f7fe640 1 -- 172.21.15.135:0/1887742215 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3015a910 con 0x7fae3015a1a0 2024-04-26T22:06:52.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae2f7fe640 1 --2- 172.21.15.135:0/1887742215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae300c9590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fae28009e50 tx=0x7fae2802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc66d703b5f54e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:52.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae2dffb640 1 -- 172.21.15.135:0/1887742215 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28036070 con 0x7fae3015a1a0 2024-04-26T22:06:52.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae2dffb640 1 -- 172.21.15.135:0/1887742215 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae28031600 con 0x7fae3015a1a0 2024-04-26T22:06:52.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae2dffb640 1 -- 172.21.15.135:0/1887742215 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae280319d0 con 0x7fae3015a1a0 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 -- 172.21.15.135:0/1887742215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 msgr2=0x7fae300c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 --2- 172.21.15.135:0/1887742215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae300c9590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fae28009e50 tx=0x7fae2802a710 comp rx=0 tx=0).stop 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 -- 172.21.15.135:0/1887742215 shutdown_connections 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 --2- 172.21.15.135:0/1887742215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae300c9590 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 --2- 172.21.15.135:0/1887742215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae300ca5a0 0x7fae30156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 -- 172.21.15.135:0/1887742215 >> 172.21.15.135:0/1887742215 conn(0x7fae300ae010 msgr2=0x7fae300ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 -- 172.21.15.135:0/1887742215 shutdown_connections 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.486+0000 7fae35739640 1 -- 172.21.15.135:0/1887742215 wait complete. 2024-04-26T22:06:52.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.488+0000 7fae35739640 1 Processor -- start 2024-04-26T22:06:52.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.489+0000 7fae35739640 1 -- start start 2024-04-26T22:06:52.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae35739640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae300ca5a0 0x7fae301503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:52.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae35739640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 0x7fae301f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:52.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae35739640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae301f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:52.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae35739640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae3015a630 con 0x7fae3015a1a0 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae35739640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae3015aed0 con 0x7fae300ca5a0 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae35739640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae300c7510 con 0x7fae30154160 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.490+0000 7fae2e7fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 0x7fae301f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 0x7fae301f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:35662/0 (socket says 172.21.15.135:35662) 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 -- 172.21.15.135:0/685959376 learned_addr learned my addr 172.21.15.135:0/685959376 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2effd640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae300ca5a0 0x7fae301503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2f7fe640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae301f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 -- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae300ca5a0 msgr2=0x7fae301503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae300ca5a0 0x7fae301503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 -- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 msgr2=0x7fae301f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae301f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.491+0000 7fae2e7fc640 1 -- 172.21.15.135:0/685959376 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae301573c0 con 0x7fae30154160 2024-04-26T22:06:52.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.492+0000 7fae2e7fc640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 0x7fae301f0180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae2400d800 tx=0x7fae2400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:52.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.493+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400b6c0 con 0x7fae30154160 2024-04-26T22:06:52.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.494+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae30154fc0 con 0x7fae30154160 2024-04-26T22:06:52.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.494+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae30150900 con 0x7fae30154160 2024-04-26T22:06:52.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.494+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae24019070 con 0x7fae30154160 2024-04-26T22:06:52.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.495+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400ba80 con 0x7fae30154160 2024-04-26T22:06:52.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.496+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae10005290 con 0x7fae30154160 2024-04-26T22:06:52.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.497+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae2400bc20 con 0x7fae30154160 2024-04-26T22:06:52.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.498+0000 7fae177fe640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae1c06c9c0 0x7fae1c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:52.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.498+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7fae2408c630 con 0x7fae30154160 2024-04-26T22:06:52.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.499+0000 7fae2effd640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae1c06c9c0 0x7fae1c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:52.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.501+0000 7fae2effd640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae1c06c9c0 0x7fae1c06ee80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae20007fd0 tx=0x7fae20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:52.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.510+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae2405a3d0 con 0x7fae30154160 2024-04-26T22:06:52.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.988+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae10005580 con 0x7fae30154160 2024-04-26T22:06:52.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.989+0000 7fae177fe640 1 -- 172.21.15.135:0/685959376 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7fae24059cd0 con 0x7fae30154160 2024-04-26T22:06:52.989 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:52.989 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:52.989 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:06:52.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae1c06c9c0 msgr2=0x7fae1c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:52.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae1c06c9c0 0x7fae1c06ee80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae20007fd0 tx=0x7fae20009040 comp rx=0 tx=0).stop 2024-04-26T22:06:52.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 msgr2=0x7fae301f0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:52.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 0x7fae301f0180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae2400d800 tx=0x7fae2400dcc0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 shutdown_connections 2024-04-26T22:06:52.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae1c06c9c0 0x7fae1c06ee80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3015a1a0 0x7fae301f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae30154160 0x7fae301f0180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 --2- 172.21.15.135:0/685959376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae300ca5a0 0x7fae301503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:52.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 >> 172.21.15.135:0/685959376 conn(0x7fae300ae010 msgr2=0x7fae3014c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:52.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.993+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 shutdown_connections 2024-04-26T22:06:52.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:52.996+0000 7fae35739640 1 -- 172.21.15.135:0/685959376 wait complete. 2024-04-26T22:06:53.016 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-04-26T22:06:53.017 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:06:53.017 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:06:53.037 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:06:53.037 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:53.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.266+0000 7fca6902a640 1 Processor -- start 2024-04-26T22:06:53.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.266+0000 7fca6902a640 1 -- start start 2024-04-26T22:06:53.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.267+0000 7fca6902a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 0x7fca640c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:53.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.268+0000 7fca6902a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca6406f190 con 0x7fca64158460 2024-04-26T22:06:53.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.268+0000 7fca6902a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca640ab570 con 0x7fca6415e3e0 2024-04-26T22:06:53.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.268+0000 7fca6902a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fca640c49e0 con 0x7fca6406fd80 2024-04-26T22:06:53.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.268+0000 7fca637fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 0x7fca640c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:53.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.268+0000 7fca637fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 0x7fca640c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:35678/0 (socket says 172.21.15.135:35678) 2024-04-26T22:06:53.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.268+0000 7fca637fe640 1 -- 172.21.15.135:0/526953931 learned_addr learned my addr 172.21.15.135:0/526953931 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:53.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.269+0000 7fca637fe640 1 -- 172.21.15.135:0/526953931 >> v1:172.21.15.139:6789/0 conn(0x7fca6415e3e0 legacy=0x7fca6415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:53.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.269+0000 7fca637fe640 1 -- 172.21.15.135:0/526953931 >> v1:172.21.15.135:6789/0 conn(0x7fca64158460 legacy=0x7fca6406f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:53.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.269+0000 7fca637fe640 1 -- 172.21.15.135:0/526953931 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6415f260 con 0x7fca6406fd80 2024-04-26T22:06:53.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.270+0000 7fca637fe640 1 --2- 172.21.15.135:0/526953931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 0x7fca640c3b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca540098a0 tx=0x7fca5402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86159f7852dcd5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:53.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.271+0000 7fca62ffd640 1 -- 172.21.15.135:0/526953931 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54037070 con 0x7fca6406fd80 2024-04-26T22:06:53.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.271+0000 7fca62ffd640 1 -- 172.21.15.135:0/526953931 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5402acf0 con 0x7fca6406fd80 2024-04-26T22:06:53.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.272+0000 7fca6902a640 1 -- 172.21.15.135:0/526953931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 msgr2=0x7fca640c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:53.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.272+0000 7fca6902a640 1 --2- 172.21.15.135:0/526953931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 0x7fca640c3b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca540098a0 tx=0x7fca5402a060 comp rx=0 tx=0).stop 2024-04-26T22:06:53.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.273+0000 7fca6902a640 1 -- 172.21.15.135:0/526953931 shutdown_connections 2024-04-26T22:06:53.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.273+0000 7fca6902a640 1 --2- 172.21.15.135:0/526953931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca6406fd80 0x7fca640c3b90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.273+0000 7fca6902a640 1 -- 172.21.15.135:0/526953931 >> 172.21.15.135:0/526953931 conn(0x7fca640c5090 msgr2=0x7fca640c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:53.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.274+0000 7fca6902a640 1 -- 172.21.15.135:0/526953931 shutdown_connections 2024-04-26T22:06:53.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.278+0000 7fca6902a640 1 -- 172.21.15.135:0/526953931 wait complete. 2024-04-26T22:06:53.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.280+0000 7fca6902a640 1 Processor -- start 2024-04-26T22:06:53.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.280+0000 7fca6902a640 1 -- start start 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca6902a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca6406fd80 0x7fca641fc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca6902a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca64158460 0x7fca64157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca6902a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 0x7fca641579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca6902a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca6415ef50 con 0x7fca6415e3e0 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca6902a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fca640c4700 con 0x7fca6406fd80 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca6902a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fca64158840 con 0x7fca64158460 2024-04-26T22:06:53.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca68829640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 0x7fca641579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca68829640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 0x7fca641579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:52722/0 (socket says 172.21.15.135:52722) 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.281+0000 7fca68829640 1 -- 172.21.15.135:0/556501253 learned_addr learned my addr 172.21.15.135:0/556501253 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.282+0000 7fca68829640 1 -- 172.21.15.135:0/556501253 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca64158460 msgr2=0x7fca64157470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.282+0000 7fca68829640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca64158460 0x7fca64157470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.282+0000 7fca68829640 1 -- 172.21.15.135:0/556501253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca6406fd80 msgr2=0x7fca641fc790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.282+0000 7fca68829640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca6406fd80 0x7fca641fc790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.282+0000 7fca68829640 1 -- 172.21.15.135:0/556501253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca640728a0 con 0x7fca6415e3e0 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.282+0000 7fca68829640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 0x7fca641579b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00c780 tx=0x7fca5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.283+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c017070 con 0x7fca6415e3e0 2024-04-26T22:06:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.283+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c003da0 con 0x7fca6415e3e0 2024-04-26T22:06:53.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.283+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0052b0 con 0x7fca6415e3e0 2024-04-26T22:06:53.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.285+0000 7fca6902a640 1 -- 172.21.15.135:0/556501253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641fd000 con 0x7fca6415e3e0 2024-04-26T22:06:53.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.286+0000 7fca6902a640 1 -- 172.21.15.135:0/556501253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca64072b20 con 0x7fca6415e3e0 2024-04-26T22:06:53.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.289+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca5c01a070 con 0x7fca6415e3e0 2024-04-26T22:06:53.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.290+0000 7fca60ff9640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca5006c8f0 0x7fca5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:53.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.290+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7fca5c08b8b0 con 0x7fca6415e3e0 2024-04-26T22:06:53.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.291+0000 7fca63fff640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca5006c8f0 0x7fca5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:53.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.292+0000 7fca63fff640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca5006c8f0 0x7fca5006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca58001640 tx=0x7fca580079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:53.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.292+0000 7fca6902a640 1 -- 172.21.15.135:0/556501253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca44005290 con 0x7fca6415e3e0 2024-04-26T22:06:53.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.301+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca640728a0 con 0x7fca6415e3e0 2024-04-26T22:06:53.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.768+0000 7fca6902a640 1 -- 172.21.15.135:0/556501253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca44005580 con 0x7fca6415e3e0 2024-04-26T22:06:53.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.771+0000 7fca60ff9640 1 -- 172.21.15.135:0/556501253 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7fca5c059660 con 0x7fca6415e3e0 2024-04-26T22:06:53.771 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:53.771 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:53.772 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 -- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca5006c8f0 msgr2=0x7fca5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca5006c8f0 0x7fca5006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca58001640 tx=0x7fca580079b0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 -- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 msgr2=0x7fca641579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 0x7fca641579b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00c780 tx=0x7fca5c00cc40 comp rx=0 tx=0).stop 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 -- 172.21.15.135:0/556501253 shutdown_connections 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fca5006c8f0 0x7fca5006edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca6415e3e0 0x7fca641579b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca64158460 0x7fca64157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 --2- 172.21.15.135:0/556501253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca6406fd80 0x7fca641fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:53.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.775+0000 7fca4a7fc640 1 -- 172.21.15.135:0/556501253 >> 172.21.15.135:0/556501253 conn(0x7fca640c5090 msgr2=0x7fca6415e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:53.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.777+0000 7fca4a7fc640 1 -- 172.21.15.135:0/556501253 shutdown_connections 2024-04-26T22:06:53.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:53.777+0000 7fca4a7fc640 1 -- 172.21.15.135:0/556501253 wait complete. 2024-04-26T22:06:53.798 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:06:55.798 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.986+0000 7f47d815f640 1 Processor -- start 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.986+0000 7f47d815f640 1 -- start start 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.987+0000 7f47d815f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f47d00c3650 0x7f47d006e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.987+0000 7f47d815f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47d00abf50 0x7f47d00ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.987+0000 7f47d815f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d015c0d0 con 0x7f47d00aa8c0 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.987+0000 7f47d815f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f47d00ab4b0 con 0x7f47d00c3650 2024-04-26T22:06:55.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.987+0000 7f47d815f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f47d015bdf0 con 0x7f47d00abf50 2024-04-26T22:06:55.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.989+0000 7f47d56fa640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f47d00aa8c0 0x7f47d00c3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:50686/0 (socket says 172.21.15.135:50686) 2024-04-26T22:06:55.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.989+0000 7f47d56fa640 1 -- 172.21.15.135:0/2737916174 learned_addr learned my addr 172.21.15.135:0/2737916174 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:55.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.990+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2237888928 0 0) 0x7f47d015c0d0 con 0x7f47d00aa8c0 2024-04-26T22:06:55.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.990+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47bc003520 con 0x7f47d00aa8c0 2024-04-26T22:06:55.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.991+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3622523295 0 0) 0x7f47bc003520 con 0x7f47d00aa8c0 2024-04-26T22:06:55.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.991+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 >> v2:172.21.15.139:3300/0 conn(0x7f47d00abf50 msgr2=0x7f47d00ac3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:55.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.991+0000 7f47d4ef9640 1 --2- 172.21.15.135:0/2737916174 >> v2:172.21.15.139:3300/0 conn(0x7f47d00abf50 0x7f47d00ac3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:55.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.991+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 >> v2:172.21.15.135:3300/0 conn(0x7f47d00c3650 msgr2=0x7f47d006e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:55.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d5efb640 1 --2- 172.21.15.135:0/2737916174 >> v2:172.21.15.135:3300/0 conn(0x7f47d00c3650 0x7f47d006e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d4ef9640 1 --2- 172.21.15.135:0/2737916174 >> v2:172.21.15.135:3300/0 conn(0x7f47d00c3650 0x7f47d006e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00ab750 con 0x7f47d00aa8c0 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f47c0001670 con 0x7f47d00aa8c0 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d5efb640 1 --2- 172.21.15.135:0/2737916174 >> v2:172.21.15.135:3300/0 conn(0x7f47d00c3650 0x7f47d006e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 >> v1:172.21.15.139:6789/0 conn(0x7f47d00aa8c0 legacy=0x7f47d00c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.992+0000 7f47d4ef9640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47bc003a00 0x7f47bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d4ef9640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d4ef9640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47bc00bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00ab4b0 con 0x7f47bc003a00 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47d015bdf0 con 0x7f47bc004800 2024-04-26T22:06:55.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc00c410 con 0x7f47bc0099b0 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d66fc640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47bc00bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d56fa640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.993+0000 7f47d5efb640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47bc003a00 0x7f47bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.994+0000 7f47d66fc640 1 -- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc004800 msgr2=0x7f47bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.994+0000 7f47d66fc640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.994+0000 7f47d66fc640 1 -- 172.21.15.135:0/2737916174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47bc003a00 msgr2=0x7f47bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.994+0000 7f47d66fc640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47bc003a00 0x7f47bc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:55.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.994+0000 7f47d66fc640 1 -- 172.21.15.135:0/2737916174 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc003520 con 0x7f47bc0099b0 2024-04-26T22:06:55.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.994+0000 7f47d5efb640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47bc003a00 0x7f47bc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:55.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.995+0000 7f47d66fc640 1 -- 172.21.15.135:0/2737916174 reap_dead start 2024-04-26T22:06:55.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.995+0000 7f47d56fa640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:55.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.995+0000 7f47d66fc640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47bc00bda0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009860 tx=0x7f47bc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2329f152a0d232b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:55.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.995+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc037070 con 0x7f47bc0099b0 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.995+0000 7f47d4ef9640 1 -- 172.21.15.135:0/2737916174 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc02aa30 con 0x7f47bc0099b0 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.996+0000 7f47d815f640 1 -- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 msgr2=0x7f47bc00bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.996+0000 7f47d815f640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47bc00bda0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009860 tx=0x7f47bc005830 comp rx=0 tx=0).stop 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.997+0000 7f47d815f640 1 -- 172.21.15.135:0/2737916174 shutdown_connections 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.997+0000 7f47d815f640 1 --2- 172.21.15.135:0/2737916174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47bc00bda0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.997+0000 7f47d815f640 1 -- 172.21.15.135:0/2737916174 >> 172.21.15.135:0/2737916174 conn(0x7f47d00c82d0 msgr2=0x7f47d00c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.997+0000 7f47d815f640 1 -- 172.21.15.135:0/2737916174 shutdown_connections 2024-04-26T22:06:55.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.997+0000 7f47d815f640 1 -- 172.21.15.135:0/2737916174 wait complete. 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.999+0000 7f47d815f640 1 Processor -- start 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:55.999+0000 7f47d815f640 1 -- start start 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.000+0000 7f47d815f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47d01453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.000+0000 7f47d815f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47d0158ee0 0x7f47d006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.000+0000 7f47d815f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47d013e3e0 0x7f47d013e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.000+0000 7f47d815f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00ac540 con 0x7f47d0158ee0 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.000+0000 7f47d815f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00ac3c0 con 0x7f47d013e3e0 2024-04-26T22:06:56.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.000+0000 7f47d815f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47d00ac6c0 con 0x7f47bc0099b0 2024-04-26T22:06:56.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.001+0000 7f47d5efb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47d01453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:56.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.001+0000 7f47d5efb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47d01453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:35700/0 (socket says 172.21.15.135:35700) 2024-04-26T22:06:56.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.001+0000 7f47d5efb640 1 -- 172.21.15.135:0/3288756728 learned_addr learned my addr 172.21.15.135:0/3288756728 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:56.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.001+0000 7f47d66fc640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47d013e3e0 0x7f47d013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:56.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.002+0000 7f47d56fa640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47d0158ee0 0x7f47d006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:56.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d56fa640 1 -- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 msgr2=0x7f47d01453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:56.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d56fa640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47d01453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:56.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d56fa640 1 -- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47d013e3e0 msgr2=0x7f47d013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:56.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d56fa640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47d013e3e0 0x7f47d013e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:56.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d56fa640 1 -- 172.21.15.135:0/3288756728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d0074560 con 0x7f47d0158ee0 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d56fa640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47d0158ee0 0x7f47d006e9d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47d00ab140 tx=0x7f47c000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d5efb640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47d01453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.003+0000 7f47d66fc640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47d013e3e0 0x7f47d013e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.007+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c000c630 con 0x7f47d0158ee0 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.007+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c0018070 con 0x7f47d0158ee0 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.007+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c000c9d0 con 0x7f47d0158ee0 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.007+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d0074840 con 0x7f47d0158ee0 2024-04-26T22:06:56.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.007+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d0073cd0 con 0x7f47d0158ee0 2024-04-26T22:06:56.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.011+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b8005290 con 0x7f47d0158ee0 2024-04-26T22:06:56.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.014+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47c000cb70 con 0x7f47d0158ee0 2024-04-26T22:06:56.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.015+0000 7f47c67fc640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b406c9f0 0x7f47b406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:56.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.015+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f47c008bbb0 con 0x7f47d0158ee0 2024-04-26T22:06:56.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.017+0000 7f47d5efb640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b406c9f0 0x7f47b406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:56.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.017+0000 7f47d5efb640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b406c9f0 0x7f47b406eeb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f47d015f370 tx=0x7f47c800f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:56.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.020+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47c0059390 con 0x7f47d0158ee0 2024-04-26T22:06:56.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.509+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b8005580 con 0x7f47d0158ee0 2024-04-26T22:06:56.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.510+0000 7f47c67fc640 1 -- 172.21.15.135:0/3288756728 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7f47c005d690 con 0x7f47d0158ee0 2024-04-26T22:06:56.514 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:56.514 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:56.514 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b406c9f0 msgr2=0x7f47b406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b406c9f0 0x7f47b406eeb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f47d015f370 tx=0x7f47c800f040 comp rx=0 tx=0).stop 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47d0158ee0 msgr2=0x7f47d006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47d0158ee0 0x7f47d006e9d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47d00ab140 tx=0x7f47c000ec10 comp rx=0 tx=0).stop 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 shutdown_connections 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f47b406c9f0 0x7f47b406eeb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47d013e3e0 0x7f47d013e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47d0158ee0 0x7f47d006e9d0 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47d00ab140 tx=0x7f47c000ec10 comp rx=0 tx=0).stop 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 --2- 172.21.15.135:0/3288756728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47bc0099b0 0x7f47d01453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:56.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 >> 172.21.15.135:0/3288756728 conn(0x7f47d00c82d0 msgr2=0x7f47d00c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:56.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 shutdown_connections 2024-04-26T22:06:56.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:56.518+0000 7f47d815f640 1 -- 172.21.15.135:0/3288756728 wait complete. 2024-04-26T22:06:56.538 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:06:58.538 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:58.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.728+0000 7f269359e640 1 Processor -- start 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.729+0000 7f269359e640 1 -- start start 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.730+0000 7f269359e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f269406d470 0x7f26940ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.730+0000 7f269359e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269406d8a0 con 0x7f2694150220 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.730+0000 7f269359e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26941537a0 con 0x7f269406db10 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.730+0000 7f269359e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f26940c2bd0 con 0x7f269406d470 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.730+0000 7f269259c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2694150220 0x7f269406cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35550/0 (socket says 172.21.15.135:35550) 2024-04-26T22:06:58.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.730+0000 7f269259c640 1 -- 172.21.15.135:0/1530075498 learned_addr learned my addr 172.21.15.135:0/1530075498 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:06:58.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.731+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2503247254 0 0) 0x7f269406d8a0 con 0x7f2694150220 2024-04-26T22:06:58.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.731+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2680003520 con 0x7f2694150220 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.732+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3785547286 0 0) 0x7f2680003520 con 0x7f2694150220 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.732+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 >> v2:172.21.15.139:3300/0 conn(0x7f269406d470 msgr2=0x7f26940ac4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.732+0000 7f2691d9b640 1 --2- 172.21.15.135:0/1530075498 >> v2:172.21.15.139:3300/0 conn(0x7f269406d470 0x7f26940ac4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.732+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 >> v1:172.21.15.139:6789/0 conn(0x7f269406db10 legacy=0x7f26940abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.732+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2694153a80 con 0x7f2694150220 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.733+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2684001670 con 0x7f2694150220 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.733+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 >> v1:172.21.15.135:6789/0 conn(0x7f2694150220 legacy=0x7f269406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:58.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.733+0000 7f2691d9b640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f2680003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.734+0000 7f269259c640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f2680003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:58.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.734+0000 7f2691d9b640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2680004800 0x7f2680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.734+0000 7f2691d9b640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26800053f0 0x7f268000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.734+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f268000c310 con 0x7f2680003a00 2024-04-26T22:06:58.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.734+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f268000c4c0 con 0x7f2680004800 2024-04-26T22:06:58.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.734+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f268000c6c0 con 0x7f26800053f0 2024-04-26T22:06:58.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.735+0000 7f269259c640 1 -- 172.21.15.135:0/1530075498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26800053f0 msgr2=0x7f268000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:58.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.735+0000 7f269259c640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26800053f0 0x7f268000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:58.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.735+0000 7f269259c640 1 -- 172.21.15.135:0/1530075498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2680004800 msgr2=0x7f2680004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:58.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.735+0000 7f269259c640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2680004800 0x7f2680004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:58.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.735+0000 7f269259c640 1 -- 172.21.15.135:0/1530075498 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2680003520 con 0x7f2680003a00 2024-04-26T22:06:58.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.736+0000 7f269259c640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f2680003de0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f268400f490 tx=0x7f2680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e1fddd6c3d0adc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:58.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.736+0000 7f2692d9d640 1 -- 172.21.15.135:0/1530075498 reap_dead start 2024-04-26T22:06:58.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.737+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268400d040 con 0x7f2680003a00 2024-04-26T22:06:58.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.737+0000 7f2691d9b640 1 -- 172.21.15.135:0/1530075498 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2684005bb0 con 0x7f2680003a00 2024-04-26T22:06:58.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.738+0000 7f269359e640 1 -- 172.21.15.135:0/1530075498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 msgr2=0x7f2680003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:58.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.738+0000 7f269359e640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f2680003de0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f268400f490 tx=0x7f2680005830 comp rx=0 tx=0).stop 2024-04-26T22:06:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.742+0000 7f269359e640 1 -- 172.21.15.135:0/1530075498 shutdown_connections 2024-04-26T22:06:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.742+0000 7f269359e640 1 --2- 172.21.15.135:0/1530075498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f2680003de0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.742+0000 7f269359e640 1 -- 172.21.15.135:0/1530075498 >> 172.21.15.135:0/1530075498 conn(0x7f26940c4cd0 msgr2=0x7f26940c50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:58.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.756+0000 7f269359e640 1 -- 172.21.15.135:0/1530075498 shutdown_connections 2024-04-26T22:06:58.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.757+0000 7f269359e640 1 -- 172.21.15.135:0/1530075498 wait complete. 2024-04-26T22:06:58.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.766+0000 7f269359e640 1 Processor -- start 2024-04-26T22:06:58.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.766+0000 7f269359e640 1 -- start start 2024-04-26T22:06:58.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.767+0000 7f269359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f26941e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.767+0000 7f269359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2694156760 0x7f2694155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.767+0000 7f269359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f269414dff0 0x7f26941eb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.768+0000 7f269359e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26940ab8b0 con 0x7f2680003a00 2024-04-26T22:06:58.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.768+0000 7f269359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26940ab380 con 0x7f269414dff0 2024-04-26T22:06:58.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.768+0000 7f269359e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2694150600 con 0x7f2694156760 2024-04-26T22:06:58.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.768+0000 7f269259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f26941e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:58.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.768+0000 7f269259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f26941e8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60836/0 (socket says 172.21.15.135:60836) 2024-04-26T22:06:58.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.768+0000 7f269259c640 1 -- 172.21.15.135:0/2347119524 learned_addr learned my addr 172.21.15.135:0/2347119524 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:06:58.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.769+0000 7f269259c640 1 -- 172.21.15.135:0/2347119524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2694156760 msgr2=0x7f2694155020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:58.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.769+0000 7f269259c640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2694156760 0x7f2694155020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:58.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.769+0000 7f269259c640 1 -- 172.21.15.135:0/2347119524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f269414dff0 msgr2=0x7f26941eb450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:06:58.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.769+0000 7f269259c640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f269414dff0 0x7f26941eb450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:58.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.769+0000 7f269259c640 1 -- 172.21.15.135:0/2347119524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2694155560 con 0x7f2680003a00 2024-04-26T22:06:58.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.770+0000 7f269259c640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f26941e8c00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f268400fa30 tx=0x7f269406c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:58.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.771+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268400d000 con 0x7f2680003a00 2024-04-26T22:06:58.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.771+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2684001890 con 0x7f2680003a00 2024-04-26T22:06:58.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.771+0000 7f269359e640 1 -- 172.21.15.135:0/2347119524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2694154050 con 0x7f2680003a00 2024-04-26T22:06:58.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.774+0000 7f269359e640 1 -- 172.21.15.135:0/2347119524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2694154610 con 0x7f2680003a00 2024-04-26T22:06:58.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.777+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840015b0 con 0x7f2680003a00 2024-04-26T22:06:58.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.778+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2684008330 con 0x7f2680003a00 2024-04-26T22:06:58.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.778+0000 7f268affd640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f267406ca20 0x7f267406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:58.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.778+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f26840ab830 con 0x7f2680003a00 2024-04-26T22:06:58.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.779+0000 7f268bfff640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f267406ca20 0x7f267406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:58.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.779+0000 7f268bfff640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f267406ca20 0x7f267406eee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2694074b40 tx=0x7f267c0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:58.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.781+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2678005290 con 0x7f2680003a00 2024-04-26T22:06:58.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:58.793+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2684079780 con 0x7f2680003a00 2024-04-26T22:06:59.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.265+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2678005850 con 0x7f2680003a00 2024-04-26T22:06:59.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.266+0000 7f268affd640 1 -- 172.21.15.135:0/2347119524 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7f2684079080 con 0x7f2680003a00 2024-04-26T22:06:59.269 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:06:59.270 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:06:59.277 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:06:59.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.280+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f267406ca20 msgr2=0x7f267406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.280+0000 7f2688ff9640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f267406ca20 0x7f267406eee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2694074b40 tx=0x7f267c0079d0 comp rx=0 tx=0).stop 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.280+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 msgr2=0x7f26941e8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.280+0000 7f2688ff9640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f26941e8c00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f268400fa30 tx=0x7f269406c6a0 comp rx=0 tx=0).stop 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.281+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 shutdown_connections 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.281+0000 7f2688ff9640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f267406ca20 0x7f267406eee0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.281+0000 7f2688ff9640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f269414dff0 0x7f26941eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.281+0000 7f2688ff9640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2694156760 0x7f2694155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.281+0000 7f2688ff9640 1 --2- 172.21.15.135:0/2347119524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2680003a00 0x7f26941e8c00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:59.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.281+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 >> 172.21.15.135:0/2347119524 conn(0x7f26940c4cd0 msgr2=0x7f269406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:59.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.282+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 shutdown_connections 2024-04-26T22:06:59.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:06:59.282+0000 7f2688ff9640 1 -- 172.21.15.135:0/2347119524 wait complete. 2024-04-26T22:06:59.304 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:07:00.330 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:07:00.336 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:07:01.305 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:01.490 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.489+0000 7f2b6ec47640 1 Processor -- start 2024-04-26T22:07:01.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.492+0000 7f2b6ec47640 1 -- start start 2024-04-26T22:07:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.494+0000 7f2b6ec47640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2b6815e170 0x7f2b6815e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.494+0000 7f2b6ec47640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2b680c3750 0x7f2b6806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.494+0000 7f2b6ec47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6815edd0 con 0x7f2b680abfc0 2024-04-26T22:07:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.494+0000 7f2b6ec47640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6815eba0 con 0x7f2b6815e170 2024-04-26T22:07:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.494+0000 7f2b6ec47640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6806e3d0 con 0x7f2b680c3750 2024-04-26T22:07:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2b6815e170 0x7f2b6815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2b6815e170 0x7f2b6815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60844/0 (socket says 172.21.15.135:60844) 2024-04-26T22:07:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 -- 172.21.15.135:0/3168546081 learned_addr learned my addr 172.21.15.135:0/3168546081 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b6d1e4640 1 --2- 172.21.15.135:0/3168546081 >> v2:172.21.15.139:3300/0 conn(0x7f2b680c3750 0x7f2b6806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:01.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 -- 172.21.15.135:0/3168546081 >> v2:172.21.15.139:3300/0 conn(0x7f2b680c3750 msgr2=0x7f2b6806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:01.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 --2- 172.21.15.135:0/3168546081 >> v2:172.21.15.139:3300/0 conn(0x7f2b680c3750 0x7f2b6806eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 -- 172.21.15.135:0/3168546081 >> v1:172.21.15.139:6789/0 conn(0x7f2b680abfc0 legacy=0x7f2b6815da60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:01.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.495+0000 7f2b67fff640 1 -- 172.21.15.135:0/3168546081 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6815f0b0 con 0x7f2b6815e170 2024-04-26T22:07:01.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.497+0000 7f2b67fff640 1 --2- 172.21.15.135:0/3168546081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 0x7f2b6815e660 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2b68071690 tx=0x7f2b5002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172c8fd000795b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:01.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.497+0000 7f2b677fe640 1 -- 172.21.15.135:0/3168546081 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50037070 con 0x7f2b6815e170 2024-04-26T22:07:01.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.497+0000 7f2b677fe640 1 -- 172.21.15.135:0/3168546081 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b50032440 con 0x7f2b6815e170 2024-04-26T22:07:01.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.497+0000 7f2b677fe640 1 -- 172.21.15.135:0/3168546081 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50032810 con 0x7f2b6815e170 2024-04-26T22:07:01.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.498+0000 7f2b6ec47640 1 -- 172.21.15.135:0/3168546081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 msgr2=0x7f2b6815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:01.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.498+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/3168546081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 0x7f2b6815e660 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2b68071690 tx=0x7f2b5002a230 comp rx=0 tx=0).stop 2024-04-26T22:07:01.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.501+0000 7f2b6ec47640 1 -- 172.21.15.135:0/3168546081 shutdown_connections 2024-04-26T22:07:01.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.501+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/3168546081 >> v2:172.21.15.139:3300/0 conn(0x7f2b680c3750 0x7f2b6806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.501+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/3168546081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 0x7f2b6815e660 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.501+0000 7f2b6ec47640 1 -- 172.21.15.135:0/3168546081 >> 172.21.15.135:0/3168546081 conn(0x7f2b6806c310 msgr2=0x7f2b6806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:01.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.503+0000 7f2b6ec47640 1 -- 172.21.15.135:0/3168546081 shutdown_connections 2024-04-26T22:07:01.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.504+0000 7f2b6ec47640 1 -- 172.21.15.135:0/3168546081 wait complete. 2024-04-26T22:07:01.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.505+0000 7f2b6ec47640 1 Processor -- start 2024-04-26T22:07:01.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.505+0000 7f2b6ec47640 1 -- start start 2024-04-26T22:07:01.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.507+0000 7f2b6ec47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 0x7f2b681568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:01.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.507+0000 7f2b6ec47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b680c3750 0x7f2b68071840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:01.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.507+0000 7f2b6ec47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 0x7f2b68071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.507+0000 7f2b6ec47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6815c560 con 0x7f2b6815e170 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.507+0000 7f2b6ec47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6806e620 con 0x7f2b680abfc0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.507+0000 7f2b6ec47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2b680ac3a0 con 0x7f2b680c3750 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.508+0000 7f2b6c9e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 0x7f2b681568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.508+0000 7f2b6c9e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 0x7f2b681568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:44588/0 (socket says 172.21.15.135:44588) 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.508+0000 7f2b6c9e3640 1 -- 172.21.15.135:0/2340188138 learned_addr learned my addr 172.21.15.135:0/2340188138 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.509+0000 7f2b6c9e3640 1 -- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b680c3750 msgr2=0x7f2b68071840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.509+0000 7f2b6c9e3640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b680c3750 0x7f2b68071840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.509+0000 7f2b6c9e3640 1 -- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 msgr2=0x7f2b68071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.509+0000 7f2b6c9e3640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 0x7f2b68071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.509+0000 7f2b6c9e3640 1 -- 172.21.15.135:0/2340188138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b681536f0 con 0x7f2b680abfc0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.510+0000 7f2b6c9e3640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 0x7f2b681568f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000b450 tx=0x7f2b6000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.510+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60015020 con 0x7f2b680abfc0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.511+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b60011890 con 0x7f2b680abfc0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.511+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60011c80 con 0x7f2b680abfc0 2024-04-26T22:07:01.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.511+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b68154510 con 0x7f2b680abfc0 2024-04-26T22:07:01.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.511+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b68153990 con 0x7f2b680abfc0 2024-04-26T22:07:01.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.512+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c005290 con 0x7f2b680abfc0 2024-04-26T22:07:01.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.515+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b60018020 con 0x7f2b680abfc0 2024-04-26T22:07:01.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.515+0000 7f2b657fa640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2b5406cb00 0x7f2b5406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:01.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.516+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f2b6008c320 con 0x7f2b680abfc0 2024-04-26T22:07:01.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.516+0000 7f2b67fff640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2b5406cb00 0x7f2b5406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:01.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.517+0000 7f2b67fff640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2b5406cb00 0x7f2b5406efc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2b50034000 tx=0x7f2b5002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:01.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.524+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b6001f020 con 0x7f2b680abfc0 2024-04-26T22:07:01.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.973+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b4c005580 con 0x7f2b680abfc0 2024-04-26T22:07:01.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.974+0000 7f2b657fa640 1 -- 172.21.15.135:0/2340188138 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7f2b6000d040 con 0x7f2b680abfc0 2024-04-26T22:07:01.975 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:01.975 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:07:01.975 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:07:01.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2b5406cb00 msgr2=0x7f2b5406efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:01.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2b5406cb00 0x7f2b5406efc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2b50034000 tx=0x7f2b5002e000 comp rx=0 tx=0).stop 2024-04-26T22:07:01.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 msgr2=0x7f2b681568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 0x7f2b681568f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000b450 tx=0x7f2b6000b910 comp rx=0 tx=0).stop 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 shutdown_connections 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2b5406cb00 0x7f2b5406efc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b6815e170 0x7f2b68071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b680c3750 0x7f2b68071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 --2- 172.21.15.135:0/2340188138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b680abfc0 0x7f2b681568f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.978+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 >> 172.21.15.135:0/2340188138 conn(0x7f2b6806c310 msgr2=0x7f2b6806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.979+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 shutdown_connections 2024-04-26T22:07:01.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:01.979+0000 7f2b6ec47640 1 -- 172.21.15.135:0/2340188138 wait complete. 2024-04-26T22:07:02.000 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:07:04.001 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:04.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.182+0000 7fb257c20640 1 Processor -- start 2024-04-26T22:07:04.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.182+0000 7fb257c20640 1 -- start start 2024-04-26T22:07:04.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb257c20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2501555a0 0x7fb250160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:04.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb2559bc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb25014ecd0 0x7fb2500ab470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35558/0 (socket says 172.21.15.135:35558) 2024-04-26T22:07:04.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb2559bc640 1 -- 172.21.15.135:0/1098637230 learned_addr learned my addr 172.21.15.135:0/1098637230 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:04.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25014f0b0 con 0x7fb25014ecd0 2024-04-26T22:07:04.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb250074c90 con 0x7fb2500ac710 2024-04-26T22:07:04.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb2500c6400 con 0x7fb2501555a0 2024-04-26T22:07:04.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.190+0000 7fb2561bd640 1 --2- 172.21.15.135:0/1098637230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2501555a0 0x7fb250160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:04.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.192+0000 7fb2561bd640 1 -- 172.21.15.135:0/1098637230 >> v1:172.21.15.139:6789/0 conn(0x7fb2500ac710 legacy=0x7fb250073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.192+0000 7fb2561bd640 1 -- 172.21.15.135:0/1098637230 >> v1:172.21.15.135:6789/0 conn(0x7fb25014ecd0 legacy=0x7fb2500ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.192+0000 7fb2561bd640 1 -- 172.21.15.135:0/1098637230 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2500c66e0 con 0x7fb2501555a0 2024-04-26T22:07:04.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.194+0000 7fb2561bd640 1 --2- 172.21.15.135:0/1098637230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2501555a0 0x7fb250160340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0097d0 tx=0x7fb24c02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=328c6c5ef80ca79c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:04.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.199+0000 7fb2549ba640 1 -- 172.21.15.135:0/1098637230 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c037070 con 0x7fb2501555a0 2024-04-26T22:07:04.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.199+0000 7fb2549ba640 1 -- 172.21.15.135:0/1098637230 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c02ad10 con 0x7fb2501555a0 2024-04-26T22:07:04.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.200+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2501555a0 msgr2=0x7fb250160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.200+0000 7fb257c20640 1 --2- 172.21.15.135:0/1098637230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2501555a0 0x7fb250160340 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0097d0 tx=0x7fb24c02a0a0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.204+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 shutdown_connections 2024-04-26T22:07:04.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.204+0000 7fb257c20640 1 --2- 172.21.15.135:0/1098637230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2501555a0 0x7fb250160340 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.204+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 >> 172.21.15.135:0/1098637230 conn(0x7fb2500c6d80 msgr2=0x7fb2500c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:04.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.205+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 shutdown_connections 2024-04-26T22:07:04.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.206+0000 7fb257c20640 1 -- 172.21.15.135:0/1098637230 wait complete. 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.208+0000 7fb257c20640 1 Processor -- start 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.208+0000 7fb257c20640 1 -- start start 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb257c20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 0x7fb25006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb257c20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25014ecd0 0x7fb250138090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb257c20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2501555a0 0x7fb2501385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb257c20640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb250155080 con 0x7fb2501555a0 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb257c20640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb250154da0 con 0x7fb25014ecd0 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb257c20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb2500c69c0 con 0x7fb2500ac710 2024-04-26T22:07:04.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb2559bc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 0x7fb25006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:04.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb2559bc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 0x7fb25006d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:39920/0 (socket says 172.21.15.135:39920) 2024-04-26T22:07:04.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb2559bc640 1 -- 172.21.15.135:0/3995117538 learned_addr learned my addr 172.21.15.135:0/3995117538 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb2551bb640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25014ecd0 0x7fb250138090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.209+0000 7fb2561bd640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2501555a0 0x7fb2501385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2559bc640 1 -- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25014ecd0 msgr2=0x7fb250138090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2559bc640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25014ecd0 0x7fb250138090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2559bc640 1 -- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2501555a0 msgr2=0x7fb2501385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2559bc640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2501555a0 0x7fb2501385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2559bc640 1 -- 172.21.15.135:0/3995117538 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb250135be0 con 0x7fb2500ac710 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2561bd640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2501555a0 0x7fb2501385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2559bc640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 0x7fb25006d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb248002140 tx=0x7fb24800ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248015a80 con 0x7fb2500ac710 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.210+0000 7fb257c20640 1 -- 172.21.15.135:0/3995117538 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2500af290 con 0x7fb2500ac710 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.211+0000 7fb257c20640 1 -- 172.21.15.135:0/3995117538 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb250135e80 con 0x7fb2500ac710 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.211+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248019030 con 0x7fb2500ac710 2024-04-26T22:07:04.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.211+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248024060 con 0x7fb2500ac710 2024-04-26T22:07:04.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.214+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250076650 con 0x7fb2500ac710 2024-04-26T22:07:04.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.215+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb24800a610 con 0x7fb2500ac710 2024-04-26T22:07:04.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.215+0000 7fb2467fc640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb23406c920 0x7fb23406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:04.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.217+0000 7fb2551bb640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb23406c920 0x7fb23406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:04.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.217+0000 7fb2551bb640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb23406c920 0x7fb23406ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb250154a00 tx=0x7fb240009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:04.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.218+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7fb24808bb50 con 0x7fb2500ac710 2024-04-26T22:07:04.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.224+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb248015e20 con 0x7fb2500ac710 2024-04-26T22:07:04.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.686+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2500c7570 con 0x7fb2500ac710 2024-04-26T22:07:04.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.687+0000 7fb2467fc640 1 -- 172.21.15.135:0/3995117538 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7fb248059190 con 0x7fb2500ac710 2024-04-26T22:07:04.688 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:04.688 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:07:04.688 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb23406c920 msgr2=0x7fb23406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb23406c920 0x7fb23406ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb250154a00 tx=0x7fb240009040 comp rx=0 tx=0).stop 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 msgr2=0x7fb25006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 0x7fb25006d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb248002140 tx=0x7fb24800ce90 comp rx=0 tx=0).stop 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 shutdown_connections 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb23406c920 0x7fb23406ede0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2501555a0 0x7fb2501385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25014ecd0 0x7fb250138090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 --2- 172.21.15.135:0/3995117538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2500ac710 0x7fb25006d360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 >> 172.21.15.135:0/3995117538 conn(0x7fb2500c6d80 msgr2=0x7fb250153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.693+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 shutdown_connections 2024-04-26T22:07:04.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:04.694+0000 7fb233fff640 1 -- 172.21.15.135:0/3995117538 wait complete. 2024-04-26T22:07:04.713 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:07:06.713 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:06.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.854+0000 7f7431aa9640 1 Processor -- start 2024-04-26T22:07:06.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.855+0000 7f7431aa9640 1 -- start start 2024-04-26T22:07:06.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.859+0000 7f7431aa9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f742c150200 0x7f742c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:06.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.859+0000 7f7431aa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0c2cd0 0x7f742c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:06.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.859+0000 7f7431aa9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742c06ff20 con 0x7f742c0ac680 2024-04-26T22:07:06.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.859+0000 7f7431aa9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f742c0c4bd0 con 0x7f742c150200 2024-04-26T22:07:06.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.859+0000 7f7431aa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f742c06f4e0 con 0x7f742c0c2cd0 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.861+0000 7f742b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f742c150200 0x7f742c0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.861+0000 7f742b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f742c150200 0x7f742c0ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60876/0 (socket says 172.21.15.135:60876) 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.861+0000 7f742b7fe640 1 -- 172.21.15.135:0/4164810310 learned_addr learned my addr 172.21.15.135:0/4164810310 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.861+0000 7f742affd640 1 --2- 172.21.15.135:0/4164810310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0c2cd0 0x7f742c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.862+0000 7f742affd640 1 -- 172.21.15.135:0/4164810310 >> v2:172.21.15.135:3300/0 conn(0x7f742c150200 msgr2=0x7f742c0ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.862+0000 7f742affd640 1 --2- 172.21.15.135:0/4164810310 >> v2:172.21.15.135:3300/0 conn(0x7f742c150200 0x7f742c0ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.862+0000 7f742affd640 1 -- 172.21.15.135:0/4164810310 >> v1:172.21.15.139:6789/0 conn(0x7f742c0ac680 legacy=0x7f742c156210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:06.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.862+0000 7f742affd640 1 -- 172.21.15.135:0/4164810310 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c0c48f0 con 0x7f742c0c2cd0 2024-04-26T22:07:06.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.864+0000 7f742affd640 1 --2- 172.21.15.135:0/4164810310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0c2cd0 0x7f742c1533b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f741c00dbc0 tx=0x7f741c029fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885af54175cdc0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:06.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.867+0000 7f742a7fc640 1 -- 172.21.15.135:0/4164810310 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c02aac0 con 0x7f742c0c2cd0 2024-04-26T22:07:06.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.867+0000 7f742a7fc640 1 -- 172.21.15.135:0/4164810310 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c02ac60 con 0x7f742c0c2cd0 2024-04-26T22:07:06.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.867+0000 7f742a7fc640 1 -- 172.21.15.135:0/4164810310 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c00b670 con 0x7f742c0c2cd0 2024-04-26T22:07:06.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.869+0000 7f7431aa9640 1 -- 172.21.15.135:0/4164810310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0c2cd0 msgr2=0x7f742c1533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:06.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.869+0000 7f7431aa9640 1 --2- 172.21.15.135:0/4164810310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0c2cd0 0x7f742c1533b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f741c00dbc0 tx=0x7f741c029fc0 comp rx=0 tx=0).stop 2024-04-26T22:07:06.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.870+0000 7f7431aa9640 1 -- 172.21.15.135:0/4164810310 shutdown_connections 2024-04-26T22:07:06.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.870+0000 7f7431aa9640 1 --2- 172.21.15.135:0/4164810310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0c2cd0 0x7f742c1533b0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f741c00dbc0 tx=0x7f741c029fc0 comp rx=0 tx=0).stop 2024-04-26T22:07:06.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.870+0000 7f7431aa9640 1 --2- 172.21.15.135:0/4164810310 >> v2:172.21.15.135:3300/0 conn(0x7f742c150200 0x7f742c0ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:06.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.870+0000 7f7431aa9640 1 -- 172.21.15.135:0/4164810310 >> 172.21.15.135:0/4164810310 conn(0x7f742c0c82d0 msgr2=0x7f742c0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:06.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.873+0000 7f7431aa9640 1 -- 172.21.15.135:0/4164810310 shutdown_connections 2024-04-26T22:07:06.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.873+0000 7f7431aa9640 1 -- 172.21.15.135:0/4164810310 wait complete. 2024-04-26T22:07:06.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.876+0000 7f7431aa9640 1 Processor -- start 2024-04-26T22:07:06.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.876+0000 7f7431aa9640 1 -- start start 2024-04-26T22:07:06.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.877+0000 7f7431aa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0ac680 0x7f742c14cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.877+0000 7f7431aa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f742c0c2cd0 0x7f742c1e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.877+0000 7f7431aa9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 0x7f742c1e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.877+0000 7f7431aa9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f742c150760 con 0x7f742c150200 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.877+0000 7f7431aa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1505e0 con 0x7f742c0c2cd0 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.877+0000 7f7431aa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f742c0c8dc0 con 0x7f742c0ac680 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 0x7f742c1e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 0x7f742c1e5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60884/0 (socket says 172.21.15.135:60884) 2024-04-26T22:07:06.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 -- 172.21.15.135:0/3939916070 learned_addr learned my addr 172.21.15.135:0/3939916070 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:06.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742b7fe640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0ac680 0x7f742c14cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:06.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742affd640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f742c0c2cd0 0x7f742c1e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:06.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 -- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0ac680 msgr2=0x7f742c14cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:06.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0ac680 0x7f742c14cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:06.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 -- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f742c0c2cd0 msgr2=0x7f742c1e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:06.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f742c0c2cd0 0x7f742c1e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.878+0000 7f742bfff640 1 -- 172.21.15.135:0/3939916070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c14afe0 con 0x7f742c150200 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.879+0000 7f742bfff640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 0x7f742c1e5c00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f742000b420 tx=0x7f742000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.879+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420011830 con 0x7f742c150200 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.879+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420016070 con 0x7f742c150200 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.879+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420011830 con 0x7f742c150200 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.880+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c14b2c0 con 0x7f742c150200 2024-04-26T22:07:06.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.881+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c146550 con 0x7f742c150200 2024-04-26T22:07:06.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.883+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74200119d0 con 0x7f742c150200 2024-04-26T22:07:06.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.884+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c147a80 con 0x7f742c150200 2024-04-26T22:07:06.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.884+0000 7f7413fff640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f741806ccf0 0x7f741806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:06.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.884+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 7354+0+0 (secure 0 0 0) 0x7f7420011630 con 0x7f742c150200 2024-04-26T22:07:06.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.884+0000 7f742b7fe640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f741806ccf0 0x7f741806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:06.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.885+0000 7f742b7fe640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f741806ccf0 0x7f741806f1b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f742c157290 tx=0x7f7424016040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:06.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:06.893+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f742c147a80 con 0x7f742c150200 2024-04-26T22:07:07.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.315+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f742c0aca90 con 0x7f742c150200 2024-04-26T22:07:07.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.316+0000 7f7413fff640 1 -- 172.21.15.135:0/3939916070 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+6503 (secure 0 0 0) 0x7f742c147a80 con 0x7f742c150200 2024-04-26T22:07:07.318 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:07.318 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5419,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:05:57.871105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5610":{"gid":5610,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":20,"addr":"172.21.15.135:6837/2165063360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":2165063360},{"type":"v1","addr":"172.21.15.135:6837","nonce":2165063360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5610,"qdb_cluster":[5610]},"id":2}]} 2024-04-26T22:07:07.318 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 53 2024-04-26T22:07:07.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f741806ccf0 msgr2=0x7f741806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:07.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f741806ccf0 0x7f741806f1b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f742c157290 tx=0x7f7424016040 comp rx=0 tx=0).stop 2024-04-26T22:07:07.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 msgr2=0x7f742c1e5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:07.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 0x7f742c1e5c00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f742000b420 tx=0x7f742000b8e0 comp rx=0 tx=0).stop 2024-04-26T22:07:07.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 shutdown_connections 2024-04-26T22:07:07.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f741806ccf0 0x7f741806f1b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:07.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f742c150200 0x7f742c1e5c00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:07.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f742c0c2cd0 0x7f742c1e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:07.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 --2- 172.21.15.135:0/3939916070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f742c0ac680 0x7f742c14cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:07.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 >> 172.21.15.135:0/3939916070 conn(0x7f742c0c82d0 msgr2=0x7f742c0aad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:07.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.329+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 shutdown_connections 2024-04-26T22:07:07.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:07.330+0000 7f7431aa9640 1 -- 172.21.15.135:0/3939916070 wait complete. 2024-04-26T22:07:07.355 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:07:09.356 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:09.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.551+0000 7f7494893640 1 Processor -- start 2024-04-26T22:07:09.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.552+0000 7f7494893640 1 -- start start 2024-04-26T22:07:09.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.553+0000 7f7494893640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74901507c0 0x7f74900ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:09.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.553+0000 7f7494893640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490153d90 con 0x7f749006e640 2024-04-26T22:07:09.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.553+0000 7f7494893640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74900c2e50 con 0x7f749006f760 2024-04-26T22:07:09.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.553+0000 7f7494893640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f74900c3130 con 0x7f74901507c0 2024-04-26T22:07:09.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74901507c0 0x7f74900ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:09.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74901507c0 0x7f74900ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56422/0 (socket says 172.21.15.135:56422) 2024-04-26T22:07:09.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748e59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f749006e640 0x7f74900c3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38758/0 (socket says 172.21.15.135:38758) 2024-04-26T22:07:09.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748ed9d640 1 -- 172.21.15.135:0/1750366298 learned_addr learned my addr 172.21.15.135:0/1750366298 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:09.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748d59a640 1 -- 172.21.15.135:0/1750366298 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441318095 0 0) 0x7f74900c2e50 con 0x7f749006f760 2024-04-26T22:07:09.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748d59a640 1 -- 172.21.15.135:0/1750366298 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c003520 con 0x7f749006f760 2024-04-26T22:07:09.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748ed9d640 1 -- 172.21.15.135:0/1750366298 >> v1:172.21.15.139:6789/0 conn(0x7f749006f760 legacy=0x7f74901500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:09.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748ed9d640 1 -- 172.21.15.135:0/1750366298 >> v1:172.21.15.135:6789/0 conn(0x7f749006e640 legacy=0x7f74900c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:09.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.554+0000 7f748ed9d640 1 -- 172.21.15.135:0/1750366298 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74900c3410 con 0x7f74901507c0 2024-04-26T22:07:09.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.556+0000 7f748ed9d640 1 --2- 172.21.15.135:0/1750366298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74901507c0 0x7f74900ab380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f748800d150 tx=0x7f748802c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bba808c3bb11a256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:09.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.557+0000 7f748e59c640 1 --1- 172.21.15.135:0/1750366298 >> v1:172.21.15.135:6789/0 conn(0x7f749006e640 0x7f74900c3930 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T22:07:09.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.558+0000 7f748d59a640 1 -- 172.21.15.135:0/1750366298 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488009600 con 0x7f74901507c0 2024-04-26T22:07:09.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.558+0000 7f748d59a640 1 -- 172.21.15.135:0/1750366298 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74880097a0 con 0x7f74901507c0 2024-04-26T22:07:09.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.559+0000 7f7494893640 1 -- 172.21.15.135:0/1750366298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74901507c0 msgr2=0x7f74900ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:09.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.559+0000 7f7494893640 1 --2- 172.21.15.135:0/1750366298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74901507c0 0x7f74900ab380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f748800d150 tx=0x7f748802c900 comp rx=0 tx=0).stop 2024-04-26T22:07:09.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.560+0000 7f7494893640 1 -- 172.21.15.135:0/1750366298 shutdown_connections 2024-04-26T22:07:09.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.560+0000 7f7494893640 1 --2- 172.21.15.135:0/1750366298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74901507c0 0x7f74900ab380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:09.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.560+0000 7f7494893640 1 -- 172.21.15.135:0/1750366298 >> 172.21.15.135:0/1750366298 conn(0x7f749006c310 msgr2=0x7f749006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:09.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.562+0000 7f7494893640 1 -- 172.21.15.135:0/1750366298 shutdown_connections 2024-04-26T22:07:09.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.562+0000 7f7494893640 1 -- 172.21.15.135:0/1750366298 wait complete. 2024-04-26T22:07:09.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.564+0000 7f7494893640 1 Processor -- start 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.565+0000 7f7494893640 1 -- start start 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f7494893640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f749006e640 0x7f74900c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f7494893640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 0x7f74900be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f7494893640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74901f11a0 0x7f74900bed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f7494893640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f749006eba0 con 0x7f74901f11a0 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f7494893640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f749006ea20 con 0x7f749006f760 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f7494893640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f74900c4720 con 0x7f749006e640 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f748dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 0x7f74900be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f748dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 0x7f74900be7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56436/0 (socket says 172.21.15.135:56436) 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.566+0000 7f748dd9b640 1 -- 172.21.15.135:0/3969027203 learned_addr learned my addr 172.21.15.135:0/3969027203 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:09.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748ed9d640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74901f11a0 0x7f74900bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748dd9b640 1 -- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f749006e640 msgr2=0x7f74900c8800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748dd9b640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f749006e640 0x7f74900c8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748dd9b640 1 -- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74901f11a0 msgr2=0x7f74900bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748dd9b640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74901f11a0 0x7f74900bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748dd9b640 1 -- 172.21.15.135:0/3969027203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74900c79e0 con 0x7f749006f760 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.567+0000 7f748ed9d640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74901f11a0 0x7f74900bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:09.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.568+0000 7f748dd9b640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 0x7f74900be7e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f748000c3c0 tx=0x7f748000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:09.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.570+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480017070 con 0x7f749006f760 2024-04-26T22:07:09.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.570+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74900c2050 con 0x7f749006f760 2024-04-26T22:07:09.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.570+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901f09a0 con 0x7f749006f760 2024-04-26T22:07:09.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.571+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74800052d0 con 0x7f749006f760 2024-04-26T22:07:09.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.571+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480011890 con 0x7f749006f760 2024-04-26T22:07:09.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.574+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7474005290 con 0x7f749006f760 2024-04-26T22:07:09.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.574+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f748001a070 con 0x7f749006f760 2024-04-26T22:07:09.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.575+0000 7f7486ffd640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747006c9f0 0x7f747006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:09.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.575+0000 7f748e59c640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747006c9f0 0x7f747006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:09.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.575+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f748008b3a0 con 0x7f749006f760 2024-04-26T22:07:09.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.576+0000 7f748e59c640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747006c9f0 0x7f747006eeb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7478001530 tx=0x7f7478007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:09.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:09.583+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7480058b50 con 0x7f749006f760 2024-04-26T22:07:10.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.055+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7474005580 con 0x7f749006f760 2024-04-26T22:07:10.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.057+0000 7f7486ffd640 1 -- 172.21.15.135:0/3969027203 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5667 (secure 0 0 0) 0x7f748005ce50 con 0x7f749006f760 2024-04-26T22:07:10.060 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:10.061 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:07.831073+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:10.062 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 54 2024-04-26T22:07:10.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747006c9f0 msgr2=0x7f747006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:10.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747006c9f0 0x7f747006eeb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7478001530 tx=0x7f7478007a80 comp rx=0 tx=0).stop 2024-04-26T22:07:10.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 msgr2=0x7f74900be7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:10.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 0x7f74900be7e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f748000c3c0 tx=0x7f748000c880 comp rx=0 tx=0).stop 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 shutdown_connections 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f747006c9f0 0x7f747006eeb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74901f11a0 0x7f74900bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f749006f760 0x7f74900be7e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 --2- 172.21.15.135:0/3969027203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f749006e640 0x7f74900c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.061+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 >> 172.21.15.135:0/3969027203 conn(0x7f749006c310 msgr2=0x7f74900ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.062+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 shutdown_connections 2024-04-26T22:07:10.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.062+0000 7f7494893640 1 -- 172.21.15.135:0/3969027203 wait complete. 2024-04-26T22:07:10.084 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-04-26T22:07:10.084 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:07:10.084 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:10.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.304+0000 7ff921a9b640 1 Processor -- start 2024-04-26T22:07:10.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.306+0000 7ff921a9b640 1 -- start start 2024-04-26T22:07:10.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.310+0000 7ff921a9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff91c150200 0x7ff91c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:10.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.310+0000 7ff921a9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:10.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.310+0000 7ff921a9b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c06ff20 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.310+0000 7ff921a9b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff91c0c4bd0 con 0x7ff91c150200 2024-04-26T22:07:10.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.311+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:10.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.311+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:41534/0 (socket says 172.21.15.135:41534) 2024-04-26T22:07:10.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.311+0000 7ff91bfff640 1 -- 172.21.15.135:0/217838229 learned_addr learned my addr 172.21.15.135:0/217838229 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:10.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.311+0000 7ff921a9b640 1 -- 172.21.15.135:0/217838229 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c06f4e0 con 0x7ff91c0ac680 2024-04-26T22:07:10.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.312+0000 7ff91bfff640 1 -- 172.21.15.135:0/217838229 >> v2:172.21.15.139:3300/0 conn(0x7ff91c150200 msgr2=0x7ff91c0ac140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:10.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.312+0000 7ff91bfff640 1 --2- 172.21.15.135:0/217838229 >> v2:172.21.15.139:3300/0 conn(0x7ff91c150200 0x7ff91c0ac140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.312+0000 7ff91bfff640 1 -- 172.21.15.135:0/217838229 >> v1:172.21.15.139:6789/0 conn(0x7ff91c0c2cd0 legacy=0x7ff91c1533b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:10.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.312+0000 7ff91bfff640 1 -- 172.21.15.135:0/217838229 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c0c48f0 con 0x7ff91c0ac680 2024-04-26T22:07:10.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.313+0000 7ff91bfff640 1 --2- 172.21.15.135:0/217838229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c15d590 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff914009e50 tx=0x7ff91402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9715c183ce52e2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:10.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.314+0000 7ff91a7fc640 1 -- 172.21.15.135:0/217838229 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914036070 con 0x7ff91c0ac680 2024-04-26T22:07:10.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.314+0000 7ff91a7fc640 1 -- 172.21.15.135:0/217838229 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff914031600 con 0x7ff91c0ac680 2024-04-26T22:07:10.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.315+0000 7ff91a7fc640 1 -- 172.21.15.135:0/217838229 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140319d0 con 0x7ff91c0ac680 2024-04-26T22:07:10.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.315+0000 7ff921a9b640 1 -- 172.21.15.135:0/217838229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 msgr2=0x7ff91c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:10.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.315+0000 7ff921a9b640 1 --2- 172.21.15.135:0/217838229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c15d590 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff914009e50 tx=0x7ff91402a660 comp rx=0 tx=0).stop 2024-04-26T22:07:10.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.319+0000 7ff921a9b640 1 -- 172.21.15.135:0/217838229 shutdown_connections 2024-04-26T22:07:10.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.319+0000 7ff921a9b640 1 --2- 172.21.15.135:0/217838229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c15d590 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.319+0000 7ff921a9b640 1 --2- 172.21.15.135:0/217838229 >> v2:172.21.15.139:3300/0 conn(0x7ff91c150200 0x7ff91c0ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.319+0000 7ff921a9b640 1 -- 172.21.15.135:0/217838229 >> 172.21.15.135:0/217838229 conn(0x7ff91c0c82d0 msgr2=0x7ff91c0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:10.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.321+0000 7ff921a9b640 1 -- 172.21.15.135:0/217838229 shutdown_connections 2024-04-26T22:07:10.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.324+0000 7ff921a9b640 1 -- 172.21.15.135:0/217838229 wait complete. 2024-04-26T22:07:10.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.326+0000 7ff921a9b640 1 Processor -- start 2024-04-26T22:07:10.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.328+0000 7ff921a9b640 1 -- start start 2024-04-26T22:07:10.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.329+0000 7ff921a9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c1ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:10.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.329+0000 7ff921a9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 0x7ff91c1e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:10.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff91affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 0x7ff91c1e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:10.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff91affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 0x7ff91c1e8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:35052/0 (socket says 172.21.15.135:35052) 2024-04-26T22:07:10.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff91affd640 1 -- 172.21.15.135:0/1452361453 learned_addr learned my addr 172.21.15.135:0/1452361453 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:10.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff91b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c1ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:10.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c150200 0x7ff91c148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:10.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c150760 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1505e0 con 0x7ff91c150200 2024-04-26T22:07:10.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.330+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c153ac0 con 0x7ff91c0ac680 2024-04-26T22:07:10.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.331+0000 7ff91affd640 1 -- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 msgr2=0x7ff91c1ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:10.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.331+0000 7ff91affd640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c1ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.331+0000 7ff91affd640 1 -- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c150200 msgr2=0x7ff91c148860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:07:10.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.332+0000 7ff91affd640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c150200 0x7ff91c148860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.332+0000 7ff91affd640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c1e91f0 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.332+0000 7ff91b7fe640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c1ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:10.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.333+0000 7ff91affd640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 0x7ff91c1e8cb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c900 tx=0x7ff91000cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:10.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.334+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910008be0 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.334+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff910008d80 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.334+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c147f90 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.334+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1eb8c0 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.334+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91000a310 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.337+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff91000a4b0 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.337+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc005290 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.350+0000 7ff903fff640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff90806c9c0 0x7ff90806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:10.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.350+0000 7ff91b7fe640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff90806c9c0 0x7ff90806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:10.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.350+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7ff91008c860 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.352+0000 7ff91b7fe640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff90806c9c0 0x7ff90806ee80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff90c006fd0 tx=0x7ff90c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:10.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.353+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff91c1e91f0 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.788+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8fc005580 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.790+0000 7ff903fff640 1 -- 172.21.15.135:0/1452361453 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5667 (secure 0 0 0) 0x7ff91005a660 con 0x7ff91c0c2cd0 2024-04-26T22:07:10.790 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:10.790 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:07.831073+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:10.790 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 54 2024-04-26T22:07:10.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff90806c9c0 msgr2=0x7ff90806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:10.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff90806c9c0 0x7ff90806ee80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff90c006fd0 tx=0x7ff90c008040 comp rx=0 tx=0).stop 2024-04-26T22:07:10.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 msgr2=0x7ff91c1e8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:10.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 0x7ff91c1e8cb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c900 tx=0x7ff91000cdc0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 shutdown_connections 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff90806c9c0 0x7ff90806ee80 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff90c006fd0 tx=0x7ff90c008040 comp rx=0 tx=0).stop 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c150200 0x7ff91c148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff91c0c2cd0 0x7ff91c1e8cb0 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c900 tx=0x7ff91000cdc0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 --2- 172.21.15.135:0/1452361453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff91c0ac680 0x7ff91c1ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.794+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 >> 172.21.15.135:0/1452361453 conn(0x7ff91c0c82d0 msgr2=0x7ff91c14ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.795+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 shutdown_connections 2024-04-26T22:07:10.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:10.795+0000 7ff921a9b640 1 -- 172.21.15.135:0/1452361453 wait complete. 2024-04-26T22:07:10.827 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 54, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 52, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5105": { "addr": "172.21.15.139:6835/3733666635", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 3733666635, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 3733666635, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5105, "incarnation": 49, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:06:34.102512+0000", "qdb_cluster": [ 5105 ], "qdb_leader": 5105, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5105 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 54, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5419": { "addr": "172.21.15.135:6835/1037298395", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 1037298395, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 1037298395, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5419, "incarnation": 54, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:07:07.831073+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5419 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:07:12.828 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:13.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.030+0000 7ff0ff025640 1 Processor -- start 2024-04-26T22:07:13.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.034+0000 7ff0ff025640 1 -- start start 2024-04-26T22:07:13.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0ff025640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f8154b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0ff025640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff0f8071cc0 0x7ff0f8073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0ff025640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0f80707e0 con 0x7ff0f8073c40 2024-04-26T22:07:13.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0ff025640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f8155450 con 0x7ff0f8071cc0 2024-04-26T22:07:13.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0ff025640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f8155730 con 0x7ff0f813ddd0 2024-04-26T22:07:13.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0f7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff0f8071cc0 0x7ff0f8073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0f7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff0f8071cc0 0x7ff0f8073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:35062/0 (socket says 172.21.15.135:35062) 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.035+0000 7ff0f7fff640 1 -- 172.21.15.135:0/3586536070 learned_addr learned my addr 172.21.15.135:0/3586536070 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.036+0000 7ff0fcdc1640 1 --2- 172.21.15.135:0/3586536070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f8154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.036+0000 7ff0f7fff640 1 -- 172.21.15.135:0/3586536070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 msgr2=0x7ff0f8154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.036+0000 7ff0f7fff640 1 --2- 172.21.15.135:0/3586536070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f8154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.036+0000 7ff0f7fff640 1 -- 172.21.15.135:0/3586536070 >> v1:172.21.15.135:6789/0 conn(0x7ff0f8073c40 legacy=0x7ff0f8070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.036+0000 7ff0f7fff640 1 -- 172.21.15.135:0/3586536070 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f8155a10 con 0x7ff0f8071cc0 2024-04-26T22:07:13.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.037+0000 7ff0f7fff640 1 --2- 172.21.15.135:0/3586536070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8071cc0 0x7ff0f8073700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009870 tx=0x7ff0e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e10013601469ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.038+0000 7ff0f77fe640 1 -- 172.21.15.135:0/3586536070 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8037070 con 0x7ff0f8071cc0 2024-04-26T22:07:13.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.038+0000 7ff0f77fe640 1 -- 172.21.15.135:0/3586536070 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e8032440 con 0x7ff0f8071cc0 2024-04-26T22:07:13.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.038+0000 7ff0f77fe640 1 -- 172.21.15.135:0/3586536070 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8032850 con 0x7ff0f8071cc0 2024-04-26T22:07:13.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.040+0000 7ff0ff025640 1 -- 172.21.15.135:0/3586536070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8071cc0 msgr2=0x7ff0f8073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.040+0000 7ff0ff025640 1 --2- 172.21.15.135:0/3586536070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8071cc0 0x7ff0f8073700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009870 tx=0x7ff0e802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.045+0000 7ff0ff025640 1 -- 172.21.15.135:0/3586536070 shutdown_connections 2024-04-26T22:07:13.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.045+0000 7ff0ff025640 1 --2- 172.21.15.135:0/3586536070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8071cc0 0x7ff0f8073700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.045+0000 7ff0ff025640 1 --2- 172.21.15.135:0/3586536070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.045+0000 7ff0ff025640 1 -- 172.21.15.135:0/3586536070 >> 172.21.15.135:0/3586536070 conn(0x7ff0f80ca2f0 msgr2=0x7ff0f80ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.047+0000 7ff0ff025640 1 -- 172.21.15.135:0/3586536070 shutdown_connections 2024-04-26T22:07:13.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.050+0000 7ff0ff025640 1 -- 172.21.15.135:0/3586536070 wait complete. 2024-04-26T22:07:13.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.052+0000 7ff0ff025640 1 Processor -- start 2024-04-26T22:07:13.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.052+0000 7ff0ff025640 1 -- start start 2024-04-26T22:07:13.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.054+0000 7ff0ff025640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f8071cc0 0x7ff0f806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.054+0000 7ff0ff025640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8073c40 0x7ff0f813b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.054+0000 7ff0fcdc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f8071cc0 0x7ff0f806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.054+0000 7ff0fcdc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f8071cc0 0x7ff0f806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56450/0 (socket says 172.21.15.135:56450) 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.054+0000 7ff0ff025640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f813bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.054+0000 7ff0fcdc1640 1 -- 172.21.15.135:0/1437083998 learned_addr learned my addr 172.21.15.135:0/1437083998 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.055+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81551c0 con 0x7ff0f8073c40 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.055+0000 7ff0fd5c2640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f813bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.055+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8155040 con 0x7ff0f8071cc0 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.055+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f80ab470 con 0x7ff0f813ddd0 2024-04-26T22:07:13.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.055+0000 7ff0f7fff640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8073c40 0x7ff0f813b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.056+0000 7ff0f7fff640 1 -- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 msgr2=0x7ff0f813bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.057+0000 7ff0f7fff640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f813bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.057+0000 7ff0f7fff640 1 -- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f8071cc0 msgr2=0x7ff0f806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.057+0000 7ff0f7fff640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f8071cc0 0x7ff0f806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.057+0000 7ff0f7fff640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f80acf70 con 0x7ff0f8073c40 2024-04-26T22:07:13.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.057+0000 7ff0f7fff640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8073c40 0x7ff0f813b5f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8031000 tx=0x7ff0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.057+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8041020 con 0x7ff0f8073c40 2024-04-26T22:07:13.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.058+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e80307c0 con 0x7ff0f8073c40 2024-04-26T22:07:13.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.058+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f8155ef0 con 0x7ff0f8073c40 2024-04-26T22:07:13.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.058+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f8151590 con 0x7ff0f8073c40 2024-04-26T22:07:13.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.058+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8037040 con 0x7ff0f8073c40 2024-04-26T22:07:13.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.061+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0e8044020 con 0x7ff0f8073c40 2024-04-26T22:07:13.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.062+0000 7ff0f57fa640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff0d806c940 0x7ff0d806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.062+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7ff0e8078130 con 0x7ff0f8073c40 2024-04-26T22:07:13.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.063+0000 7ff0fcdc1640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff0d806c940 0x7ff0d806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.064+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc005290 con 0x7ff0f8073c40 2024-04-26T22:07:13.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.067+0000 7ff0fcdc1640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff0d806c940 0x7ff0d806ee00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.073+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0e804b030 con 0x7ff0f8073c40 2024-04-26T22:07:13.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.541+0000 7ff0ff025640 1 -- 172.21.15.135:0/1437083998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0dc005850 con 0x7ff0f8073c40 2024-04-26T22:07:13.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.543+0000 7ff0f57fa640 1 -- 172.21.15.135:0/1437083998 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5670 (secure 0 0 0) 0x7ff0e804b030 con 0x7ff0f8073c40 2024-04-26T22:07:13.544 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:13.545 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:12.410548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:13.545 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 55 2024-04-26T22:07:13.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.549+0000 7ff0eeffd640 1 -- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff0d806c940 msgr2=0x7ff0d806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.549+0000 7ff0eeffd640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff0d806c940 0x7ff0d806ee00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0008040 comp rx=0 tx=0).stop 2024-04-26T22:07:13.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.549+0000 7ff0eeffd640 1 -- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8073c40 msgr2=0x7ff0f813b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8073c40 0x7ff0f813b5f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8031000 tx=0x7ff0e80046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 -- 172.21.15.135:0/1437083998 shutdown_connections 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff0d806c940 0x7ff0d806ee00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0f813ddd0 0x7ff0f813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f8073c40 0x7ff0f813b5f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 --2- 172.21.15.135:0/1437083998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f8071cc0 0x7ff0f806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 -- 172.21.15.135:0/1437083998 >> 172.21.15.135:0/1437083998 conn(0x7ff0f80ca2f0 msgr2=0x7ff0f813c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 -- 172.21.15.135:0/1437083998 shutdown_connections 2024-04-26T22:07:13.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:13.550+0000 7ff0eeffd640 1 -- 172.21.15.135:0/1437083998 wait complete. 2024-04-26T22:07:13.569 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:07:15.570 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:15.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.754+0000 7f9a53378640 1 Processor -- start 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.755+0000 7f9a53378640 1 -- start start 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.756+0000 7f9a53378640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9a440d7810 0x7f9a440da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.756+0000 7f9a53378640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a4400a4e0 0x7f9a4400a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.756+0000 7f9a53378640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a440da950 con 0x7f9a440496d0 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.756+0000 7f9a53378640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9a440dac30 con 0x7f9a440d7810 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.756+0000 7f9a53378640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a440daf10 con 0x7f9a4400a4e0 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.758+0000 7f9a51b75640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9a440496d0 0x7f9a44006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:41810/0 (socket says 172.21.15.135:41810) 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.758+0000 7f9a51b75640 1 -- 172.21.15.135:0/3883563637 learned_addr learned my addr 172.21.15.135:0/3883563637 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.758+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720401177 0 0) 0x7f9a440da950 con 0x7f9a440496d0 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.758+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a38003520 con 0x7f9a440496d0 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4016958931 0 0) 0x7f9a38003520 con 0x7f9a440496d0 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a4400a4e0 msgr2=0x7f9a4400a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a4400a4e0 0x7f9a4400a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 >> v2:172.21.15.135:3300/0 conn(0x7f9a440d7810 msgr2=0x7f9a440da320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 --2- 172.21.15.135:0/3883563637 >> v2:172.21.15.135:3300/0 conn(0x7f9a440d7810 0x7f9a440da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a440db1f0 con 0x7f9a440496d0 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f9a4c050be0 con 0x7f9a440496d0 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 >> v1:172.21.15.139:6789/0 conn(0x7f9a440496d0 legacy=0x7f9a44006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a51374640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a4400a4e0 0x7f9a3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440dac30 con 0x7f9a380039b0 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440daf10 con 0x7f9a38004800 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380055f0 con 0x7f9a4400a4e0 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.759+0000 7f9a52376640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51b75640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51b75640 1 -- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a4400a4e0 msgr2=0x7f9a3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51b75640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a4400a4e0 0x7f9a3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51b75640 1 -- 172.21.15.135:0/3883563637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a380039b0 msgr2=0x7f9a38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51b75640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a380039b0 0x7f9a38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.760+0000 7f9a51b75640 1 -- 172.21.15.135:0/3883563637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a38003520 con 0x7f9a38004800 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.762+0000 7f9a51b75640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a38004c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9a44047d40 tx=0x7f9a38005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e389d28d4ca1b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.763+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0b3070 con 0x7f9a38004800 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.763+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0ae620 con 0x7f9a38004800 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.763+0000 7f9a51374640 1 -- 172.21.15.135:0/3883563637 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c050f10 con 0x7f9a38004800 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.764+0000 7f9a53378640 1 -- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 msgr2=0x7f9a38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.764+0000 7f9a53378640 1 --2- 172.21.15.135:0/3883563637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a38004c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9a44047d40 tx=0x7f9a38005830 comp rx=0 tx=0).stop 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.764+0000 7f9a52b77640 1 -- 172.21.15.135:0/3883563637 reap_dead start 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.765+0000 7f9a53378640 1 -- 172.21.15.135:0/3883563637 shutdown_connections 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.765+0000 7f9a53378640 1 -- 172.21.15.135:0/3883563637 >> 172.21.15.135:0/3883563637 conn(0x7f9a44048250 msgr2=0x7f9a44048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.765+0000 7f9a53378640 1 -- 172.21.15.135:0/3883563637 shutdown_connections 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.765+0000 7f9a53378640 1 -- 172.21.15.135:0/3883563637 wait complete. 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.767+0000 7f9a53378640 1 Processor -- start 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.767+0000 7f9a53378640 1 -- start start 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.768+0000 7f9a53378640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a44167310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.768+0000 7f9a53378640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a440d2880 0x7f9a440ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.769+0000 7f9a53378640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a440d2d00 0x7f9a440cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.769+0000 7f9a53378640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44049df0 con 0x7f9a440d2d00 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.769+0000 7f9a51b75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a440d2880 0x7f9a440ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.769+0000 7f9a51b75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a440d2880 0x7f9a440ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:41586/0 (socket says 172.21.15.135:41586) 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.769+0000 7f9a51b75640 1 -- 172.21.15.135:0/3579142557 learned_addr learned my addr 172.21.15.135:0/3579142557 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.769+0000 7f9a52376640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a44167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.770+0000 7f9a52b77640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a440d2d00 0x7f9a440cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.770+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4400ba20 con 0x7f9a38004800 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.770+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a440075b0 con 0x7f9a440d2880 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.771+0000 7f9a52b77640 1 -- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a440d2880 msgr2=0x7f9a440ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.771+0000 7f9a52b77640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a440d2880 0x7f9a440ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.771+0000 7f9a52b77640 1 -- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 msgr2=0x7f9a44167310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.771+0000 7f9a52b77640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a44167310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.771+0000 7f9a52b77640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44173440 con 0x7f9a440d2d00 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.771+0000 7f9a52376640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a44167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.772+0000 7f9a52b77640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a440d2d00 0x7f9a440cec40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a44049350 tx=0x7f9a48001550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.772+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48005af0 con 0x7f9a440d2d00 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.772+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a48005c90 con 0x7f9a440d2d00 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.772+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441736b0 con 0x7f9a440d2d00 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.775+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a44167850 con 0x7f9a440d2d00 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.776+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48007310 con 0x7f9a440d2d00 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.776+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a480074b0 con 0x7f9a440d2d00 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.777+0000 7f9a42ffd640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a3006c9f0 0x7f9a3006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.777+0000 7f9a52376640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a3006c9f0 0x7f9a3006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.778+0000 7f9a52376640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a3006c9f0 0x7f9a3006eeb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c006fd0 tx=0x7f9a3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.778+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f9a4808ac90 con 0x7f9a440d2d00 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.779+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a34005290 con 0x7f9a440d2d00 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.789+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a480171c0 con 0x7f9a440d2d00 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.905+0000 7fb15d86f640 1 Processor -- start 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.905+0000 7fb15d86f640 1 -- start start 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15d86f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 0x7fb15000b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15d86f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb150007460 0x7fb1500df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15d86f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1500e1fd0 con 0x7fb150007080 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15d86f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb1500e2be0 con 0x7fb150007460 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15d86f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1500e1d80 con 0x7fb15000ba30 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15ead1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb150007460 0x7fb1500df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15ead1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb150007460 0x7fb1500df0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:35120/0 (socket says 172.21.15.135:35120) 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.906+0000 7fb15ead1640 1 -- 172.21.15.135:0/3868930127 learned_addr learned my addr 172.21.15.135:0/3868930127 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.907+0000 7fb15ead1640 1 -- 172.21.15.135:0/3868930127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 msgr2=0x7fb15000b2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.907+0000 7fb15ead1640 1 --2- 172.21.15.135:0/3868930127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 0x7fb15000b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.907+0000 7fb15ead1640 1 -- 172.21.15.135:0/3868930127 >> v1:172.21.15.135:6789/0 conn(0x7fb150007080 legacy=0x7fb1500de9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.907+0000 7fb15ead1640 1 -- 172.21.15.135:0/3868930127 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1500e22b0 con 0x7fb150007460 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.908+0000 7fb15ead1640 1 --2- 172.21.15.135:0/3868930127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500df0c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb158075230 tx=0x7fb1580a7210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d08ce6c663aa46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.909+0000 7fb1577fe640 1 -- 172.21.15.135:0/3868930127 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580b4070 con 0x7fb150007460 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.909+0000 7fb1577fe640 1 -- 172.21.15.135:0/3868930127 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1580af680 con 0x7fb150007460 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.909+0000 7fb1577fe640 1 -- 172.21.15.135:0/3868930127 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580afaf0 con 0x7fb150007460 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.910+0000 7fb15d86f640 1 -- 172.21.15.135:0/3868930127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 msgr2=0x7fb1500df0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.910+0000 7fb15d86f640 1 --2- 172.21.15.135:0/3868930127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500df0c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb158075230 tx=0x7fb1580a7210 comp rx=0 tx=0).stop 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.910+0000 7fb15d86f640 1 -- 172.21.15.135:0/3868930127 shutdown_connections 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.910+0000 7fb15d86f640 1 --2- 172.21.15.135:0/3868930127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 0x7fb15000b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.910+0000 7fb15d86f640 1 --2- 172.21.15.135:0/3868930127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500df0c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.910+0000 7fb15d86f640 1 -- 172.21.15.135:0/3868930127 >> 172.21.15.135:0/3868930127 conn(0x7fb150048430 msgr2=0x7fb150048840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.914+0000 7fb15d86f640 1 -- 172.21.15.135:0/3868930127 shutdown_connections 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.914+0000 7fb15d86f640 1 -- 172.21.15.135:0/3868930127 wait complete. 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.915+0000 7fb15d86f640 1 Processor -- start 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.916+0000 7fb15d86f640 1 -- start start 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.917+0000 7fb15d86f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 0x7fb150043230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.917+0000 7fb15d86f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.917+0000 7fb15d86f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 0x7fb1500420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.918+0000 7fb15d86f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb150048200 con 0x7fb150007460 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.918+0000 7fb15d86f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1500078e0 con 0x7fb150007080 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.918+0000 7fb15d86f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1500e2870 con 0x7fb15000ba30 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.918+0000 7fb157fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.919+0000 7fb15c86d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 0x7fb150043230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.919+0000 7fb15c86d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 0x7fb150043230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56478/0 (socket says 172.21.15.135:56478) 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.919+0000 7fb157fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:35130/0 (socket says 172.21.15.135:35130) 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.919+0000 7fb157fff640 1 -- 172.21.15.135:0/797648593 learned_addr learned my addr 172.21.15.135:0/797648593 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.920+0000 7fb157fff640 1 -- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 msgr2=0x7fb1500420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.920+0000 7fb157fff640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 0x7fb1500420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.920+0000 7fb157fff640 1 -- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 msgr2=0x7fb150043230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.920+0000 7fb157fff640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 0x7fb150043230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.920+0000 7fb157fff640 1 -- 172.21.15.135:0/797648593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1500db1b0 con 0x7fb150007460 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.921+0000 7fb157fff640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500d27a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb14400b750 tx=0x7fb14400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.921+0000 7fb15c86d640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 0x7fb150043230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.921+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144019070 con 0x7fb150007460 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.922+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1500db490 con 0x7fb150007460 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.922+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb150042a80 con 0x7fb150007460 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.924+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14400c720 con 0x7fb150007460 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.924+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400cb10 con 0x7fb150007460 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.926+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb14401c070 con 0x7fb150007460 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.926+0000 7fb1557fa640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb13c06c8f0 0x7fb13c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.927+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7fb144056db0 con 0x7fb150007460 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.927+0000 7fb142ffd640 1 -- 172.21.15.135:0/797648593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb150005570 con 0x7fb150007460 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.927+0000 7fb15c86d640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb13c06c8f0 0x7fb13c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.928+0000 7fb15c86d640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb13c06c8f0 0x7fb13c06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb14c006fd0 tx=0x7fb14c009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:15.936+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb144058d10 con 0x7fb150007460 2024-04-26T22:07:16.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.292+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a34005850 con 0x7f9a440d2d00 2024-04-26T22:07:16.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.293+0000 7f9a42ffd640 1 -- 172.21.15.135:0/3579142557 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5675 (secure 0 0 0) 0x7f9a48058a40 con 0x7f9a440d2d00 2024-04-26T22:07:16.297 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:16.297 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:16.298 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 57 2024-04-26T22:07:16.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a3006c9f0 msgr2=0x7f9a3006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a3006c9f0 0x7f9a3006eeb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c006fd0 tx=0x7f9a3c008040 comp rx=0 tx=0).stop 2024-04-26T22:07:16.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a440d2d00 msgr2=0x7f9a440cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a440d2d00 0x7f9a440cec40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a44049350 tx=0x7f9a48001550 comp rx=0 tx=0).stop 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 shutdown_connections 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9a3006c9f0 0x7f9a3006eeb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a440d2d00 0x7f9a440cec40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a440d2880 0x7f9a440ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 --2- 172.21.15.135:0/3579142557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a38004800 0x7f9a44167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 >> 172.21.15.135:0/3579142557 conn(0x7f9a44048250 msgr2=0x7f9a4402cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 shutdown_connections 2024-04-26T22:07:16.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.298+0000 7f9a53378640 1 -- 172.21.15.135:0/3579142557 wait complete. 2024-04-26T22:07:16.320 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=5610 2024-04-26T22:07:16.320 INFO:tasks.mds_thrash.fs.[b]:waiting for 5 secs before reviving mds.a 2024-04-26T22:07:16.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.469+0000 7fb142ffd640 1 -- 172.21.15.135:0/797648593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb15003ef60 con 0x7fb150007460 2024-04-26T22:07:16.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.470+0000 7fb1557fa640 1 -- 172.21.15.135:0/797648593 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5675 (secure 0 0 0) 0x7fb14405d010 con 0x7fb150007460 2024-04-26T22:07:16.475 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:16.475 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:16.476 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 57 2024-04-26T22:07:16.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb13c06c8f0 msgr2=0x7fb13c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb13c06c8f0 0x7fb13c06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb14c006fd0 tx=0x7fb14c009060 comp rx=0 tx=0).stop 2024-04-26T22:07:16.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 msgr2=0x7fb1500d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500d27a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb14400b750 tx=0x7fb14400bc10 comp rx=0 tx=0).stop 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 shutdown_connections 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb13c06c8f0 0x7fb13c06edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb15000ba30 0x7fb1500420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb150007460 0x7fb1500d27a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 --2- 172.21.15.135:0/797648593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb150007080 0x7fb150043230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 >> 172.21.15.135:0/797648593 conn(0x7fb150048430 msgr2=0x7fb1500dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 shutdown_connections 2024-04-26T22:07:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.479+0000 7fb15d86f640 1 -- 172.21.15.135:0/797648593 wait complete. 2024-04-26T22:07:16.504 INFO:tasks.mds_thrash.fs.[a]:kill mds.b (rank=0) 2024-04-26T22:07:16.505 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T22:07:16.505 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:07:16.524 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T22:07:16.524 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:16.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.725+0000 7fb104189640 1 Processor -- start 2024-04-26T22:07:16.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.731+0000 7fb104189640 1 -- start start 2024-04-26T22:07:16.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.732+0000 7fb104189640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb0fc0abfc0 0x7fb0fc15e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:16.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.732+0000 7fb104189640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb0fc15e170 0x7fb0fc06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:16.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.732+0000 7fb104189640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0fc15edd0 con 0x7fb0fc0c3750 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.732+0000 7fb104189640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc15eba0 con 0x7fb0fc15e170 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.732+0000 7fb104189640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc06e3d0 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb101f25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb0fc0abfc0 0x7fb0fc15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb102726640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb0fc15e170 0x7fb0fc06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb102726640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb0fc15e170 0x7fb0fc06eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:35138/0 (socket says 172.21.15.135:35138) 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb102726640 1 -- 172.21.15.135:0/1481444238 learned_addr learned my addr 172.21.15.135:0/1481444238 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb101f25640 1 -- 172.21.15.135:0/1481444238 >> v2:172.21.15.135:3300/0 conn(0x7fb0fc15e170 msgr2=0x7fb0fc06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb101f25640 1 --2- 172.21.15.135:0/1481444238 >> v2:172.21.15.135:3300/0 conn(0x7fb0fc15e170 0x7fb0fc06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb101f25640 1 -- 172.21.15.135:0/1481444238 >> v1:172.21.15.139:6789/0 conn(0x7fb0fc0c3750 legacy=0x7fb0fc15da60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:16.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.734+0000 7fb101f25640 1 -- 172.21.15.135:0/1481444238 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc15f0b0 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.739+0000 7fb101f25640 1 --2- 172.21.15.135:0/1481444238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc15e660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc06fa00 tx=0x7fb0f802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8de583af76f20ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:16.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.741+0000 7fb100f23640 1 -- 172.21.15.135:0/1481444238 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8037070 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.741+0000 7fb100f23640 1 -- 172.21.15.135:0/1481444238 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8032440 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.741+0000 7fb100f23640 1 -- 172.21.15.135:0/1481444238 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8032850 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.742+0000 7fb104189640 1 -- 172.21.15.135:0/1481444238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 msgr2=0x7fb0fc15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.742+0000 7fb104189640 1 --2- 172.21.15.135:0/1481444238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc15e660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc06fa00 tx=0x7fb0f802a180 comp rx=0 tx=0).stop 2024-04-26T22:07:16.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.746+0000 7fb104189640 1 -- 172.21.15.135:0/1481444238 shutdown_connections 2024-04-26T22:07:16.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.746+0000 7fb104189640 1 --2- 172.21.15.135:0/1481444238 >> v2:172.21.15.135:3300/0 conn(0x7fb0fc15e170 0x7fb0fc06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.746+0000 7fb104189640 1 --2- 172.21.15.135:0/1481444238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc15e660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.746+0000 7fb104189640 1 -- 172.21.15.135:0/1481444238 >> 172.21.15.135:0/1481444238 conn(0x7fb0fc06c310 msgr2=0x7fb0fc06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:16.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.754+0000 7fb104189640 1 -- 172.21.15.135:0/1481444238 shutdown_connections 2024-04-26T22:07:16.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.754+0000 7fb104189640 1 -- 172.21.15.135:0/1481444238 wait complete. 2024-04-26T22:07:16.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.755+0000 7fb104189640 1 Processor -- start 2024-04-26T22:07:16.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.756+0000 7fb104189640 1 -- start start 2024-04-26T22:07:16.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.757+0000 7fb104189640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:16.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.757+0000 7fb104189640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0fc0c3750 0x7fb0fc1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:16.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.757+0000 7fb104189640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0fc15e170 0x7fb0fc149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:16.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.757+0000 7fb101f25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:16.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.757+0000 7fb101f25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:56496/0 (socket says 172.21.15.135:56496) 2024-04-26T22:07:16.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.757+0000 7fb101f25640 1 -- 172.21.15.135:0/1028061050 learned_addr learned my addr 172.21.15.135:0/1028061050 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:16.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.758+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc15c560 con 0x7fb0fc0c3750 2024-04-26T22:07:16.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.758+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc06e620 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.758+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc0ac3a0 con 0x7fb0fc15e170 2024-04-26T22:07:16.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.759+0000 7fb102726640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0fc15e170 0x7fb0fc149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:16.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.759+0000 7fb101f25640 1 -- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0fc15e170 msgr2=0x7fb0fc149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.759+0000 7fb101f25640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0fc15e170 0x7fb0fc149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.759+0000 7fb101f25640 1 -- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0fc0c3750 msgr2=0x7fb0fc1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:16.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.759+0000 7fb101f25640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0fc0c3750 0x7fb0fc1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:16.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.760+0000 7fb101f25640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc071880 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.761+0000 7fb101f25640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc140cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0f802a770 tx=0x7fb0f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:16.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.762+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8037040 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.762+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc071b60 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.762+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc14a9a0 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.762+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8042070 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.762+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8030960 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.767+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0f8030b00 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.768+0000 7fb0f27fc640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0e006c920 0x7fb0e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:16.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.768+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7fb0f80aca70 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.768+0000 7fb101724640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0e006c920 0x7fb0e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:16.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.769+0000 7fb101724640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0e006c920 0x7fb0e006ede0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0aa820 tx=0x7fb0ec006c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:16.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.769+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e4005290 con 0x7fb0fc0abfc0 2024-04-26T22:07:16.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:16.783+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0f807a910 con 0x7fb0fc0abfc0 2024-04-26T22:07:17.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.278+0000 7fb104189640 1 -- 172.21.15.135:0/1028061050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0e4005580 con 0x7fb0fc0abfc0 2024-04-26T22:07:17.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.280+0000 7fb0f27fc640 1 -- 172.21.15.135:0/1028061050 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5675 (secure 0 0 0) 0x7fb0f807a210 con 0x7fb0fc0abfc0 2024-04-26T22:07:17.281 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:17.281 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:17.282 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 57 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 -- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0e006c920 msgr2=0x7fb0e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0e006c920 0x7fb0e006ede0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0aa820 tx=0x7fb0ec006c40 comp rx=0 tx=0).stop 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 -- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 msgr2=0x7fb0fc140cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc140cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0f802a770 tx=0x7fb0f80046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 -- 172.21.15.135:0/1028061050 shutdown_connections 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0e006c920 0x7fb0e006ede0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0fc15e170 0x7fb0fc149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0fc0c3750 0x7fb0fc1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 --2- 172.21.15.135:0/1028061050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0fc0abfc0 0x7fb0fc140cc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:17.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.286+0000 7fb0dffff640 1 -- 172.21.15.135:0/1028061050 >> 172.21.15.135:0/1028061050 conn(0x7fb0fc06c310 msgr2=0x7fb0fc06eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:17.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.287+0000 7fb0dffff640 1 -- 172.21.15.135:0/1028061050 shutdown_connections 2024-04-26T22:07:17.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:17.287+0000 7fb0dffff640 1 -- 172.21.15.135:0/1028061050 wait complete. 2024-04-26T22:07:17.308 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:07:18.322 INFO:tasks.ceph.mds.c.smithi135.stderr:2024-04-26T22:07:18.322+0000 7faf78cdb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:07:19.310 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:19.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.504+0000 7f1481452640 1 Processor -- start 2024-04-26T22:07:19.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.505+0000 7f1481452640 1 -- start start 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f1481452640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f14740e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f1481452640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1474046be0 con 0x7f14740d3a00 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f1481452640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14740d3e80 con 0x7f147400b100 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f1481452640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1474047690 con 0x7f14740d71a0 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f147b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f14740e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f147b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f14740e4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48808/0 (socket says 172.21.15.135:48808) 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.506+0000 7f147b7fe640 1 -- 172.21.15.135:0/748147576 learned_addr learned my addr 172.21.15.135:0/748147576 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.507+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980221707 0 0) 0x7f1474046be0 con 0x7f14740d3a00 2024-04-26T22:07:19.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.508+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1464003520 con 0x7f14740d3a00 2024-04-26T22:07:19.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2987477085 0 0) 0x7f14740d3e80 con 0x7f147400b100 2024-04-26T22:07:19.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1474046be0 con 0x7f147400b100 2024-04-26T22:07:19.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3733811731 0 0) 0x7f1464003520 con 0x7f14740d3a00 2024-04-26T22:07:19.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 msgr2=0x7f14740e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:19.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f14740e4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 >> v1:172.21.15.139:6789/0 conn(0x7f147400b100 legacy=0x7f147404aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:19.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.509+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740096f0 con 0x7f14740d3a00 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.510+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f147c050be0 con 0x7f14740d3a00 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.510+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 >> v1:172.21.15.135:6789/0 conn(0x7f14740d3a00 legacy=0x7f14740d6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.510+0000 7f1479ffb640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f1464003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.510+0000 7f1479ffb640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14640040b0 0x7f1464004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.511+0000 7f147a7fc640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14640040b0 0x7f1464004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.511+0000 7f147affd640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f1464003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:19.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.511+0000 7f1479ffb640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1464004d40 0x7f14640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.511+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1474047690 con 0x7f14640040b0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.511+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f146400baa0 con 0x7f1464004d40 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.511+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f146400bca0 con 0x7f14740d71a0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.512+0000 7f147b7fe640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1464004d40 0x7f14640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.512+0000 7f147a7fc640 1 -- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 msgr2=0x7f1464003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.512+0000 7f147a7fc640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f1464003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.512+0000 7f147a7fc640 1 -- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1464004d40 msgr2=0x7f14640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.512+0000 7f147a7fc640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1464004d40 0x7f14640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.512+0000 7f147a7fc640 1 -- 172.21.15.135:0/748147576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1464003520 con 0x7f14640040b0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.513+0000 7f147a7fc640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14640040b0 0x7f1464004800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f147c075190 tx=0x7f1464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54f323c0f9c9439 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.514+0000 7f147affd640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14740d71a0 0x7f1464003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.515+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0b5070 con 0x7f14640040b0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.515+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0a8d30 con 0x7f14640040b0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.515+0000 7f1479ffb640 1 -- 172.21.15.135:0/748147576 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0690f0 con 0x7f14640040b0 2024-04-26T22:07:19.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.516+0000 7f1481452640 1 -- 172.21.15.135:0/748147576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14640040b0 msgr2=0x7f1464004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:19.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.516+0000 7f1481452640 1 --2- 172.21.15.135:0/748147576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14640040b0 0x7f1464004800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f147c075190 tx=0x7f1464005830 comp rx=0 tx=0).stop 2024-04-26T22:07:19.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.516+0000 7f147b7fe640 1 -- 172.21.15.135:0/748147576 reap_dead start 2024-04-26T22:07:19.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.518+0000 7f1481452640 1 -- 172.21.15.135:0/748147576 shutdown_connections 2024-04-26T22:07:19.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.518+0000 7f1481452640 1 -- 172.21.15.135:0/748147576 >> 172.21.15.135:0/748147576 conn(0x7f1474005ec0 msgr2=0x7f14740062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:19.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.518+0000 7f1481452640 1 -- 172.21.15.135:0/748147576 shutdown_connections 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.518+0000 7f1481452640 1 -- 172.21.15.135:0/748147576 wait complete. 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.520+0000 7f1481452640 1 Processor -- start 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.520+0000 7f1481452640 1 -- start start 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.521+0000 7f1481452640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14640040b0 0x7f14740d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.521+0000 7f1481452640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1474030b60 0x7f14740c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.521+0000 7f1481452640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 0x7f147402fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.521+0000 7f1481452640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1474046ec0 con 0x7f1474030b60 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.521+0000 7f1481452640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14740d6600 con 0x7f14740c4a70 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.521+0000 7f1481452640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1474047040 con 0x7f14640040b0 2024-04-26T22:07:19.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 0x7f147402fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 0x7f147402fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55708/0 (socket says 172.21.15.135:55708) 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 -- 172.21.15.135:0/1458745220 learned_addr learned my addr 172.21.15.135:0/1458745220 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 -- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14640040b0 msgr2=0x7f14740d7c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14640040b0 0x7f14740d7c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 -- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1474030b60 msgr2=0x7f14740c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1474030b60 0x7f14740c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 -- 172.21.15.135:0/1458745220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740ca960 con 0x7f14740c4a70 2024-04-26T22:07:19.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.522+0000 7f147b7fe640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 0x7f147402fd60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1474008420 tx=0x7f147000a800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:19.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.525+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470004440 con 0x7f14740c4a70 2024-04-26T22:07:19.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.525+0000 7f1481452640 1 -- 172.21.15.135:0/1458745220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14740cac40 con 0x7f14740c4a70 2024-04-26T22:07:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.525+0000 7f1481452640 1 -- 172.21.15.135:0/1458745220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14740d8190 con 0x7f14740c4a70 2024-04-26T22:07:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.527+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470003610 con 0x7f14740c4a70 2024-04-26T22:07:19.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.527+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470003a00 con 0x7f14740c4a70 2024-04-26T22:07:19.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.528+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1470003ba0 con 0x7f14740c4a70 2024-04-26T22:07:19.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.529+0000 7f14637fe640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f146806c9c0 0x7f146806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:19.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.529+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f147008b0d0 con 0x7f14740c4a70 2024-04-26T22:07:19.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.529+0000 7f147affd640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f146806c9c0 0x7f146806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:19.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.529+0000 7f1481452640 1 -- 172.21.15.135:0/1458745220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c005290 con 0x7f14740c4a70 2024-04-26T22:07:19.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.530+0000 7f147affd640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f146806c9c0 0x7f146806ee80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f146c0011e0 tx=0x7f146c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:19.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.540+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f147000d070 con 0x7f14740c4a70 2024-04-26T22:07:19.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.979+0000 7f1481452640 1 -- 172.21.15.135:0/1458745220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f145c005580 con 0x7f14740c4a70 2024-04-26T22:07:19.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.984+0000 7f14637fe640 1 -- 172.21.15.135:0/1458745220 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5675 (secure 0 0 0) 0x7f1470058ed0 con 0x7f14740c4a70 2024-04-26T22:07:19.986 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:19.986 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:19.987 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 57 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.988+0000 7f14617fa640 1 -- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f146806c9c0 msgr2=0x7f146806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.988+0000 7f14617fa640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f146806c9c0 0x7f146806ee80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f146c0011e0 tx=0x7f146c009040 comp rx=0 tx=0).stop 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.988+0000 7f14617fa640 1 -- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 msgr2=0x7f147402fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.988+0000 7f14617fa640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 0x7f147402fd60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1474008420 tx=0x7f147000a800 comp rx=0 tx=0).stop 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 -- 172.21.15.135:0/1458745220 shutdown_connections 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f146806c9c0 0x7f146806ee80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14740c4a70 0x7f147402fd60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1474030b60 0x7f14740c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 --2- 172.21.15.135:0/1458745220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14640040b0 0x7f14740d7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:19.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 -- 172.21.15.135:0/1458745220 >> 172.21.15.135:0/1458745220 conn(0x7f1474005ec0 msgr2=0x7f147404abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:19.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 -- 172.21.15.135:0/1458745220 shutdown_connections 2024-04-26T22:07:19.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:19.989+0000 7f14617fa640 1 -- 172.21.15.135:0/1458745220 wait complete. 2024-04-26T22:07:20.016 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:07:21.321 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-04-26T22:07:21.321 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:07:21.322 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:07:21.327 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:07:21.327 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:21.471 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-26T22:07:21.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.539+0000 7f7afdbc3640 1 Processor -- start 2024-04-26T22:07:21.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.539+0000 7f7afdbc3640 1 -- start start 2024-04-26T22:07:21.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.541+0000 7f7afdbc3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af0004b00 0x7f7af00d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:21.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.541+0000 7f7afdbc3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af0007330 con 0x7f7af00465a0 2024-04-26T22:07:21.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.541+0000 7f7afdbc3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af00d6510 con 0x7f7af00d6d10 2024-04-26T22:07:21.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.541+0000 7f7afdbc3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7af0006e00 con 0x7f7af0004b00 2024-04-26T22:07:21.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.543+0000 7f7afd3c2640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7af00465a0 0x7f7af00dacc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44126/0 (socket says 172.21.15.135:44126) 2024-04-26T22:07:21.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.543+0000 7f7afd3c2640 1 -- 172.21.15.135:0/2386883555 learned_addr learned my addr 172.21.15.135:0/2386883555 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:21.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.544+0000 7f7afcbc1640 1 --2- 172.21.15.135:0/2386883555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af0004b00 0x7f7af00d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:21.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.544+0000 7f7af77fe640 1 -- 172.21.15.135:0/2386883555 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610748436 0 0) 0x7f7af0007330 con 0x7f7af00465a0 2024-04-26T22:07:21.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.544+0000 7f7af77fe640 1 -- 172.21.15.135:0/2386883555 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ae8003520 con 0x7f7af00465a0 2024-04-26T22:07:21.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.545+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2386883555 >> v1:172.21.15.139:6789/0 conn(0x7f7af00d6d10 legacy=0x7f7af00d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:21.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.545+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2386883555 >> v1:172.21.15.135:6789/0 conn(0x7f7af00465a0 legacy=0x7f7af00dacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:21.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.545+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2386883555 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af00d67f0 con 0x7f7af0004b00 2024-04-26T22:07:21.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.548+0000 7f7afcbc1640 1 --2- 172.21.15.135:0/2386883555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af0004b00 0x7f7af00d7860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40097d0 tx=0x7f7ae402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e16f580327013e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:21.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.549+0000 7f7af77fe640 1 -- 172.21.15.135:0/2386883555 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4037070 con 0x7f7af0004b00 2024-04-26T22:07:21.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.549+0000 7f7af77fe640 1 -- 172.21.15.135:0/2386883555 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae4032440 con 0x7f7af0004b00 2024-04-26T22:07:21.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.549+0000 7f7af77fe640 1 -- 172.21.15.135:0/2386883555 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4037070 con 0x7f7af0004b00 2024-04-26T22:07:21.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.550+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2386883555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af0004b00 msgr2=0x7f7af00d7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:21.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.550+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2386883555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af0004b00 0x7f7af00d7860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40097d0 tx=0x7f7ae402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:07:21.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.551+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2386883555 shutdown_connections 2024-04-26T22:07:21.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.551+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2386883555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af0004b00 0x7f7af00d7860 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40097d0 tx=0x7f7ae402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:07:21.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.551+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2386883555 >> 172.21.15.135:0/2386883555 conn(0x7f7af00062e0 msgr2=0x7f7af00066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:21.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.552+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2386883555 shutdown_connections 2024-04-26T22:07:21.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.552+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2386883555 wait complete. 2024-04-26T22:07:21.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.554+0000 7f7afdbc3640 1 Processor -- start 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.556+0000 7f7afdbc3640 1 -- start start 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.557+0000 7f7afdbc3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 0x7f7af0178290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.557+0000 7f7afdbc3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af00d6d10 0x7f7af00e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.557+0000 7f7afdbc3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7af00d2d40 0x7f7af0174c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.557+0000 7f7afdbc3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0009be0 con 0x7f7af00d2d40 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.557+0000 7f7afdbc3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0009a60 con 0x7f7af00465a0 2024-04-26T22:07:21.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.557+0000 7f7afdbc3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7af002e4d0 con 0x7f7af00d6d10 2024-04-26T22:07:21.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.559+0000 7f7afcbc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 0x7f7af0178290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:21.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.559+0000 7f7afcbc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 0x7f7af0178290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55744/0 (socket says 172.21.15.135:55744) 2024-04-26T22:07:21.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.559+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2280086669 learned_addr learned my addr 172.21.15.135:0/2280086669 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:21.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.559+0000 7f7af7fff640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af00d6d10 0x7f7af00e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:21.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.560+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af00d6d10 msgr2=0x7f7af00e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:21.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.560+0000 7f7afcbc1640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af00d6d10 0x7f7af00e32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:21.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.560+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7af00d2d40 msgr2=0x7f7af0174c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:21.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.560+0000 7f7afcbc1640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7af00d2d40 0x7f7af0174c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:21.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.560+0000 7f7afcbc1640 1 -- 172.21.15.135:0/2280086669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af00459c0 con 0x7f7af00465a0 2024-04-26T22:07:21.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.560+0000 7f7afcbc1640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 0x7f7af0178290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:21.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.561+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4041020 con 0x7f7af00465a0 2024-04-26T22:07:21.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.561+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae40306d0 con 0x7f7af00465a0 2024-04-26T22:07:21.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.561+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af0045ca0 con 0x7f7af00465a0 2024-04-26T22:07:21.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.561+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af0042e80 con 0x7f7af00465a0 2024-04-26T22:07:21.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.563+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4037070 con 0x7f7af00465a0 2024-04-26T22:07:21.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.563+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af00057e0 con 0x7f7af00465a0 2024-04-26T22:07:21.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.566+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae4044020 con 0x7f7af00465a0 2024-04-26T22:07:21.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.567+0000 7f7af57fa640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7adc06c970 0x7f7adc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:21.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.567+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f7ae40acb30 con 0x7f7af00465a0 2024-04-26T22:07:21.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.568+0000 7f7af7fff640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7adc06c970 0x7f7adc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:21.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.569+0000 7f7af7fff640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7adc06c970 0x7f7adc06ee30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7af806bac0 tx=0x7f7af806b630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:21.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:21.575+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ae404b070 con 0x7f7af00465a0 2024-04-26T22:07:22.017 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:22.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.062+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7af00d9150 con 0x7f7af00465a0 2024-04-26T22:07:22.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.063+0000 7f7af57fa640 1 -- 172.21.15.135:0/2280086669 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7f7ae407aac0 con 0x7f7af00465a0 2024-04-26T22:07:22.063 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:22.063 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:22.064 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:22.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.068+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7adc06c970 msgr2=0x7f7adc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.068+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7adc06c970 0x7f7adc06ee30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7af806bac0 tx=0x7f7af806b630 comp rx=0 tx=0).stop 2024-04-26T22:07:22.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 msgr2=0x7f7af0178290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 0x7f7af0178290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae40046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 shutdown_connections 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7adc06c970 0x7f7adc06ee30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7af00d2d40 0x7f7af0174c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7af00d6d10 0x7f7af00e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 --2- 172.21.15.135:0/2280086669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7af00465a0 0x7f7af0178290 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae40046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.069+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 >> 172.21.15.135:0/2280086669 conn(0x7f7af00062e0 msgr2=0x7f7af000a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:22.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.070+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 shutdown_connections 2024-04-26T22:07:22.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.072+0000 7f7afdbc3640 1 -- 172.21.15.135:0/2280086669 wait complete. 2024-04-26T22:07:22.094 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-04-26T22:07:22.094 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:22.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.397+0000 7f06fb989640 1 Processor -- start 2024-04-26T22:07:22.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.397+0000 7f06fb989640 1 -- start start 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06fb989640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06f40ab380 0x7f06f4152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06fb989640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f06f4150090 0x7f06f40c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06fb989640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f40c40e0 con 0x7f06f406f990 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06fb989640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f06f406e3d0 con 0x7f06f4150090 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06fb989640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f06f406e6b0 con 0x7f06f40ab380 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06f9725640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06f40ab380 0x7f06f4152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06f9725640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06f40ab380 0x7f06f4152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55758/0 (socket says 172.21.15.135:55758) 2024-04-26T22:07:22.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.399+0000 7f06f9725640 1 -- 172.21.15.135:0/3824565822 learned_addr learned my addr 172.21.15.135:0/3824565822 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:22.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.400+0000 7f06f9f26640 1 --2- 172.21.15.135:0/3824565822 >> v2:172.21.15.135:3300/0 conn(0x7f06f4150090 0x7f06f40c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.400+0000 7f06f9725640 1 -- 172.21.15.135:0/3824565822 >> v2:172.21.15.135:3300/0 conn(0x7f06f4150090 msgr2=0x7f06f40c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:22.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.400+0000 7f06f9725640 1 --2- 172.21.15.135:0/3824565822 >> v2:172.21.15.135:3300/0 conn(0x7f06f4150090 0x7f06f40c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.400+0000 7f06f9725640 1 -- 172.21.15.135:0/3824565822 >> v1:172.21.15.135:6789/0 conn(0x7f06f406f990 legacy=0x7f06f40c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.400+0000 7f06f9725640 1 -- 172.21.15.135:0/3824565822 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f40c43c0 con 0x7f06f40ab380 2024-04-26T22:07:22.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.402+0000 7f06f9725640 1 --2- 172.21.15.135:0/3824565822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 0x7f06f4152cf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06f40c2ba0 tx=0x7f06f002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cbe1ac2d6bee54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:22.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.402+0000 7f06ebfff640 1 -- 172.21.15.135:0/3824565822 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0037070 con 0x7f06f40ab380 2024-04-26T22:07:22.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.402+0000 7f06ebfff640 1 -- 172.21.15.135:0/3824565822 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f0032440 con 0x7f06f40ab380 2024-04-26T22:07:22.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.402+0000 7f06ebfff640 1 -- 172.21.15.135:0/3824565822 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0032810 con 0x7f06f40ab380 2024-04-26T22:07:22.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.404+0000 7f06fb989640 1 -- 172.21.15.135:0/3824565822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 msgr2=0x7f06f4152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.404+0000 7f06fb989640 1 --2- 172.21.15.135:0/3824565822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 0x7f06f4152cf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06f40c2ba0 tx=0x7f06f002a230 comp rx=0 tx=0).stop 2024-04-26T22:07:22.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.407+0000 7f06fb989640 1 -- 172.21.15.135:0/3824565822 shutdown_connections 2024-04-26T22:07:22.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.407+0000 7f06fb989640 1 --2- 172.21.15.135:0/3824565822 >> v2:172.21.15.135:3300/0 conn(0x7f06f4150090 0x7f06f40c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.407+0000 7f06fb989640 1 --2- 172.21.15.135:0/3824565822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 0x7f06f4152cf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.407+0000 7f06fb989640 1 -- 172.21.15.135:0/3824565822 >> 172.21.15.135:0/3824565822 conn(0x7f06f406c310 msgr2=0x7f06f406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:22.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.408+0000 7fe0ae80b640 1 Processor -- start 2024-04-26T22:07:22.408 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.408+0000 7f06fb989640 1 -- 172.21.15.135:0/3824565822 shutdown_connections 2024-04-26T22:07:22.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.408+0000 7f06fb989640 1 -- 172.21.15.135:0/3824565822 wait complete. 2024-04-26T22:07:22.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.408+0000 7fe0ae80b640 1 -- start start 2024-04-26T22:07:22.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ae80b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0a00d7810 0x7fe0a00da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ae80b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe0a00496d0 0x7fe0a0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ae80b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a000a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ae80b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a00da950 con 0x7fe0a00496d0 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ae80b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a00dac30 con 0x7fe0a00d7810 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ae80b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a00daf10 con 0x7fe0a00073e0 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe0a00496d0 0x7fe0a0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe0a00496d0 0x7fe0a0006ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53630/0 (socket says 172.21.15.135:53630) 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 -- 172.21.15.135:0/3137616816 learned_addr learned my addr 172.21.15.135:0/3137616816 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 -- 172.21.15.135:0/3137616816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 msgr2=0x7fe0a000a4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 --2- 172.21.15.135:0/3137616816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a000a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 -- 172.21.15.135:0/3137616816 >> v2:172.21.15.139:3300/0 conn(0x7fe0a00d7810 msgr2=0x7fe0a00da320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 --2- 172.21.15.135:0/3137616816 >> v2:172.21.15.139:3300/0 conn(0x7fe0a00d7810 0x7fe0a00da320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.410+0000 7fe0ad008640 1 -- 172.21.15.135:0/3137616816 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a00db1f0 con 0x7fe0a00496d0 2024-04-26T22:07:22.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.411+0000 7fe0ad008640 1 --2- 172.21.15.135:0/3137616816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00496d0 0x7fe0a0006ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8065e50 tx=0x7fe0a80a7390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7070076bd6954b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:22.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.413+0000 7fe0ac807640 1 -- 172.21.15.135:0/3137616816 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80b4070 con 0x7fe0a00496d0 2024-04-26T22:07:22.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.413+0000 7fe0ac807640 1 -- 172.21.15.135:0/3137616816 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a80af440 con 0x7fe0a00496d0 2024-04-26T22:07:22.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.413+0000 7fe0ac807640 1 -- 172.21.15.135:0/3137616816 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80af8b0 con 0x7fe0a00496d0 2024-04-26T22:07:22.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7f06fb989640 1 Processor -- start 2024-04-26T22:07:22.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 -- 172.21.15.135:0/3137616816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00496d0 msgr2=0x7fe0a0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/3137616816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00496d0 0x7fe0a0006ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8065e50 tx=0x7fe0a80a7390 comp rx=0 tx=0).stop 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 -- 172.21.15.135:0/3137616816 shutdown_connections 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/3137616816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00496d0 0x7fe0a0006ea0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/3137616816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a000a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/3137616816 >> v2:172.21.15.139:3300/0 conn(0x7fe0a00d7810 0x7fe0a00da320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.414+0000 7fe0ae80b640 1 -- 172.21.15.135:0/3137616816 >> 172.21.15.135:0/3137616816 conn(0x7fe0a0048250 msgr2=0x7fe0a0048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.415+0000 7f06fb989640 1 -- start start 2024-04-26T22:07:22.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.415+0000 7fe0ae80b640 1 -- 172.21.15.135:0/3137616816 shutdown_connections 2024-04-26T22:07:22.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06fb989640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f406f990 0x7f06f4071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06fb989640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 0x7f06f4072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06fb989640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 0x7f06f40a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06f9f26640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 0x7f06f40a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06f9f26640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 0x7f06f40a9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53642/0 (socket says 172.21.15.135:53642) 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06f9f26640 1 -- 172.21.15.135:0/1979134101 learned_addr learned my addr 172.21.15.135:0/1979134101 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06fb989640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06f40c51d0 con 0x7f06f4150090 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06fb989640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4150750 con 0x7f06f40ab380 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7f06fb989640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f06f40c5350 con 0x7f06f406f990 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7f06f9f26640 1 -- 172.21.15.135:0/1979134101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f406f990 msgr2=0x7f06f4071ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7f06f9f26640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f406f990 0x7f06f4071ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7f06f9f26640 1 -- 172.21.15.135:0/1979134101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 msgr2=0x7f06f4072400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7f06f9f26640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 0x7f06f4072400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7f06f9f26640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f40c75e0 con 0x7f06f4150090 2024-04-26T22:07:22.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7f06f9f26640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 0x7f06f40a9fc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00efc0 tx=0x7f06ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.419+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec018020 con 0x7f06f4150090 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.419+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ec014480 con 0x7f06f4150090 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.419+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec014870 con 0x7f06f4150090 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.415+0000 7fe0ae80b640 1 -- 172.21.15.135:0/3137616816 wait complete. 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7fe0ae80b640 1 Processor -- start 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.417+0000 7fe0ae80b640 1 -- start start 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7fe0ae80b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a00d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7fe0ae80b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0a00496d0 0x7fe0a0174660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7fe0ae80b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00d7810 0x7fe0a0174ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7fe0ae80b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0049df0 con 0x7fe0a00d7810 2024-04-26T22:07:22.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7fe0ae80b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a000ba20 con 0x7fe0a00496d0 2024-04-26T22:07:22.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.418+0000 7fe0ae80b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a00d7bf0 con 0x7fe0a00073e0 2024-04-26T22:07:22.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.419+0000 7f06fb989640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f40c7b80 con 0x7f06f4150090 2024-04-26T22:07:22.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.419+0000 7f06fb989640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f40c6880 con 0x7f06f4150090 2024-04-26T22:07:22.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.419+0000 7fe0ae00a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00d7810 0x7fe0a0174ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.421+0000 7fe0ad008640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0a00496d0 0x7fe0a0174660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.421+0000 7fe0ad809640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a00d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.421+0000 7fe0ad008640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0a00496d0 0x7fe0a0174660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55778/0 (socket says 172.21.15.135:55778) 2024-04-26T22:07:22.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.421+0000 7fe0ad008640 1 -- 172.21.15.135:0/1001568709 learned_addr learned my addr 172.21.15.135:0/1001568709 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.422+0000 7fe0ad809640 1 -- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0a00496d0 msgr2=0x7fe0a0174660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.422+0000 7fe0ad809640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0a00496d0 0x7fe0a0174660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.422+0000 7fe0ad809640 1 -- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00d7810 msgr2=0x7fe0a0174ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.422+0000 7fe0ad809640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00d7810 0x7fe0a0174ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.422+0000 7fe0ad809640 1 -- 172.21.15.135:0/1001568709 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a01750e0 con 0x7fe0a00073e0 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.422+0000 7fe0ae00a640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00d7810 0x7fe0a0174ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.423+0000 7fe0ad809640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a00d5d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe098009fd0 tx=0x7fe09800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:22.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.424+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06ec01b020 con 0x7f06f4150090 2024-04-26T22:07:22.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.424+0000 7f06e9ffb640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f06d806c940 0x7f06d806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.425+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f06ec08cf90 con 0x7f06f4150090 2024-04-26T22:07:22.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.425+0000 7f06f9725640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f06d806c940 0x7f06d806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.426+0000 7f06f9725640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f06d806c940 0x7f06d806ee00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f06f0034000 tx=0x7f06f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:22.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.426+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098015020 con 0x7fe0a00073e0 2024-04-26T22:07:22.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.426+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe098011850 con 0x7fe0a00073e0 2024-04-26T22:07:22.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.426+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098011c40 con 0x7fe0a00073e0 2024-04-26T22:07:22.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.427+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f414fb60 con 0x7f06f4150090 2024-04-26T22:07:22.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.430+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a0174160 con 0x7fe0a00073e0 2024-04-26T22:07:22.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.430+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a00dda80 con 0x7fe0a00073e0 2024-04-26T22:07:22.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.434+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a0005390 con 0x7fe0a00073e0 2024-04-26T22:07:22.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.436+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06f414fb60 con 0x7f06f4150090 2024-04-26T22:07:22.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.436+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe098018020 con 0x7fe0a00073e0 2024-04-26T22:07:22.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.437+0000 7fe09e7fc640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe08c06c9c0 0x7fe08c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:22.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.437+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7fe09808bf40 con 0x7fe0a00073e0 2024-04-26T22:07:22.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.438+0000 7fe0ad008640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe08c06c9c0 0x7fe08c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:22.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.439+0000 7fe0ad008640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe08c06c9c0 0x7fe08c06ee80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8065e50 tx=0x7fe0a80ab000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:22.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.448+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe09801f020 con 0x7fe0a00073e0 2024-04-26T22:07:22.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.975+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06f40c2900 con 0x7f06f4150090 2024-04-26T22:07:22.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.977+0000 7f06e9ffb640 1 -- 172.21.15.135:0/1979134101 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7f06ec022020 con 0x7f06f4150090 2024-04-26T22:07:22.977 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:22.977 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:22.977 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:22.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.980+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f06d806c940 msgr2=0x7f06d806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.980+0000 7f06d77fe640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f06d806c940 0x7f06d806ee00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f06f0034000 tx=0x7f06f002e000 comp rx=0 tx=0).stop 2024-04-26T22:07:22.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 msgr2=0x7f06f40a9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:22.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 0x7f06f40a9fc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00efc0 tx=0x7f06ec00c490 comp rx=0 tx=0).stop 2024-04-26T22:07:22.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 shutdown_connections 2024-04-26T22:07:22.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f06d806c940 0x7f06d806ee00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06f4150090 0x7f06f40a9fc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f40ab380 0x7f06f4072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 --2- 172.21.15.135:0/1979134101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f406f990 0x7f06f4071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:22.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.981+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 >> 172.21.15.135:0/1979134101 conn(0x7f06f406c310 msgr2=0x7f06f416afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:22.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.985+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 shutdown_connections 2024-04-26T22:07:22.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:22.987+0000 7f06d77fe640 1 -- 172.21.15.135:0/1979134101 wait complete. 2024-04-26T22:07:23.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.006+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0a000b800 con 0x7fe0a00073e0 2024-04-26T22:07:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.008+0000 7fe09e7fc640 1 -- 172.21.15.135:0/1001568709 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7fe0a01750e0 con 0x7fe0a00073e0 2024-04-26T22:07:23.010 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-04-26T22:07:23.010 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:07:23.010 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:07:23.012 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:23.013 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:23.013 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:23.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe08c06c9c0 msgr2=0x7fe08c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe08c06c9c0 0x7fe08c06ee80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8065e50 tx=0x7fe0a80ab000 comp rx=0 tx=0).stop 2024-04-26T22:07:23.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 msgr2=0x7fe0a00d5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a00d5d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe098009fd0 tx=0x7fe09800b870 comp rx=0 tx=0).stop 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 shutdown_connections 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe08c06c9c0 0x7fe08c06ee80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0a00d7810 0x7fe0a0174ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0a00496d0 0x7fe0a0174660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 --2- 172.21.15.135:0/1001568709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0a00073e0 0x7fe0a00d5d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 >> 172.21.15.135:0/1001568709 conn(0x7fe0a0048250 msgr2=0x7fe0a002bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 shutdown_connections 2024-04-26T22:07:23.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.013+0000 7fe0ae80b640 1 -- 172.21.15.135:0/1001568709 wait complete. 2024-04-26T22:07:23.030 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:07:23.031 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:23.038 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:07:23.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.272+0000 7f5c40293640 1 Processor -- start 2024-04-26T22:07:23.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.272+0000 7f5c40293640 1 -- start start 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c40293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c380caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c40293640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c380cba00 0x7f5c380c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c40293640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c38158ec0 con 0x7f5c380cb4b0 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c40293640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5c380cbf50 con 0x7f5c380cba00 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c40293640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3815b310 con 0x7f5c38158ae0 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c3e830640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5c380cb4b0 0x7f5c380c9050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44154/0 (socket says 172.21.15.135:44154) 2024-04-26T22:07:23.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.273+0000 7f5c3e830640 1 -- 172.21.15.135:0/3646906663 learned_addr learned my addr 172.21.15.135:0/3646906663 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.274+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700022967 0 0) 0x7f5c38158ec0 con 0x7f5c380cb4b0 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.274+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c24003520 con 0x7f5c380cb4b0 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.274+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c380caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d82e640 1 --2- 172.21.15.135:0/3646906663 >> v2:172.21.15.139:3300/0 conn(0x7f5c380cba00 0x7f5c380c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3607589399 0 0) 0x7f5c24003520 con 0x7f5c380cb4b0 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 msgr2=0x7f5c380caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:23.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c380caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 >> v2:172.21.15.139:3300/0 conn(0x7f5c380cba00 msgr2=0x7f5c380c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 --2- 172.21.15.135:0/3646906663 >> v2:172.21.15.139:3300/0 conn(0x7f5c380cba00 0x7f5c380c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3815b5f0 con 0x7f5c380cb4b0 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f5c34001670 con 0x7f5c380cb4b0 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 >> v1:172.21.15.135:6789/0 conn(0x7f5c380cb4b0 legacy=0x7f5c380c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c24004030 0x7f5c24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c24004d40 0x7f5c240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380cbf50 con 0x7f5c24004030 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3815b310 con 0x7f5c24004d40 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c2400bca0 con 0x7f5c38158ae0 2024-04-26T22:07:23.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.275+0000 7f5c3e830640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c24004d40 0x7f5c240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.276+0000 7f5c3e02f640 1 -- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c24004d40 msgr2=0x7f5c240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:23.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.276+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c24004d40 0x7f5c240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.276+0000 7f5c3e02f640 1 -- 172.21.15.135:0/3646906663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c24004030 msgr2=0x7f5c24004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:23.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.276+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c24004030 0x7f5c24004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.276+0000 7f5c3e02f640 1 -- 172.21.15.135:0/3646906663 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c24003520 con 0x7f5c38158ae0 2024-04-26T22:07:23.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.278+0000 7f5c3d82e640 1 --2- 172.21.15.135:0/3646906663 >> v2:172.21.15.139:3300/0 conn(0x7f5c380cba00 0x7f5c380c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:23.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.279+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c240039b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5c30007360 tx=0x7f5c24005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8bdeb9e767bf830 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:23.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.280+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30009be0 con 0x7f5c38158ae0 2024-04-26T22:07:23.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.280+0000 7f5c3d02d640 1 -- 172.21.15.135:0/3646906663 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3002e640 con 0x7f5c38158ae0 2024-04-26T22:07:23.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.281+0000 7f5c40293640 1 -- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 msgr2=0x7f5c240039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.281+0000 7f5c40293640 1 --2- 172.21.15.135:0/3646906663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c38158ae0 0x7f5c240039b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5c30007360 tx=0x7f5c24005830 comp rx=0 tx=0).stop 2024-04-26T22:07:23.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.281+0000 7f5c3e830640 1 -- 172.21.15.135:0/3646906663 reap_dead start 2024-04-26T22:07:23.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.283+0000 7f5c40293640 1 -- 172.21.15.135:0/3646906663 shutdown_connections 2024-04-26T22:07:23.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.283+0000 7f5c40293640 1 -- 172.21.15.135:0/3646906663 >> 172.21.15.135:0/3646906663 conn(0x7f5c3806dc10 msgr2=0x7f5c3806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:23.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.287+0000 7f5c40293640 1 -- 172.21.15.135:0/3646906663 shutdown_connections 2024-04-26T22:07:23.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.288+0000 7f5c40293640 1 -- 172.21.15.135:0/3646906663 wait complete. 2024-04-26T22:07:23.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.289+0000 7f5c40293640 1 Processor -- start 2024-04-26T22:07:23.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.289+0000 7f5c40293640 1 -- start start 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.290+0000 7f5c40293640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 0x7f5c381435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.290+0000 7f5c40293640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c381416f0 0x7f5c380ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.290+0000 7f5c40293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c380ad550 0x7f5c3815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.290+0000 7f5c40293640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380cab90 con 0x7f5c381416f0 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.290+0000 7f5c40293640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3815c0d0 con 0x7f5c38158ae0 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.290+0000 7f5c40293640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c380cad10 con 0x7f5c380ad550 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 0x7f5c381435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 0x7f5c381435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55814/0 (socket says 172.21.15.135:55814) 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 -- 172.21.15.135:0/190926705 learned_addr learned my addr 172.21.15.135:0/190926705 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e830640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c380ad550 0x7f5c3815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 -- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c380ad550 msgr2=0x7f5c3815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c380ad550 0x7f5c3815cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 -- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c381416f0 msgr2=0x7f5c380ad010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c381416f0 0x7f5c380ad010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.291+0000 7f5c3e02f640 1 -- 172.21.15.135:0/190926705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c380c7970 con 0x7f5c38158ae0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.292+0000 7f5c3e02f640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 0x7f5c381435f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c3002b0c0 tx=0x7f5c30004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.293+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30036e10 con 0x7f5c38158ae0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.293+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c38140c10 con 0x7f5c38158ae0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.293+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c38143b30 con 0x7f5c38158ae0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.293+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c30003990 con 0x7f5c38158ae0 2024-04-26T22:07:23.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.293+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30003d60 con 0x7f5c38158ae0 2024-04-26T22:07:23.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.296+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c20005290 con 0x7f5c38158ae0 2024-04-26T22:07:23.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.297+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c30005d10 con 0x7f5c38158ae0 2024-04-26T22:07:23.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.299+0000 7f5c2effd640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c1c06c9c0 0x7f5c1c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:23.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.299+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f5c300adda0 con 0x7f5c38158ae0 2024-04-26T22:07:23.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.307+0000 7f5c3d82e640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c1c06c9c0 0x7f5c1c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:23.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.307+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c3003c040 con 0x7f5c38158ae0 2024-04-26T22:07:23.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.308+0000 7f5c3d82e640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c1c06c9c0 0x7f5c1c06ee80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5c380ccd10 tx=0x7f5c28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:23.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.773+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c20005580 con 0x7f5c38158ae0 2024-04-26T22:07:23.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.775+0000 7f5c2effd640 1 -- 172.21.15.135:0/190926705 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7f5c30032030 con 0x7f5c38158ae0 2024-04-26T22:07:23.777 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:23.777 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:23.777 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c1c06c9c0 msgr2=0x7f5c1c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c1c06c9c0 0x7f5c1c06ee80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5c380ccd10 tx=0x7f5c28008040 comp rx=0 tx=0).stop 2024-04-26T22:07:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 msgr2=0x7f5c381435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 0x7f5c381435f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c3002b0c0 tx=0x7f5c30004ab0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 shutdown_connections 2024-04-26T22:07:23.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5c1c06c9c0 0x7f5c1c06ee80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c380ad550 0x7f5c3815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c381416f0 0x7f5c380ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 --2- 172.21.15.135:0/190926705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c38158ae0 0x7f5c381435f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:23.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.780+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 >> 172.21.15.135:0/190926705 conn(0x7f5c3806dc10 msgr2=0x7f5c380b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:23.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.782+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 shutdown_connections 2024-04-26T22:07:23.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:23.782+0000 7f5c40293640 1 -- 172.21.15.135:0/190926705 wait complete. 2024-04-26T22:07:23.805 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:07:25.039 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.246+0000 7f6bee4a8640 1 Processor -- start 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.247+0000 7f6bee4a8640 1 -- start start 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.248+0000 7f6bee4a8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6be80aa7c0 0x7f6be80c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.248+0000 7f6bee4a8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80abdc0 0x7f6be80ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.248+0000 7f6bee4a8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be806dec0 con 0x7f6be815b940 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.248+0000 7f6bee4a8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6be80c2ac0 con 0x7f6be80aa7c0 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.248+0000 7f6bee4a8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6be806e1a0 con 0x7f6be80abdc0 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.248+0000 7f6becca5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6be80aa7c0 0x7f6be80c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.249+0000 7f6bed4a6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6be815b940 0x7f6be80c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:34780/0 (socket says 172.21.15.135:34780) 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.249+0000 7f6bed4a6640 1 -- 172.21.15.135:0/1380354086 learned_addr learned my addr 172.21.15.135:0/1380354086 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:25.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.249+0000 7f6bedca7640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80abdc0 0x7f6be80ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.250+0000 7f6becca5640 1 -- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80abdc0 msgr2=0x7f6be80ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.250+0000 7f6becca5640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80abdc0 0x7f6be80ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.250+0000 7f6becca5640 1 -- 172.21.15.135:0/1380354086 >> v1:172.21.15.139:6789/0 conn(0x7f6be815b940 legacy=0x7f6be80c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.250+0000 7f6becca5640 1 -- 172.21.15.135:0/1380354086 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be806e480 con 0x7f6be80aa7c0 2024-04-26T22:07:25.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.251+0000 7f6bedca7640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80abdc0 0x7f6be80ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:25.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.255+0000 7f6becca5640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be80aa7c0 0x7f6be80c3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80097a0 tx=0x7f6bd802a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0c2ada15722e9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:25.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.256+0000 7f6bdffff640 1 -- 172.21.15.135:0/1380354086 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8037070 con 0x7f6be80aa7c0 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.256+0000 7f6bdffff640 1 -- 172.21.15.135:0/1380354086 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd802aca0 con 0x7f6be80aa7c0 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.256+0000 7f6bdffff640 1 -- 172.21.15.135:0/1380354086 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8032660 con 0x7f6be80aa7c0 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.257+0000 7f6bee4a8640 1 -- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be80aa7c0 msgr2=0x7f6be80c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.257+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be80aa7c0 0x7f6be80c3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80097a0 tx=0x7f6bd802a010 comp rx=0 tx=0).stop 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.257+0000 7f6bee4a8640 1 -- 172.21.15.135:0/1380354086 shutdown_connections 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.257+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80abdc0 0x7f6be80ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.257+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/1380354086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be80aa7c0 0x7f6be80c3940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.257+0000 7f6bee4a8640 1 -- 172.21.15.135:0/1380354086 >> 172.21.15.135:0/1380354086 conn(0x7f6be80c80c0 msgr2=0x7f6be80c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:25.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.262+0000 7f6bee4a8640 1 -- 172.21.15.135:0/1380354086 shutdown_connections 2024-04-26T22:07:25.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.262+0000 7f6bee4a8640 1 -- 172.21.15.135:0/1380354086 wait complete. 2024-04-26T22:07:25.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.264+0000 7f6bee4a8640 1 Processor -- start 2024-04-26T22:07:25.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.264+0000 7f6bee4a8640 1 -- start start 2024-04-26T22:07:25.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.265+0000 7f6bee4a8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 0x7f6be81fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.265+0000 7f6bee4a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6be80abdc0 0x7f6be8158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.265+0000 7f6bee4a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be815b940 0x7f6be8157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.265+0000 7f6bee4a8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6be815bea0 con 0x7f6be80abdc0 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.265+0000 7f6bee4a8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6be815bd20 con 0x7f6be815b940 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.266+0000 7f6bed4a6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 0x7f6be81fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.266+0000 7f6bed4a6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 0x7f6be81fcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48912/0 (socket says 172.21.15.135:48912) 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.266+0000 7f6bed4a6640 1 -- 172.21.15.135:0/2875925965 learned_addr learned my addr 172.21.15.135:0/2875925965 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.266+0000 7f6becca5640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6be80abdc0 0x7f6be8158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.266+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6be80ac780 con 0x7f6be80aa7c0 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.266+0000 7f6bedca7640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be815b940 0x7f6be8157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6bed4a6640 1 -- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be815b940 msgr2=0x7f6be8157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6bed4a6640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be815b940 0x7f6be8157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6bed4a6640 1 -- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6be80abdc0 msgr2=0x7f6be8158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6bed4a6640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6be80abdc0 0x7f6be8158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6bed4a6640 1 -- 172.21.15.135:0/2875925965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be8158750 con 0x7f6be80aa7c0 2024-04-26T22:07:25.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6becca5640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6be80abdc0 0x7f6be8158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:25.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.267+0000 7f6bed4a6640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 0x7f6be81fcec0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6be000caf0 tx=0x7f6be0002090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:25.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.270+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0018070 con 0x7f6be80aa7c0 2024-04-26T22:07:25.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.270+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be8157c90 con 0x7f6be80aa7c0 2024-04-26T22:07:25.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.270+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be80bf570 con 0x7f6be80aa7c0 2024-04-26T22:07:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.275+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be000a700 con 0x7f6be80aa7c0 2024-04-26T22:07:25.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.275+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000ab30 con 0x7f6be80aa7c0 2024-04-26T22:07:25.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.276+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6be001b070 con 0x7f6be80aa7c0 2024-04-26T22:07:25.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.277+0000 7f6bddffb640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6bcc06c9f0 0x7f6bcc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.282+0000 7f6becca5640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6bcc06c9f0 0x7f6bcc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.282+0000 7f6becca5640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6bcc06c9f0 0x7f6bcc06eeb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8034000 tx=0x7f6bd802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:25.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.283+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f6be008baa0 con 0x7f6be80aa7c0 2024-04-26T22:07:25.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.283+0000 7f6bd37fe640 1 -- 172.21.15.135:0/2875925965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc8005290 con 0x7f6be80aa7c0 2024-04-26T22:07:25.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.292+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6be0059250 con 0x7f6be80aa7c0 2024-04-26T22:07:25.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.743+0000 7f6bd37fe640 1 -- 172.21.15.135:0/2875925965 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bc8005580 con 0x7f6be80aa7c0 2024-04-26T22:07:25.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.745+0000 7f6bddffb640 1 -- 172.21.15.135:0/2875925965 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7f6be005d550 con 0x7f6be80aa7c0 2024-04-26T22:07:25.748 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:25.748 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:25.748 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:25.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6bcc06c9f0 msgr2=0x7f6bcc06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6bcc06c9f0 0x7f6bcc06eeb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8034000 tx=0x7f6bd802e000 comp rx=0 tx=0).stop 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 msgr2=0x7f6be81fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 0x7f6be81fcec0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6be000caf0 tx=0x7f6be0002090 comp rx=0 tx=0).stop 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 shutdown_connections 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6bcc06c9f0 0x7f6bcc06eeb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6be815b940 0x7f6be8157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6be80abdc0 0x7f6be8158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 --2- 172.21.15.135:0/2875925965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6be80aa7c0 0x7f6be81fcec0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 >> 172.21.15.135:0/2875925965 conn(0x7f6be80c80c0 msgr2=0x7f6be806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:25.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.749+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 shutdown_connections 2024-04-26T22:07:25.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.750+0000 7f6bee4a8640 1 -- 172.21.15.135:0/2875925965 wait complete. 2024-04-26T22:07:25.771 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:07:25.807 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:25.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.990+0000 7f3501952640 1 Processor -- start 2024-04-26T22:07:25.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.991+0000 7f3501952640 1 -- start start 2024-04-26T22:07:25.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.993+0000 7f3501952640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 0x7f34f4007970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.993+0000 7f3501952640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f34f404aea0 0x7f34f404b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:25.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.993+0000 7f3501952640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f404b830 con 0x7f34f40d45b0 2024-04-26T22:07:25.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.993+0000 7f3501952640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f34f40de800 con 0x7f34f404aea0 2024-04-26T22:07:25.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.993+0000 7f3501952640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f34f404aa10 con 0x7f34f40de2a0 2024-04-26T22:07:25.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.994+0000 7f34fbfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 0x7f34f4007970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:25.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.994+0000 7f34fbfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 0x7f34f4007970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48920/0 (socket says 172.21.15.135:48920) 2024-04-26T22:07:25.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.994+0000 7f34fbfff640 1 -- 172.21.15.135:0/1513279015 learned_addr learned my addr 172.21.15.135:0/1513279015 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:25.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.997+0000 7f34fbfff640 1 -- 172.21.15.135:0/1513279015 >> v2:172.21.15.135:3300/0 conn(0x7f34f404aea0 msgr2=0x7f34f404b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:25.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.997+0000 7f34fbfff640 1 --2- 172.21.15.135:0/1513279015 >> v2:172.21.15.135:3300/0 conn(0x7f34f404aea0 0x7f34f404b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:25.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.997+0000 7f34fbfff640 1 -- 172.21.15.135:0/1513279015 >> v1:172.21.15.135:6789/0 conn(0x7f34f40d45b0 legacy=0x7f34f400b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:25.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.997+0000 7f34fbfff640 1 -- 172.21.15.135:0/1513279015 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f404a0e0 con 0x7f34f40de2a0 2024-04-26T22:07:25.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:25.998+0000 7f34fbfff640 1 --2- 172.21.15.135:0/1513279015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 0x7f34f4007970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009870 tx=0x7f34e802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b70c97905b47bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:26.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.002+0000 7f34fb7fe640 1 -- 172.21.15.135:0/1513279015 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8037070 con 0x7f34f40de2a0 2024-04-26T22:07:26.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.002+0000 7f34fb7fe640 1 -- 172.21.15.135:0/1513279015 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8032440 con 0x7f34f40de2a0 2024-04-26T22:07:26.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.002+0000 7f34fb7fe640 1 -- 172.21.15.135:0/1513279015 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8032810 con 0x7f34f40de2a0 2024-04-26T22:07:26.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.004+0000 7f3501952640 1 -- 172.21.15.135:0/1513279015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 msgr2=0x7f34f4007970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:26.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.004+0000 7f3501952640 1 --2- 172.21.15.135:0/1513279015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 0x7f34f4007970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009870 tx=0x7f34e802a230 comp rx=0 tx=0).stop 2024-04-26T22:07:26.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.005+0000 7f3501952640 1 -- 172.21.15.135:0/1513279015 shutdown_connections 2024-04-26T22:07:26.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.005+0000 7f3501952640 1 --2- 172.21.15.135:0/1513279015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40de2a0 0x7f34f4007970 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.005+0000 7f3501952640 1 --2- 172.21.15.135:0/1513279015 >> v2:172.21.15.135:3300/0 conn(0x7f34f404aea0 0x7f34f404b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.005+0000 7f3501952640 1 -- 172.21.15.135:0/1513279015 >> 172.21.15.135:0/1513279015 conn(0x7f34f4048ac0 msgr2=0x7f34f4048ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:26.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.005+0000 7f3501952640 1 -- 172.21.15.135:0/1513279015 shutdown_connections 2024-04-26T22:07:26.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.005+0000 7f3501952640 1 -- 172.21.15.135:0/1513279015 wait complete. 2024-04-26T22:07:26.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.007+0000 7f3501952640 1 Processor -- start 2024-04-26T22:07:26.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.008+0000 7f3501952640 1 -- start start 2024-04-26T22:07:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.009+0000 7f3501952640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f404aea0 0x7f34f40cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.009+0000 7f3501952640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40d45b0 0x7f34f40dd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.009+0000 7f3501952640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34f40de2a0 0x7f34f40ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.009+0000 7f3501952640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34f40d4990 con 0x7f34f40de2a0 2024-04-26T22:07:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.010+0000 7f3501952640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34f404a6a0 con 0x7f34f404aea0 2024-04-26T22:07:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.010+0000 7f3501952640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f34f400b790 con 0x7f34f40d45b0 2024-04-26T22:07:26.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.010+0000 7f34fbfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40d45b0 0x7f34f40dd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:26.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.010+0000 7f34fbfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40d45b0 0x7f34f40dd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:48936/0 (socket says 172.21.15.135:48936) 2024-04-26T22:07:26.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.010+0000 7f34fbfff640 1 -- 172.21.15.135:0/3077862810 learned_addr learned my addr 172.21.15.135:0/3077862810 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:26.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.011+0000 7f3500950640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f404aea0 0x7f34f40cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.013+0000 7f3500950640 1 -- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40d45b0 msgr2=0x7f34f40dd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.013+0000 7f3500950640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40d45b0 0x7f34f40dd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.013+0000 7f3500950640 1 -- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34f40de2a0 msgr2=0x7f34f40ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.013+0000 7f3500950640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34f40de2a0 0x7f34f40ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.013+0000 7f3500950640 1 -- 172.21.15.135:0/3077862810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f41731f0 con 0x7f34f404aea0 2024-04-26T22:07:26.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.015+0000 7f3500950640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f404aea0 0x7f34f40cde20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c440 tx=0x7f34f000c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:26.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.020+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0017070 con 0x7f34f404aea0 2024-04-26T22:07:26.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.020+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00035f0 con 0x7f34f404aea0 2024-04-26T22:07:26.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.020+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00039e0 con 0x7f34f404aea0 2024-04-26T22:07:26.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.021+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f41734d0 con 0x7f34f404aea0 2024-04-26T22:07:26.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.021+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f4173a90 con 0x7f34f404aea0 2024-04-26T22:07:26.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.023+0000 7f34e6ffd640 1 -- 172.21.15.135:0/3077862810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f40057e0 con 0x7f34f404aea0 2024-04-26T22:07:26.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.025+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34f0003b80 con 0x7f34f404aea0 2024-04-26T22:07:26.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.026+0000 7f34f97fa640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f34e006c8f0 0x7f34e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:26.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.026+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7f34f008b8a0 con 0x7f34f404aea0 2024-04-26T22:07:26.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.026+0000 7f34fbfff640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f34e006c8f0 0x7f34e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:26.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.027+0000 7f34fbfff640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f34e006c8f0 0x7f34e006edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f34e802a270 tx=0x7f34e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:26.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.033+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34f0020020 con 0x7f34f404aea0 2024-04-26T22:07:26.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.496+0000 7f34e6ffd640 1 -- 172.21.15.135:0/3077862810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34f40079e0 con 0x7f34f404aea0 2024-04-26T22:07:26.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.498+0000 7f34f97fa640 1 -- 172.21.15.135:0/3077862810 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7f34f0059110 con 0x7f34f404aea0 2024-04-26T22:07:26.499 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:26.499 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:26.499 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f34e006c8f0 msgr2=0x7f34e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f34e006c8f0 0x7f34e006edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f34e802a270 tx=0x7f34e802e000 comp rx=0 tx=0).stop 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f404aea0 msgr2=0x7f34f40cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f404aea0 0x7f34f40cde20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c440 tx=0x7f34f000c840 comp rx=0 tx=0).stop 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 shutdown_connections 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f34e006c8f0 0x7f34e006edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34f40de2a0 0x7f34f40ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34f40d45b0 0x7f34f40dd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 --2- 172.21.15.135:0/3077862810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34f404aea0 0x7f34f40cde20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:26.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.503+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 >> 172.21.15.135:0/3077862810 conn(0x7f34f4048ac0 msgr2=0x7f34f400ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.507+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 shutdown_connections 2024-04-26T22:07:26.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:26.507+0000 7f3501952640 1 -- 172.21.15.135:0/3077862810 wait complete. 2024-04-26T22:07:26.530 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:07:27.772 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:27.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:27.999+0000 7faf6b322640 1 Processor -- start 2024-04-26T22:07:28.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:27.999+0000 7faf6b322640 1 -- start start 2024-04-26T22:07:28.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.000+0000 7faf6b322640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf640c2cd0 0x7faf641533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.000+0000 7faf6b322640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6406ff20 con 0x7faf64150200 2024-04-26T22:07:28.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.000+0000 7faf6b322640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6406f4e0 con 0x7faf640ab6a0 2024-04-26T22:07:28.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.000+0000 7faf6b322640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faf640c4bd0 con 0x7faf640c2cd0 2024-04-26T22:07:28.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.000+0000 7faf690be640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7faf64150200 0x7faf640ac140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:44172/0 (socket says 172.21.15.135:44172) 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.000+0000 7faf690be640 1 -- 172.21.15.135:0/961651908 learned_addr learned my addr 172.21.15.135:0/961651908 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438132739 0 0) 0x7faf5c001490 con 0x7faf64150200 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf50003500 con 0x7faf64150200 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3436993523 0 0) 0x7faf50003500 con 0x7faf64150200 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 >> v2:172.21.15.139:3300/0 conn(0x7faf640c2cd0 msgr2=0x7faf641533b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 --2- 172.21.15.135:0/961651908 >> v2:172.21.15.139:3300/0 conn(0x7faf640c2cd0 0x7faf641533b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 >> v1:172.21.15.139:6789/0 conn(0x7faf640ab6a0 legacy=0x7faf6415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf640c48f0 con 0x7faf64150200 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7faf5c005050 con 0x7faf64150200 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 >> v1:172.21.15.135:6789/0 conn(0x7faf64150200 legacy=0x7faf640ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf50003bb0 0x7faf50003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf500067e0 0x7faf50008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf50009510 0x7faf500099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf640c4bd0 con 0x7faf50009510 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf5000a2a0 con 0x7faf50003bb0 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.001+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faf5000a4a0 con 0x7faf500067e0 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.002+0000 7faf690be640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf50003bb0 0x7faf50003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.003+0000 7faf688bd640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf500067e0 0x7faf50008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.003+0000 7faf688bd640 1 -- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf50003bb0 msgr2=0x7faf50003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.003+0000 7faf688bd640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf50003bb0 0x7faf50003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.003+0000 7faf688bd640 1 -- 172.21.15.135:0/961651908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf50009510 msgr2=0x7faf500099c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.003+0000 7faf688bd640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf50009510 0x7faf500099c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.003+0000 7faf688bd640 1 -- 172.21.15.135:0/961651908 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf50003500 con 0x7faf500067e0 2024-04-26T22:07:28.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.004+0000 7faf698bf640 1 -- 172.21.15.135:0/961651908 reap_dead start 2024-04-26T22:07:28.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.005+0000 7faf688bd640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf500067e0 0x7faf50008c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf60009390 tx=0x7faf500037f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba1cf8c29fe03d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:28.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.006+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60059070 con 0x7faf500067e0 2024-04-26T22:07:28.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.006+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6004c9e0 con 0x7faf500067e0 2024-04-26T22:07:28.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.006+0000 7faf5bfff640 1 -- 172.21.15.135:0/961651908 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600093d0 con 0x7faf500067e0 2024-04-26T22:07:28.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.007+0000 7faf6b322640 1 -- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf500067e0 msgr2=0x7faf50008c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.007+0000 7faf6b322640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf500067e0 0x7faf50008c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf60009390 tx=0x7faf500037f0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.007+0000 7faf6b322640 1 -- 172.21.15.135:0/961651908 shutdown_connections 2024-04-26T22:07:28.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.007+0000 7faf6b322640 1 --2- 172.21.15.135:0/961651908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf500067e0 0x7faf50008c70 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf60009390 tx=0x7faf500037f0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.007+0000 7faf6b322640 1 -- 172.21.15.135:0/961651908 >> 172.21.15.135:0/961651908 conn(0x7faf640c82d0 msgr2=0x7faf640c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:28.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.010+0000 7faf6b322640 1 -- 172.21.15.135:0/961651908 shutdown_connections 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.014+0000 7faf6b322640 1 -- 172.21.15.135:0/961651908 wait complete. 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.015+0000 7faf6b322640 1 Processor -- start 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.015+0000 7faf6b322640 1 -- start start 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.016+0000 7faf6b322640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf64151dd0 0x7faf640731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.016+0000 7faf6b322640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf640c2790 0x7faf640736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.016+0000 7faf6b322640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 0x7faf64071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.016+0000 7faf6b322640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf64150760 con 0x7faf64151dd0 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.016+0000 7faf6b322640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf641505e0 con 0x7faf6414a8f0 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.016+0000 7faf6b322640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faf640c8dc0 con 0x7faf640c2790 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 0x7faf64071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 0x7faf64071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55866/0 (socket says 172.21.15.135:55866) 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 -- 172.21.15.135:0/3209567212 learned_addr learned my addr 172.21.15.135:0/3209567212 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 -- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf640c2790 msgr2=0x7faf640736e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf640c2790 0x7faf640736e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 -- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf64151dd0 msgr2=0x7faf640731a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf64151dd0 0x7faf640731a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.017+0000 7faf698bf640 1 -- 172.21.15.135:0/3209567212 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6414d0c0 con 0x7faf6414a8f0 2024-04-26T22:07:28.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.018+0000 7faf698bf640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 0x7faf64071d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf640768c0 tx=0x7faf5400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:28.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.018+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54007440 con 0x7faf6414a8f0 2024-04-26T22:07:28.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.019+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf64156bb0 con 0x7faf6414a8f0 2024-04-26T22:07:28.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.019+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf640c4300 con 0x7faf6414a8f0 2024-04-26T22:07:28.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.019+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf54001d20 con 0x7faf6414a8f0 2024-04-26T22:07:28.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.019+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf540049d0 con 0x7faf6414a8f0 2024-04-26T22:07:28.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.022+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf54004b70 con 0x7faf6414a8f0 2024-04-26T22:07:28.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.023+0000 7faf59ffb640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7faf4806c9c0 0x7faf4806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.023+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7faf5408ba30 con 0x7faf6414a8f0 2024-04-26T22:07:28.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.023+0000 7faf690be640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7faf4806c9c0 0x7faf4806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.024+0000 7faf690be640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7faf4806c9c0 0x7faf4806ee80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001630 tx=0x7faf5c008850 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:28.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.024+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf44005290 con 0x7faf6414a8f0 2024-04-26T22:07:28.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.033+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf54007600 con 0x7faf6414a8f0 2024-04-26T22:07:28.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.492+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf44005580 con 0x7faf6414a8f0 2024-04-26T22:07:28.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.494+0000 7faf59ffb640 1 -- 172.21.15.135:0/3209567212 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7faf540597e0 con 0x7faf6414a8f0 2024-04-26T22:07:28.495 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:28.495 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:28.495 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:28.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7faf4806c9c0 msgr2=0x7faf4806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7faf4806c9c0 0x7faf4806ee80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001630 tx=0x7faf5c008850 comp rx=0 tx=0).stop 2024-04-26T22:07:28.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 msgr2=0x7faf64071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 0x7faf64071d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf640768c0 tx=0x7faf5400c820 comp rx=0 tx=0).stop 2024-04-26T22:07:28.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 shutdown_connections 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7faf4806c9c0 0x7faf4806ee80 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001630 tx=0x7faf5c008850 comp rx=0 tx=0).stop 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf6414a8f0 0x7faf64071d30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf640c2790 0x7faf640736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 --2- 172.21.15.135:0/3209567212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf64151dd0 0x7faf640731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.499+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 >> 172.21.15.135:0/3209567212 conn(0x7faf640c82d0 msgr2=0x7faf640c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.500+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 shutdown_connections 2024-04-26T22:07:28.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.500+0000 7faf6b322640 1 -- 172.21.15.135:0/3209567212 wait complete. 2024-04-26T22:07:28.520 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:07:28.531 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:28.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.743+0000 7fdba2d25640 1 Processor -- start 2024-04-26T22:07:28.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.746+0000 7fdba2d25640 1 -- start start 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2d25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb94006ca0 0x7fdb940db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2d25640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdb940de5e0 0x7fdb940480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2d25640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb940dea60 con 0x7fdb940485f0 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2d25640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdb940dac20 con 0x7fdb940de5e0 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2d25640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdb940da5f0 con 0x7fdb94006ca0 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2524640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fdb940485f0 0x7fdb940d77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:54178/0 (socket says 172.21.15.135:54178) 2024-04-26T22:07:28.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba2524640 1 -- 172.21.15.135:0/785665687 learned_addr learned my addr 172.21.15.135:0/785665687 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.747+0000 7fdba1d23640 1 --2- 172.21.15.135:0/785665687 >> v2:172.21.15.139:3300/0 conn(0x7fdb94006ca0 0x7fdb940db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.748+0000 7fdba0d21640 1 -- 172.21.15.135:0/785665687 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981245377 0 0) 0x7fdb940dea60 con 0x7fdb940485f0 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.748+0000 7fdba0d21640 1 -- 172.21.15.135:0/785665687 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb88003520 con 0x7fdb940485f0 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.749+0000 7fdba1d23640 1 -- 172.21.15.135:0/785665687 >> v2:172.21.15.135:3300/0 conn(0x7fdb940de5e0 msgr2=0x7fdb940480b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.749+0000 7fdba1d23640 1 --2- 172.21.15.135:0/785665687 >> v2:172.21.15.135:3300/0 conn(0x7fdb940de5e0 0x7fdb940480b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.749+0000 7fdba1d23640 1 -- 172.21.15.135:0/785665687 >> v1:172.21.15.135:6789/0 conn(0x7fdb940485f0 legacy=0x7fdb940d77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.749+0000 7fdba1d23640 1 -- 172.21.15.135:0/785665687 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9400c830 con 0x7fdb94006ca0 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.751+0000 7fdba1d23640 1 --2- 172.21.15.135:0/785665687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb94006ca0 0x7fdb940db0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb940075d0 tx=0x7fdb9802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0c256546a7db71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.752+0000 7fdba0d21640 1 -- 172.21.15.135:0/785665687 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98037070 con 0x7fdb94006ca0 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.752+0000 7fdba0d21640 1 -- 172.21.15.135:0/785665687 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb98032440 con 0x7fdb94006ca0 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba2d25640 1 -- 172.21.15.135:0/785665687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb94006ca0 msgr2=0x7fdb940db0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba0d21640 1 -- 172.21.15.135:0/785665687 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98037070 con 0x7fdb94006ca0 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba2d25640 1 --2- 172.21.15.135:0/785665687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb94006ca0 0x7fdb940db0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb940075d0 tx=0x7fdb9802a180 comp rx=0 tx=0).stop 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba2d25640 1 -- 172.21.15.135:0/785665687 shutdown_connections 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba2d25640 1 --2- 172.21.15.135:0/785665687 >> v2:172.21.15.135:3300/0 conn(0x7fdb940de5e0 0x7fdb940480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba2d25640 1 --2- 172.21.15.135:0/785665687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb94006ca0 0x7fdb940db0b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.753+0000 7fdba2d25640 1 -- 172.21.15.135:0/785665687 >> 172.21.15.135:0/785665687 conn(0x7fdb94007b40 msgr2=0x7fdb94007f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:28.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.756+0000 7fdba2d25640 1 -- 172.21.15.135:0/785665687 shutdown_connections 2024-04-26T22:07:28.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.756+0000 7fdba2d25640 1 -- 172.21.15.135:0/785665687 wait complete. 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.757+0000 7fdba2d25640 1 Processor -- start 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.758+0000 7fdba2d25640 1 -- start start 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba2d25640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb94006ca0 0x7fdb9402fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba2d25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb940485f0 0x7fdb94031dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba2d25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 0x7fdb94032310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba2d25640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9400c470 con 0x7fdb94006ca0 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba2d25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940da8b0 con 0x7fdb940485f0 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba2d25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9400c5f0 con 0x7fdb940de5e0 2024-04-26T22:07:28.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.759+0000 7fdba1522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb940485f0 0x7fdb94031dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.760+0000 7fdba2524640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 0x7fdb94032310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.760+0000 7fdba2524640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 0x7fdb94032310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:51498/0 (socket says 172.21.15.135:51498) 2024-04-26T22:07:28.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.760+0000 7fdba2524640 1 -- 172.21.15.135:0/4282559838 learned_addr learned my addr 172.21.15.135:0/4282559838 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:28.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.760+0000 7fdba1d23640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb94006ca0 0x7fdb9402fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.761+0000 7fdba2524640 1 -- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb940485f0 msgr2=0x7fdb94031dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.761+0000 7fdba2524640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb940485f0 0x7fdb94031dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.761+0000 7fdba2524640 1 -- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb94006ca0 msgr2=0x7fdb9402fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:28.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.761+0000 7fdba2524640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb94006ca0 0x7fdb9402fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:28.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.761+0000 7fdba2524640 1 -- 172.21.15.135:0/4282559838 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb940ddf70 con 0x7fdb940de5e0 2024-04-26T22:07:28.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.761+0000 7fdba1d23640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb94006ca0 0x7fdb9402fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:28.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.764+0000 7fdba2524640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 0x7fdb94032310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c06bd80 tx=0x7fdb9c069f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:28.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.771+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c06f840 con 0x7fdb940de5e0 2024-04-26T22:07:28.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.771+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c06f9e0 con 0x7fdb940de5e0 2024-04-26T22:07:28.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.771+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c06fe10 con 0x7fdb940de5e0 2024-04-26T22:07:28.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.773+0000 7fdba2d25640 1 -- 172.21.15.135:0/4282559838 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb940d3fc0 con 0x7fdb940de5e0 2024-04-26T22:07:28.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.773+0000 7fdba2d25640 1 -- 172.21.15.135:0/4282559838 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb940468c0 con 0x7fdb940de5e0 2024-04-26T22:07:28.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.775+0000 7fdba2d25640 1 -- 172.21.15.135:0/4282559838 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94046b00 con 0x7fdb940de5e0 2024-04-26T22:07:28.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.782+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb9c0a0480 con 0x7fdb940de5e0 2024-04-26T22:07:28.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.782+0000 7fdb927fc640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb8006c9c0 0x7fdb8006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:28.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.783+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7fdb9c109570 con 0x7fdb940de5e0 2024-04-26T22:07:28.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.783+0000 7fdba1d23640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb8006c9c0 0x7fdb8006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:28.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.784+0000 7fdba1d23640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb8006c9c0 0x7fdb8006ee80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdb98034000 tx=0x7fdb9802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:28.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:28.790+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb9c0d7370 con 0x7fdb940de5e0 2024-04-26T22:07:29.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.300+0000 7fdba2d25640 1 -- 172.21.15.135:0/4282559838 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb940057e0 con 0x7fdb940de5e0 2024-04-26T22:07:29.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.303+0000 7fdb927fc640 1 -- 172.21.15.135:0/4282559838 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7fdb9c0dd030 con 0x7fdb940de5e0 2024-04-26T22:07:29.306 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:29.306 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:29.306 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:29.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 -- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb8006c9c0 msgr2=0x7fdb8006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:29.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb8006c9c0 0x7fdb8006ee80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdb98034000 tx=0x7fdb9802e000 comp rx=0 tx=0).stop 2024-04-26T22:07:29.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 -- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 msgr2=0x7fdb94032310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 0x7fdb94032310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c06bd80 tx=0x7fdb9c069f90 comp rx=0 tx=0).stop 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 -- 172.21.15.135:0/4282559838 shutdown_connections 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb8006c9c0 0x7fdb8006ee80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb940de5e0 0x7fdb94032310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb940485f0 0x7fdb94031dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 --2- 172.21.15.135:0/4282559838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb94006ca0 0x7fdb9402fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 -- 172.21.15.135:0/4282559838 >> 172.21.15.135:0/4282559838 conn(0x7fdb94007b40 msgr2=0x7fdb940d9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.308+0000 7fdb7ffff640 1 -- 172.21.15.135:0/4282559838 shutdown_connections 2024-04-26T22:07:29.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:29.309+0000 7fdb7ffff640 1 -- 172.21.15.135:0/4282559838 wait complete. 2024-04-26T22:07:29.332 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:07:30.517 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:07:30.520 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:07:30.521 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:30.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.762+0000 7fba0c7f7640 1 Processor -- start 2024-04-26T22:07:30.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.763+0000 7fba0c7f7640 1 -- start start 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.764+0000 7fba0c7f7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fba040c3650 0x7fba0406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.764+0000 7fba0c7f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fba040aa8c0 0x7fba040c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.764+0000 7fba0c7f7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba0415c0d0 con 0x7fba040c3f70 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.764+0000 7fba0c7f7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fba040ab4b0 con 0x7fba040c3650 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.764+0000 7fba0c7f7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fba0415bdf0 con 0x7fba040aa8c0 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.764+0000 7fba0a593640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fba040c3650 0x7fba0406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.765+0000 7fba0a593640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fba040c3650 0x7fba0406e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34252/0 (socket says 172.21.15.135:34252) 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.765+0000 7fba0a593640 1 -- 172.21.15.135:0/3520429054 learned_addr learned my addr 172.21.15.135:0/3520429054 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.765+0000 7fba0a593640 1 -- 172.21.15.135:0/3520429054 >> v2:172.21.15.139:3300/0 conn(0x7fba040aa8c0 msgr2=0x7fba040c3a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:30.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.765+0000 7fba0a593640 1 --2- 172.21.15.135:0/3520429054 >> v2:172.21.15.139:3300/0 conn(0x7fba040aa8c0 0x7fba040c3a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:30.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.765+0000 7fba0a593640 1 -- 172.21.15.135:0/3520429054 >> v1:172.21.15.135:6789/0 conn(0x7fba040c3f70 legacy=0x7fba040abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:30.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.765+0000 7fba0a593640 1 -- 172.21.15.135:0/3520429054 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba040ab750 con 0x7fba040c3650 2024-04-26T22:07:30.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.774+0000 7fba0a593640 1 --2- 172.21.15.135:0/3520429054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040c3650 0x7fba0406e760 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fba04071320 tx=0x7fba0002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f3cb9afbabf08e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.774+0000 7fba09591640 1 -- 172.21.15.135:0/3520429054 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00037070 con 0x7fba040c3650 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.775+0000 7fba09591640 1 -- 172.21.15.135:0/3520429054 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0002ae40 con 0x7fba040c3650 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.775+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3520429054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040c3650 msgr2=0x7fba0406e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.775+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3520429054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040c3650 0x7fba0406e760 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fba04071320 tx=0x7fba0002a130 comp rx=0 tx=0).stop 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.776+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3520429054 shutdown_connections 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.776+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3520429054 >> v2:172.21.15.139:3300/0 conn(0x7fba040aa8c0 0x7fba040c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.776+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3520429054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040c3650 0x7fba0406e760 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.776+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3520429054 >> 172.21.15.135:0/3520429054 conn(0x7fba040c82d0 msgr2=0x7fba040c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:30.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.776+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3520429054 shutdown_connections 2024-04-26T22:07:30.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.779+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3520429054 wait complete. 2024-04-26T22:07:30.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.780+0000 7fba0c7f7640 1 Processor -- start 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.780+0000 7fba0c7f7640 1 -- start start 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.781+0000 7fba0c7f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040aa8c0 0x7fba042025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.781+0000 7fba0c7f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040c3650 0x7fba040c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.781+0000 7fba0c7f7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 0x7fba04151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.781+0000 7fba0c7f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba040ac7e0 con 0x7fba040aa8c0 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.781+0000 7fba0c7f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fba040ac660 con 0x7fba040c3650 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.781+0000 7fba0c7f7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fba0415a6a0 con 0x7fba040c3f70 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 0x7fba04151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 0x7fba04151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:51510/0 (socket says 172.21.15.135:51510) 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 -- 172.21.15.135:0/3270061125 learned_addr learned my addr 172.21.15.135:0/3270061125 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba09d92640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040c3650 0x7fba040c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:30.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0a593640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040aa8c0 0x7fba042025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:30.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 -- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040c3650 msgr2=0x7fba040c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:30.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040c3650 0x7fba040c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:30.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 -- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040aa8c0 msgr2=0x7fba042025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:30.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040aa8c0 0x7fba042025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:30.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.782+0000 7fba0ad94640 1 -- 172.21.15.135:0/3270061125 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba041531b0 con 0x7fba040c3f70 2024-04-26T22:07:30.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.783+0000 7fba0ad94640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 0x7fba04151ec0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00b990 tx=0x7fb9fc00be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:30.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.784+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc019070 con 0x7fba040c3f70 2024-04-26T22:07:30.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.784+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00cba0 con 0x7fba040c3f70 2024-04-26T22:07:30.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.784+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0415d1a0 con 0x7fba040c3f70 2024-04-26T22:07:30.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.784+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba04201da0 con 0x7fba040c3f70 2024-04-26T22:07:30.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.785+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc014440 con 0x7fba040c3f70 2024-04-26T22:07:30.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.792+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ec005290 con 0x7fba040c3f70 2024-04-26T22:07:30.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.795+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9fc01c070 con 0x7fba040c3f70 2024-04-26T22:07:30.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.796+0000 7fb9faffd640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb9e806c9c0 0x7fb9e806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:30.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.796+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7fb9fc08bef0 con 0x7fba040c3f70 2024-04-26T22:07:30.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.798+0000 7fba0a593640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb9e806c9c0 0x7fb9e806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:30.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.799+0000 7fba0a593640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb9e806c9c0 0x7fb9e806ee80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba00034000 tx=0x7fba0002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:30.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:30.806+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9fc0596a0 con 0x7fba040c3f70 2024-04-26T22:07:31.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.258+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9ec005580 con 0x7fba040c3f70 2024-04-26T22:07:31.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.260+0000 7fb9faffd640 1 -- 172.21.15.135:0/3270061125 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7fb9fc05d9a0 con 0x7fba040c3f70 2024-04-26T22:07:31.263 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:31.263 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:31.264 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.266+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb9e806c9c0 msgr2=0x7fb9e806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.266+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb9e806c9c0 0x7fb9e806ee80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba00034000 tx=0x7fba0002e000 comp rx=0 tx=0).stop 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.266+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 msgr2=0x7fba04151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.266+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 0x7fba04151ec0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00b990 tx=0x7fb9fc00be50 comp rx=0 tx=0).stop 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.266+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 shutdown_connections 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb9e806c9c0 0x7fb9e806ee80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba040c3f70 0x7fba04151ec0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040c3650 0x7fba040c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 --2- 172.21.15.135:0/3270061125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba040aa8c0 0x7fba042025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 >> 172.21.15.135:0/3270061125 conn(0x7fba040c82d0 msgr2=0x7fba04159550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 shutdown_connections 2024-04-26T22:07:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.267+0000 7fba0c7f7640 1 -- 172.21.15.135:0/3270061125 wait complete. 2024-04-26T22:07:31.288 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-26T22:07:31.333 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:31.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.507+0000 7fd579919640 1 Processor -- start 2024-04-26T22:07:31.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.508+0000 7fd579919640 1 -- start start 2024-04-26T22:07:31.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.509+0000 7fd579919640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd57406d320 0x7fd57406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.509+0000 7fd579919640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c4730 0x7fd57415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.509+0000 7fd579919640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57406f1d0 con 0x7fd5740c5710 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.509+0000 7fd579919640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd5740c4c80 con 0x7fd57406d320 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.509+0000 7fd579919640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd57415f7c0 con 0x7fd5740c4730 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.510+0000 7fd5737fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd5740c5710 0x7fd57415f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:60248/0 (socket says 172.21.15.135:60248) 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.510+0000 7fd5737fe640 1 -- 172.21.15.135:0/3040958168 learned_addr learned my addr 172.21.15.135:0/3040958168 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:31.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.511+0000 7fd571ffb640 1 -- 172.21.15.135:0/3040958168 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371177494 0 0) 0x7fd57406f1d0 con 0x7fd5740c5710 2024-04-26T22:07:31.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.511+0000 7fd571ffb640 1 -- 172.21.15.135:0/3040958168 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd55c003520 con 0x7fd5740c5710 2024-04-26T22:07:31.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.512+0000 7fd572ffd640 1 --2- 172.21.15.135:0/3040958168 >> v2:172.21.15.135:3300/0 conn(0x7fd57406d320 0x7fd57406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.512+0000 7fd5727fc640 1 --2- 172.21.15.135:0/3040958168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c4730 0x7fd57415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.513+0000 7fd5727fc640 1 -- 172.21.15.135:0/3040958168 >> v2:172.21.15.135:3300/0 conn(0x7fd57406d320 msgr2=0x7fd57406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.513+0000 7fd5727fc640 1 --2- 172.21.15.135:0/3040958168 >> v2:172.21.15.135:3300/0 conn(0x7fd57406d320 0x7fd57406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.513+0000 7fd5727fc640 1 -- 172.21.15.135:0/3040958168 >> v1:172.21.15.139:6789/0 conn(0x7fd5740c5710 legacy=0x7fd57415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.513+0000 7fd5727fc640 1 -- 172.21.15.135:0/3040958168 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57406f4b0 con 0x7fd5740c4730 2024-04-26T22:07:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.513+0000 7fd572ffd640 1 --2- 172.21.15.135:0/3040958168 >> v2:172.21.15.135:3300/0 conn(0x7fd57406d320 0x7fd57406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:31.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.516+0000 7fd5727fc640 1 --2- 172.21.15.135:0/3040958168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c4730 0x7fd57415eb70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5680097a0 tx=0x7fd56802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32dc84fa0bcb15bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:31.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.517+0000 7fd571ffb640 1 -- 172.21.15.135:0/3040958168 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568037070 con 0x7fd5740c4730 2024-04-26T22:07:31.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.517+0000 7fd571ffb640 1 -- 172.21.15.135:0/3040958168 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56802adb0 con 0x7fd5740c4730 2024-04-26T22:07:31.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.517+0000 7fd571ffb640 1 -- 172.21.15.135:0/3040958168 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568032640 con 0x7fd5740c4730 2024-04-26T22:07:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.518+0000 7fd579919640 1 -- 172.21.15.135:0/3040958168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c4730 msgr2=0x7fd57415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.518+0000 7fd579919640 1 --2- 172.21.15.135:0/3040958168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c4730 0x7fd57415eb70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5680097a0 tx=0x7fd56802a060 comp rx=0 tx=0).stop 2024-04-26T22:07:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.519+0000 7fd579919640 1 -- 172.21.15.135:0/3040958168 shutdown_connections 2024-04-26T22:07:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.519+0000 7fd579919640 1 --2- 172.21.15.135:0/3040958168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c4730 0x7fd57415eb70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.519+0000 7fd579919640 1 --2- 172.21.15.135:0/3040958168 >> v2:172.21.15.135:3300/0 conn(0x7fd57406d320 0x7fd57406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.519+0000 7fd579919640 1 -- 172.21.15.135:0/3040958168 >> 172.21.15.135:0/3040958168 conn(0x7fd5740c91e0 msgr2=0x7fd5740c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:31.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.523+0000 7fd579919640 1 -- 172.21.15.135:0/3040958168 shutdown_connections 2024-04-26T22:07:31.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.526+0000 7fd579919640 1 -- 172.21.15.135:0/3040958168 wait complete. 2024-04-26T22:07:31.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.527+0000 7fd579919640 1 Processor -- start 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.528+0000 7fd579919640 1 -- start start 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.529+0000 7fd579919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57406d320 0x7fd57415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.529+0000 7fd579919640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5740c4730 0x7fd57415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.529+0000 7fd579919640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 0x7fd574144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.530+0000 7fd579919640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd57406f9c0 con 0x7fd5740c4730 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.530+0000 7fd579919640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd57406f840 con 0x7fd57406d320 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.530+0000 7fd579919640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd574158ed0 con 0x7fd5740c5710 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.530+0000 7fd5737fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 0x7fd574144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:31.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.530+0000 7fd5737fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 0x7fd574144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:51520/0 (socket says 172.21.15.135:51520) 2024-04-26T22:07:31.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.530+0000 7fd5737fe640 1 -- 172.21.15.135:0/3725419150 learned_addr learned my addr 172.21.15.135:0/3725419150 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:31.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5727fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5740c4730 0x7fd57415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:31.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5737fe640 1 -- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57406d320 msgr2=0x7fd57415bda0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:07:31.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5737fe640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57406d320 0x7fd57415bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5737fe640 1 -- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5740c4730 msgr2=0x7fd57415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:31.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5737fe640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5740c4730 0x7fd57415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:31.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5737fe640 1 -- 172.21.15.135:0/3725419150 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741467f0 con 0x7fd5740c5710 2024-04-26T22:07:31.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.532+0000 7fd5727fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5740c4730 0x7fd57415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:31.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.533+0000 7fd5737fe640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 0x7fd574144ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00cb40 tx=0x7fd56c00a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:31.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.533+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c017070 con 0x7fd5740c5710 2024-04-26T22:07:31.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.533+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c00aca0 con 0x7fd5740c5710 2024-04-26T22:07:31.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.533+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c012620 con 0x7fd5740c5710 2024-04-26T22:07:31.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.533+0000 7fd579919640 1 -- 172.21.15.135:0/3725419150 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741471b0 con 0x7fd5740c5710 2024-04-26T22:07:31.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.534+0000 7fd579919640 1 -- 172.21.15.135:0/3725419150 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd574147770 con 0x7fd5740c5710 2024-04-26T22:07:31.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.537+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd56c01a070 con 0x7fd5740c5710 2024-04-26T22:07:31.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.538+0000 7fd55bfff640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd56006c8f0 0x7fd56006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:31.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.538+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 7440+0+0 (secure 0 0 0) 0x7fd56c08bff0 con 0x7fd5740c5710 2024-04-26T22:07:31.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.538+0000 7fd579919640 1 -- 172.21.15.135:0/3725419150 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5740507a0 con 0x7fd5740c5710 2024-04-26T22:07:31.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.538+0000 7fd572ffd640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd56006c8f0 0x7fd56006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:31.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.539+0000 7fd572ffd640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd56006c8f0 0x7fd56006edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd564006fd0 tx=0x7fd564008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:31.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.547+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd56c059d70 con 0x7fd5740c5710 2024-04-26T22:07:32.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:31.999+0000 7fd579919640 1 -- 172.21.15.135:0/3725419150 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5740c54f0 con 0x7fd5740c5710 2024-04-26T22:07:32.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.000+0000 7fd55bfff640 1 -- 172.21.15.135:0/3725419150 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+6503 (secure 0 0 0) 0x7fd56c059670 con 0x7fd5740c5710 2024-04-26T22:07:32.001 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:32.002 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:06:34.102512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5105":{"gid":5105,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6835/3733666635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3733666635},{"type":"v1","addr":"172.21.15.139:6835","nonce":3733666635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5105,"qdb_cluster":[5105]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 58 2024-04-26T22:07:32.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 -- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd56006c8f0 msgr2=0x7fd56006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd56006c8f0 0x7fd56006edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd564006fd0 tx=0x7fd564008040 comp rx=0 tx=0).stop 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 -- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 msgr2=0x7fd574144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 0x7fd574144ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00cb40 tx=0x7fd56c00a060 comp rx=0 tx=0).stop 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 -- 172.21.15.135:0/3725419150 shutdown_connections 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fd56006c8f0 0x7fd56006edb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5740c5710 0x7fd574144ac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5740c4730 0x7fd57415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 --2- 172.21.15.135:0/3725419150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57406d320 0x7fd57415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:32.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.005+0000 7fd55a7fc640 1 -- 172.21.15.135:0/3725419150 >> 172.21.15.135:0/3725419150 conn(0x7fd5740c91e0 msgr2=0x7fd57406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:32.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.012+0000 7fd55a7fc640 1 -- 172.21.15.135:0/3725419150 shutdown_connections 2024-04-26T22:07:32.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:32.012+0000 7fd55a7fc640 1 -- 172.21.15.135:0/3725419150 wait complete. 2024-04-26T22:07:32.036 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:07:33.289 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:33.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.433+0000 7ff1db94e640 1 Processor -- start 2024-04-26T22:07:33.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.433+0000 7ff1db94e640 1 -- start start 2024-04-26T22:07:33.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.435+0000 7ff1db94e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1d40bfbb0 0x7ff1d415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:33.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.435+0000 7ff1db94e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c6e00 0x7ff1d4158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:33.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.435+0000 7ff1db94e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d40cbde0 con 0x7ff1d40c90c0 2024-04-26T22:07:33.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.436+0000 7ff1d96ea640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff1d40c90c0 0x7ff1d40cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:60256/0 (socket says 172.21.15.135:60256) 2024-04-26T22:07:33.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.436+0000 7ff1d96ea640 1 -- 172.21.15.135:0/2521474038 learned_addr learned my addr 172.21.15.135:0/2521474038 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:33.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.436+0000 7ff1d9eeb640 1 --2- 172.21.15.135:0/2521474038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c6e00 0x7ff1d4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:33.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.436+0000 7ff1d8ee9640 1 --2- 172.21.15.135:0/2521474038 >> v2:172.21.15.139:3300/0 conn(0x7ff1d40bfbb0 0x7ff1d415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.436+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d415c150 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.436+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d40ca380 con 0x7ff1d40c6e00 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.437+0000 7ff1d9eeb640 1 -- 172.21.15.135:0/2521474038 >> v2:172.21.15.139:3300/0 conn(0x7ff1d40bfbb0 msgr2=0x7ff1d415bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.437+0000 7ff1d9eeb640 1 --2- 172.21.15.135:0/2521474038 >> v2:172.21.15.139:3300/0 conn(0x7ff1d40bfbb0 0x7ff1d415bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.437+0000 7ff1d9eeb640 1 -- 172.21.15.135:0/2521474038 >> v1:172.21.15.139:6789/0 conn(0x7ff1d40c90c0 legacy=0x7ff1d40cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.437+0000 7ff1d9eeb640 1 -- 172.21.15.135:0/2521474038 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d40cc0c0 con 0x7ff1d40c6e00 2024-04-26T22:07:33.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.438+0000 7ff1d9eeb640 1 --2- 172.21.15.135:0/2521474038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c6e00 0x7ff1d4158510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009d90 tx=0x7ff1cc02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6deeccf779eb9f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:33.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.438+0000 7ff1cbfff640 1 -- 172.21.15.135:0/2521474038 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc037070 con 0x7ff1d40c6e00 2024-04-26T22:07:33.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.438+0000 7ff1cbfff640 1 -- 172.21.15.135:0/2521474038 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc032640 con 0x7ff1d40c6e00 2024-04-26T22:07:33.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1cbfff640 1 -- 172.21.15.135:0/2521474038 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc032a10 con 0x7ff1d40c6e00 2024-04-26T22:07:33.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c6e00 msgr2=0x7ff1d4158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1db94e640 1 --2- 172.21.15.135:0/2521474038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c6e00 0x7ff1d4158510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009d90 tx=0x7ff1cc02a440 comp rx=0 tx=0).stop 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 shutdown_connections 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1db94e640 1 --2- 172.21.15.135:0/2521474038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c6e00 0x7ff1d4158510 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1db94e640 1 --2- 172.21.15.135:0/2521474038 >> v2:172.21.15.139:3300/0 conn(0x7ff1d40bfbb0 0x7ff1d415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.439+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 >> 172.21.15.135:0/2521474038 conn(0x7ff1d40ae010 msgr2=0x7ff1d40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.440+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 shutdown_connections 2024-04-26T22:07:33.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.440+0000 7ff1db94e640 1 -- 172.21.15.135:0/2521474038 wait complete. 2024-04-26T22:07:33.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.441+0000 7ff1db94e640 1 Processor -- start 2024-04-26T22:07:33.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.442+0000 7ff1db94e640 1 -- start start 2024-04-26T22:07:33.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1db94e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 0x7ff1d415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:33.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1db94e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d40c6e00 0x7ff1d4150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:33.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1db94e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c90c0 0x7ff1d4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1d96ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 0x7ff1d415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1db94e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d40c94a0 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1db94e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d40cb4e0 con 0x7ff1d40c6e00 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1db94e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d40c9620 con 0x7ff1d40c90c0 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1d96ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 0x7ff1d415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34312/0 (socket says 172.21.15.135:34312) 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1d96ea640 1 -- 172.21.15.135:0/3484475817 learned_addr learned my addr 172.21.15.135:0/3484475817 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:33.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1d8ee9640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d40c6e00 0x7ff1d4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.443+0000 7ff1d9eeb640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c90c0 0x7ff1d4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.444+0000 7ff1d96ea640 1 -- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c90c0 msgr2=0x7ff1d4150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.444+0000 7ff1d96ea640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c90c0 0x7ff1d4150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.444+0000 7ff1d96ea640 1 -- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d40c6e00 msgr2=0x7ff1d4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.444+0000 7ff1d96ea640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d40c6e00 0x7ff1d4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.444+0000 7ff1d96ea640 1 -- 172.21.15.135:0/3484475817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d415ed40 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.445+0000 7ff1d96ea640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 0x7ff1d415a3b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c780 tx=0x7ff1d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:33.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.445+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017070 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.445+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d0005ce0 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.445+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d41512d0 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.446+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d4151570 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.446+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0007270 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.448+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1d001a070 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.448+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b4005290 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.449+0000 7ff1c9ffb640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1b806c9f0 0x7ff1b806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:33.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.449+0000 7ff1d8ee9640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1b806c9f0 0x7ff1b806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:33.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.450+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 7526+0+0 (secure 0 0 0) 0x7ff1d008b680 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.452+0000 7ff1d8ee9640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1b806c9f0 0x7ff1b806eeb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4006fd0 tx=0x7ff1c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:33.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.457+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1d0058d20 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.908+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1b4005580 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.912+0000 7ff1c9ffb640 1 -- 172.21.15.135:0/3484475817 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5667 (secure 0 0 0) 0x7ff1d005f070 con 0x7ff1d40bfbb0 2024-04-26T22:07:33.913 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:33.913 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:32.835333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:33.913 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 59 2024-04-26T22:07:33.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.916+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1b806c9f0 msgr2=0x7ff1b806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:33.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.916+0000 7ff1db94e640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1b806c9f0 0x7ff1b806eeb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4006fd0 tx=0x7ff1c4008040 comp rx=0 tx=0).stop 2024-04-26T22:07:33.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.916+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 msgr2=0x7ff1d415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:33.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.916+0000 7ff1db94e640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 0x7ff1d415a3b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c780 tx=0x7ff1d000cc40 comp rx=0 tx=0).stop 2024-04-26T22:07:33.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 shutdown_connections 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff1b806c9f0 0x7ff1b806eeb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1d40c90c0 0x7ff1d4150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d40c6e00 0x7ff1d4150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 --2- 172.21.15.135:0/3484475817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d40bfbb0 0x7ff1d415a3b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 >> 172.21.15.135:0/3484475817 conn(0x7ff1d40ae010 msgr2=0x7ff1d40b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 shutdown_connections 2024-04-26T22:07:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:33.917+0000 7ff1db94e640 1 -- 172.21.15.135:0/3484475817 wait complete. 2024-04-26T22:07:33.944 INFO:tasks.mds_thrash.fs.[a]:mds.b down, removed from mdsmap 2024-04-26T22:07:33.944 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-04-26T22:07:33.944 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:34.036 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:34.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.139+0000 7efdbeb78640 1 Processor -- start 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.139+0000 7efdbeb78640 1 -- start start 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbeb78640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efdb80c3ee0 0x7efdb8167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbeb78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb81679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbeb78640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb806d530 con 0x7efdb80c5880 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbeb78640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efdb806db60 con 0x7efdb80c3ee0 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbeb78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efdb80748c0 con 0x7efdb80c4730 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdb7fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7efdb80c5880 0x7efdb80ac7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:54186/0 (socket says 172.21.15.135:54186) 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdb7fff640 1 -- 172.21.15.135:0/3591386481 learned_addr learned my addr 172.21.15.135:0/3591386481 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbc914640 1 --2- 172.21.15.135:0/3591386481 >> v2:172.21.15.135:3300/0 conn(0x7efdb80c3ee0 0x7efdb8167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.140+0000 7efdbd115640 1 --2- 172.21.15.135:0/3591386481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb81679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.141+0000 7efdbc914640 1 -- 172.21.15.135:0/3591386481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 msgr2=0x7efdb81679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:34.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.141+0000 7efdbc914640 1 --2- 172.21.15.135:0/3591386481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb81679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.141+0000 7efdbc914640 1 -- 172.21.15.135:0/3591386481 >> v1:172.21.15.135:6789/0 conn(0x7efdb80c5880 legacy=0x7efdb80ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.141+0000 7efdbc914640 1 -- 172.21.15.135:0/3591386481 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb8074ba0 con 0x7efdb80c3ee0 2024-04-26T22:07:34.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.142+0000 7efdbc914640 1 --2- 172.21.15.135:0/3591386481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb8167480 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efda800d240 tx=0x7efda8032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c83cf3ad125c904 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:34.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.143+0000 7efdb77fe640 1 -- 172.21.15.135:0/3591386481 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8039730 con 0x7efdb80c3ee0 2024-04-26T22:07:34.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.143+0000 7efdb77fe640 1 -- 172.21.15.135:0/3591386481 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda80398d0 con 0x7efdb80c3ee0 2024-04-26T22:07:34.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.144+0000 7efdbeb78640 1 -- 172.21.15.135:0/3591386481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 msgr2=0x7efdb8167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.144+0000 7efdbeb78640 1 --2- 172.21.15.135:0/3591386481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb8167480 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efda800d240 tx=0x7efda8032980 comp rx=0 tx=0).stop 2024-04-26T22:07:34.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.145+0000 7efdbeb78640 1 -- 172.21.15.135:0/3591386481 shutdown_connections 2024-04-26T22:07:34.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.145+0000 7efdbeb78640 1 --2- 172.21.15.135:0/3591386481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb81679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.145+0000 7efdbeb78640 1 --2- 172.21.15.135:0/3591386481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb8167480 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.145+0000 7efdbeb78640 1 -- 172.21.15.135:0/3591386481 >> 172.21.15.135:0/3591386481 conn(0x7efdb80c4d60 msgr2=0x7efdb80c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:34.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.145+0000 7efdbeb78640 1 -- 172.21.15.135:0/3591386481 shutdown_connections 2024-04-26T22:07:34.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.145+0000 7efdbeb78640 1 -- 172.21.15.135:0/3591386481 wait complete. 2024-04-26T22:07:34.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.150+0000 7efdbeb78640 1 Processor -- start 2024-04-26T22:07:34.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.150+0000 7efdbeb78640 1 -- start start 2024-04-26T22:07:34.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.151+0000 7efdbeb78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb820ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.151+0000 7efdbeb78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.151+0000 7efdbeb78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb80c5880 0x7efdb808c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.151+0000 7efdbeb78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efdb80c42c0 con 0x7efdb80c3ee0 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.151+0000 7efdbeb78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8074570 con 0x7efdb80c5880 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.151+0000 7efdbeb78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efdb80c4440 con 0x7efdb80c4730 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.152+0000 7efdbd115640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb80c5880 0x7efdb808c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.153+0000 7efdbd115640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb80c5880 0x7efdb808c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:41352/0 (socket says 172.21.15.135:41352) 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.153+0000 7efdbd115640 1 -- 172.21.15.135:0/4231129920 learned_addr learned my addr 172.21.15.135:0/4231129920 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.154+0000 7efdb7fff640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.155+0000 7efdbc914640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb820ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.155+0000 7efdbc914640 1 -- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 msgr2=0x7efdb80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.155+0000 7efdbc914640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb80c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.155+0000 7efdbc914640 1 -- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb80c5880 msgr2=0x7efdb808c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.155+0000 7efdbc914640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb80c5880 0x7efdb808c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.156+0000 7efdbc914640 1 -- 172.21.15.135:0/4231129920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb8071710 con 0x7efdb80c3ee0 2024-04-26T22:07:34.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.156+0000 7efdbc914640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb820ddd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efda803b000 tx=0x7efda80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:34.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.157+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8049070 con 0x7efdb80c3ee0 2024-04-26T22:07:34.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.157+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb820d610 con 0x7efdb80c3ee0 2024-04-26T22:07:34.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.157+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb808ce00 con 0x7efdb80c3ee0 2024-04-26T22:07:34.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.161+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda80165e0 con 0x7efdb80c3ee0 2024-04-26T22:07:34.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.161+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8039e60 con 0x7efdb80c3ee0 2024-04-26T22:07:34.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.163+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efda804c070 con 0x7efdb80c3ee0 2024-04-26T22:07:34.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.164+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd98005290 con 0x7efdb80c3ee0 2024-04-26T22:07:34.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.164+0000 7efdb57fa640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda406c9c0 0x7efda406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.165+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 7526+0+0 (secure 0 0 0) 0x7efda80ad420 con 0x7efdb80c3ee0 2024-04-26T22:07:34.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.165+0000 7efdb7fff640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda406c9c0 0x7efda406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.166+0000 7efdb7fff640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda406c9c0 0x7efda406ee80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdb8165a60 tx=0x7efdb00017d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:34.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.173+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efda807adc0 con 0x7efdb80c3ee0 2024-04-26T22:07:34.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.229+0000 7fb3239c1640 1 Processor -- start 2024-04-26T22:07:34.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.230+0000 7fb3239c1640 1 -- start start 2024-04-26T22:07:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.231+0000 7fb3239c1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb31c153570 0x7fb31c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.231+0000 7fb3239c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.231+0000 7fb3239c1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb31c1539f0 con 0x7fb31c14cae0 2024-04-26T22:07:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.231+0000 7fb3239c1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb31c06da70 con 0x7fb31c153570 2024-04-26T22:07:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.231+0000 7fb3239c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c0ab9d0 con 0x7fb31c0c3c70 2024-04-26T22:07:34.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb31c153570 0x7fb31c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb31c153570 0x7fb31c06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:41356/0 (socket says 172.21.15.135:41356) 2024-04-26T22:07:34.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 -- 172.21.15.135:0/3891749456 learned_addr learned my addr 172.21.15.135:0/3891749456 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:34.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb321f5e640 1 --2- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 -- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 msgr2=0x7fb31c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:34.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 --2- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 -- 172.21.15.135:0/3891749456 >> v1:172.21.15.135:6789/0 conn(0x7fb31c14cae0 legacy=0x7fb31c0abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.232+0000 7fb320f5c640 1 -- 172.21.15.135:0/3891749456 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0ad3a0 con 0x7fb31c153570 2024-04-26T22:07:34.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.237+0000 7fb320f5c640 1 --2- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 0x7fb31c06d530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb31000a910 tx=0x7fb31002e8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d003e96a24fa6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:34.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.241+0000 7fb31bfff640 1 -- 172.21.15.135:0/3891749456 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310031070 con 0x7fb31c153570 2024-04-26T22:07:34.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.241+0000 7fb31bfff640 1 -- 172.21.15.135:0/3891749456 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb310035750 con 0x7fb31c153570 2024-04-26T22:07:34.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.241+0000 7fb31bfff640 1 -- 172.21.15.135:0/3891749456 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310035b20 con 0x7fb31c153570 2024-04-26T22:07:34.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.242+0000 7fb3239c1640 1 -- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 msgr2=0x7fb31c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.242+0000 7fb3239c1640 1 --2- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 0x7fb31c06d530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb31000a910 tx=0x7fb31002e8e0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.245+0000 7fb3239c1640 1 -- 172.21.15.135:0/3891749456 shutdown_connections 2024-04-26T22:07:34.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.245+0000 7fb3239c1640 1 --2- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.245+0000 7fb3239c1640 1 --2- 172.21.15.135:0/3891749456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 0x7fb31c06d530 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.245+0000 7fb3239c1640 1 -- 172.21.15.135:0/3891749456 >> 172.21.15.135:0/3891749456 conn(0x7fb31c0c4d60 msgr2=0x7fb31c0c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:34.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.249+0000 7fb3239c1640 1 -- 172.21.15.135:0/3891749456 shutdown_connections 2024-04-26T22:07:34.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.250+0000 7fb3239c1640 1 -- 172.21.15.135:0/3891749456 wait complete. 2024-04-26T22:07:34.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.252+0000 7fb3239c1640 1 Processor -- start 2024-04-26T22:07:34.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.252+0000 7fb3239c1640 1 -- start start 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.253+0000 7fb3239c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c1e8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.253+0000 7fb3239c1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 0x7fb31c074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.253+0000 7fb3239c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 0x7fb31c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.253+0000 7fb3239c1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0ab660 con 0x7fb31c14cae0 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.253+0000 7fb3239c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0ab380 con 0x7fb31c153570 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.253+0000 7fb3239c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c14cec0 con 0x7fb31c0c3c70 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.254+0000 7fb320f5c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 0x7fb31c074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.254+0000 7fb320f5c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 0x7fb31c074440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34334/0 (socket says 172.21.15.135:34334) 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.254+0000 7fb320f5c640 1 -- 172.21.15.135:0/2345083596 learned_addr learned my addr 172.21.15.135:0/2345083596 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:34.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.255+0000 7fb320f5c640 1 -- 172.21.15.135:0/2345083596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 msgr2=0x7fb31c1e8200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:34.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.255+0000 7fb320f5c640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c1e8200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.255+0000 7fb320f5c640 1 -- 172.21.15.135:0/2345083596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 msgr2=0x7fb31c151ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:07:34.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.255+0000 7fb320f5c640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 0x7fb31c151ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.255+0000 7fb320f5c640 1 -- 172.21.15.135:0/2345083596 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c1ec330 con 0x7fb31c14cae0 2024-04-26T22:07:34.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.255+0000 7fb320f5c640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 0x7fb31c074440 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:34.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.256+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310046020 con 0x7fb31c14cae0 2024-04-26T22:07:34.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.256+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb310038dc0 con 0x7fb31c14cae0 2024-04-26T22:07:34.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.256+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3100426b0 con 0x7fb31c14cae0 2024-04-26T22:07:34.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.257+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1ec610 con 0x7fb31c14cae0 2024-04-26T22:07:34.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.257+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c14ecc0 con 0x7fb31c14cae0 2024-04-26T22:07:34.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.267+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb310049020 con 0x7fb31c14cae0 2024-04-26T22:07:34.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.267+0000 7fb319ffb640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb30806c9c0 0x7fb30806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:34.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.268+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 7526+0+0 (secure 0 0 0) 0x7fb3100b19d0 con 0x7fb31c14cae0 2024-04-26T22:07:34.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.269+0000 7fb32175d640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb30806c9c0 0x7fb30806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:34.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.270+0000 7fb3037fe640 1 -- 172.21.15.135:0/2345083596 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c0506c0 con 0x7fb31c14cae0 2024-04-26T22:07:34.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.280+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb310050030 con 0x7fb31c14cae0 2024-04-26T22:07:34.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.281+0000 7fb32175d640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb30806c9c0 0x7fb30806ee80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3140012e0 tx=0x7fb314007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:34.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.668+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd98005580 con 0x7efdb80c3ee0 2024-04-26T22:07:34.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.670+0000 7efdb57fa640 1 -- 172.21.15.135:0/4231129920 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5667 (secure 0 0 0) 0x7efda807f0c0 con 0x7efdb80c3ee0 2024-04-26T22:07:34.671 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:34.671 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:32.835333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:34.672 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 59 2024-04-26T22:07:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda406c9c0 msgr2=0x7efda406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda406c9c0 0x7efda406ee80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdb8165a60 tx=0x7efdb00017d0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 msgr2=0x7efdb820ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb820ddd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efda803b000 tx=0x7efda80046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 shutdown_connections 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda406c9c0 0x7efda406ee80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.674+0000 7efdbeb78640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb80c5880 0x7efdb808c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.675+0000 7efdbeb78640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb80c4730 0x7efdb80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.675+0000 7efdbeb78640 1 --2- 172.21.15.135:0/4231129920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdb80c3ee0 0x7efdb820ddd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.675+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 >> 172.21.15.135:0/4231129920 conn(0x7efdb80c4d60 msgr2=0x7efdb8165fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.679+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 shutdown_connections 2024-04-26T22:07:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.679+0000 7efdbeb78640 1 -- 172.21.15.135:0/4231129920 wait complete. 2024-04-26T22:07:34.699 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 59, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 59, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5210": { "addr": "172.21.15.135:6837/3969385858", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 3969385858, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 3969385858, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5210, "incarnation": 59, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:07:32.835333+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5210 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 57, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5419": { "addr": "172.21.15.135:6835/1037298395", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 1037298395, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 1037298395, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5419, "incarnation": 54, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:07:14.544129+0000", "qdb_cluster": [ 5419 ], "qdb_leader": 5419, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5419 } } } ], "standbys": [ { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.862+0000 7fb3037fe640 1 -- 172.21.15.135:0/2345083596 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31c0ac640 con 0x7fb31c14cae0 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.863+0000 7fb319ffb640 1 -- 172.21.15.135:0/2345083596 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5667 (secure 0 0 0) 0x7fb31007f750 con 0x7fb31c14cae0 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:32.835333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 59 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.867+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb30806c9c0 msgr2=0x7fb30806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.867+0000 7fb3239c1640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb30806c9c0 0x7fb30806ee80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3140012e0 tx=0x7fb314007d30 comp rx=0 tx=0).stop 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.867+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 msgr2=0x7fb31c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.867+0000 7fb3239c1640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 0x7fb31c074440 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb310004730 comp rx=0 tx=0).stop 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.868+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 shutdown_connections 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.869+0000 7fb3239c1640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb30806c9c0 0x7fb30806ee80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.869+0000 7fb3239c1640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb31c153570 0x7fb31c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.869+0000 7fb3239c1640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb31c14cae0 0x7fb31c074440 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.869+0000 7fb3239c1640 1 --2- 172.21.15.135:0/2345083596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb31c0c3c70 0x7fb31c1e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.869+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 >> 172.21.15.135:0/2345083596 conn(0x7fb31c0c4d60 msgr2=0x7fb31c149a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.870+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 shutdown_connections 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:34.870+0000 7fb3239c1640 1 -- 172.21.15.135:0/2345083596 wait complete. 2024-04-26T22:07:34.895 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:07:36.700 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:36.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.812+0000 7fa4a6a47640 1 Processor -- start 2024-04-26T22:07:36.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.812+0000 7fa4a6a47640 1 -- start start 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.812+0000 7fa4a6a47640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4a01510c0 0x7fa4a0072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.812+0000 7fa4a6a47640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a00c4b50 con 0x7fa4a00734d0 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.812+0000 7fa4a6a47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a01514a0 con 0x7fa4a00c4160 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.812+0000 7fa4a6a47640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa4a00c50d0 con 0x7fa4a01510c0 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa4a00734d0 0x7fa4a006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:54200/0 (socket says 172.21.15.135:54200) 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49f7fe640 1 -- 172.21.15.135:0/2459413454 learned_addr learned my addr 172.21.15.135:0/2459413454 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:36.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49ffff640 1 --2- 172.21.15.135:0/2459413454 >> v2:172.21.15.139:3300/0 conn(0x7fa4a01510c0 0x7fa4a0072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49effd640 1 -- 172.21.15.135:0/2459413454 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009312711 0 0) 0x7fa4a00c4b50 con 0x7fa4a00734d0 2024-04-26T22:07:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49effd640 1 -- 172.21.15.135:0/2459413454 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa488003520 con 0x7fa4a00734d0 2024-04-26T22:07:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49ffff640 1 -- 172.21.15.135:0/2459413454 >> v1:172.21.15.139:6789/0 conn(0x7fa4a00c4160 legacy=0x7fa4a015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49effd640 1 -- 172.21.15.135:0/2459413454 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442703827 0 0) 0x7fa4a01514a0 con 0x7fa4a00c4160 2024-04-26T22:07:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49ffff640 1 -- 172.21.15.135:0/2459413454 >> v1:172.21.15.135:6789/0 conn(0x7fa4a00734d0 legacy=0x7fa4a006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:36.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.813+0000 7fa49ffff640 1 -- 172.21.15.135:0/2459413454 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a0154920 con 0x7fa4a01510c0 2024-04-26T22:07:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.815+0000 7fa49ffff640 1 --2- 172.21.15.135:0/2459413454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a0072f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00c5250 tx=0x7fa49002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63fa9ad3a3acaa6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.815+0000 7fa49effd640 1 -- 172.21.15.135:0/2459413454 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490037070 con 0x7fa4a01510c0 2024-04-26T22:07:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.815+0000 7fa49effd640 1 -- 172.21.15.135:0/2459413454 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49002ae50 con 0x7fa4a01510c0 2024-04-26T22:07:36.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.815+0000 7fa49effd640 1 -- 172.21.15.135:0/2459413454 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490032640 con 0x7fa4a01510c0 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 -- 172.21.15.135:0/2459413454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 msgr2=0x7fa4a0072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/2459413454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a0072f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00c5250 tx=0x7fa49002a100 comp rx=0 tx=0).stop 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 -- 172.21.15.135:0/2459413454 shutdown_connections 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/2459413454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a0072f90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 -- 172.21.15.135:0/2459413454 >> 172.21.15.135:0/2459413454 conn(0x7fa4a00c5ce0 msgr2=0x7fa4a00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 -- 172.21.15.135:0/2459413454 shutdown_connections 2024-04-26T22:07:36.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.816+0000 7fa4a6a47640 1 -- 172.21.15.135:0/2459413454 wait complete. 2024-04-26T22:07:36.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.817+0000 7fa4a6a47640 1 Processor -- start 2024-04-26T22:07:36.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.817+0000 7fa4a6a47640 1 -- start start 2024-04-26T22:07:36.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a6a47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 0x7fa4a00c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:36.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a6a47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a00aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a6a47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4a00acda0 0x7fa4a00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a6a47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a00c5920 con 0x7fa4a00c4160 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a6a47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a00c5400 con 0x7fa4a01510c0 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a6a47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a006f500 con 0x7fa4a00acda0 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 0x7fa4a00c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 0x7fa4a00c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34348/0 (socket says 172.21.15.135:34348) 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 -- 172.21.15.135:0/3740905834 learned_addr learned my addr 172.21.15.135:0/3740905834 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49f7fe640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a00aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa4a4fe4640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4a00acda0 0x7fa4a00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 -- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4a00acda0 msgr2=0x7fa4a00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4a00acda0 0x7fa4a00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 -- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 msgr2=0x7fa4a00aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a00aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:36.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 -- 172.21.15.135:0/3740905834 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a0144630 con 0x7fa4a00c4160 2024-04-26T22:07:36.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.818+0000 7fa49ffff640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 0x7fa4a00c2910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa49002a140 tx=0x7fa4900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:36.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.819+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490040020 con 0x7fa4a00c4160 2024-04-26T22:07:36.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.819+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa490038040 con 0x7fa4a00c4160 2024-04-26T22:07:36.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.819+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a00ad6d0 con 0x7fa4a00c4160 2024-04-26T22:07:36.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.819+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490037070 con 0x7fa4a00c4160 2024-04-26T22:07:36.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.819+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a014d590 con 0x7fa4a00c4160 2024-04-26T22:07:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.820+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a0068dc0 con 0x7fa4a00c4160 2024-04-26T22:07:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.820+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa490043020 con 0x7fa4a00c4160 2024-04-26T22:07:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.821+0000 7fa49cff9640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa48c06c8f0 0x7fa48c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.821+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 7526+0+0 (secure 0 0 0) 0x7fa4900ac000 con 0x7fa4a00c4160 2024-04-26T22:07:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.821+0000 7fa49f7fe640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa48c06c8f0 0x7fa48c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.821+0000 7fa49f7fe640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa48c06c8f0 0x7fa48c06edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00c3180 tx=0x7fa494009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:36.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:36.827+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa49004a070 con 0x7fa4a00c4160 2024-04-26T22:07:36.896 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:37.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.018+0000 7fb25e02a640 1 Processor -- start 2024-04-26T22:07:37.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.018+0000 7fb25e02a640 1 -- start start 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.019+0000 7fb25e02a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb25806dc60 0x7fb25815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.019+0000 7fb25e02a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2580ac870 0x7fb2581582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.019+0000 7fb25e02a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb25e02a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb2580c4960 con 0x7fb2580ac870 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb25e02a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb25815c040 con 0x7fb25806dc60 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb2577fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb25806dc60 0x7fb25815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb256ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2580ac870 0x7fb2581582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb2577fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb25806dc60 0x7fb25815b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:41374/0 (socket says 172.21.15.135:41374) 2024-04-26T22:07:37.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb2577fe640 1 -- 172.21.15.135:0/2783509649 learned_addr learned my addr 172.21.15.135:0/2783509649 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:37.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb257fff640 1 --2- 172.21.15.135:0/2783509649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.020+0000 7fb25e02a640 1 -- 172.21.15.135:0/2783509649 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb25815b7a0 con 0x7fb258158830 2024-04-26T22:07:37.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.021+0000 7fb256ffd640 1 -- 172.21.15.135:0/2783509649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 msgr2=0x7fb2580aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:37.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.021+0000 7fb256ffd640 1 --2- 172.21.15.135:0/2783509649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.021+0000 7fb256ffd640 1 -- 172.21.15.135:0/2783509649 >> v2:172.21.15.139:3300/0 conn(0x7fb25806dc60 msgr2=0x7fb25815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.021+0000 7fb256ffd640 1 --2- 172.21.15.135:0/2783509649 >> v2:172.21.15.139:3300/0 conn(0x7fb25806dc60 0x7fb25815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.021+0000 7fb256ffd640 1 -- 172.21.15.135:0/2783509649 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25815ba80 con 0x7fb2580ac870 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.021+0000 7fb256ffd640 1 --2- 172.21.15.135:0/2783509649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2580ac870 0x7fb2581582f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb2580cb2c0 tx=0x7fb24c02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e9f85addb910c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.022+0000 7fb2567fc640 1 -- 172.21.15.135:0/2783509649 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c037070 con 0x7fb2580ac870 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.022+0000 7fb2567fc640 1 -- 172.21.15.135:0/2783509649 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c02ad50 con 0x7fb2580ac870 2024-04-26T22:07:37.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.022+0000 7fb2567fc640 1 -- 172.21.15.135:0/2783509649 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c032660 con 0x7fb2580ac870 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.022+0000 7fb25e02a640 1 -- 172.21.15.135:0/2783509649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2580ac870 msgr2=0x7fb2581582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.022+0000 7fb25e02a640 1 --2- 172.21.15.135:0/2783509649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2580ac870 0x7fb2581582f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb2580cb2c0 tx=0x7fb24c02a000 comp rx=0 tx=0).stop 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 -- 172.21.15.135:0/2783509649 shutdown_connections 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 --2- 172.21.15.135:0/2783509649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2580ac870 0x7fb2581582f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 --2- 172.21.15.135:0/2783509649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 --2- 172.21.15.135:0/2783509649 >> v2:172.21.15.139:3300/0 conn(0x7fb25806dc60 0x7fb25815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 -- 172.21.15.135:0/2783509649 >> 172.21.15.135:0/2783509649 conn(0x7fb2580c6d80 msgr2=0x7fb2580c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 -- 172.21.15.135:0/2783509649 shutdown_connections 2024-04-26T22:07:37.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.023+0000 7fb25e02a640 1 -- 172.21.15.135:0/2783509649 wait complete. 2024-04-26T22:07:37.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.024+0000 7fb25e02a640 1 Processor -- start 2024-04-26T22:07:37.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.024+0000 7fb25e02a640 1 -- start start 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb25e02a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 0x7fb2581f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb25e02a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb25e02a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2580ad750 0x7fb258159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb25e02a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb258158cc0 con 0x7fb25806dc60 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb25e02a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb2580ab510 con 0x7fb2580ad750 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb25e02a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb25806e040 con 0x7fb258158830 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 0x7fb2581f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 0x7fb2581f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34372/0 (socket says 172.21.15.135:34372) 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 -- 172.21.15.135:0/1357203054 learned_addr learned my addr 172.21.15.135:0/1357203054 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:37.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb257fff640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2580ad750 0x7fb258159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb256ffd640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 -- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 msgr2=0x7fb2580707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 -- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2580ad750 msgr2=0x7fb258159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2580ad750 0x7fb258159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.025+0000 7fb2577fe640 1 -- 172.21.15.135:0/1357203054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25815a4e0 con 0x7fb25806dc60 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.026+0000 7fb2577fe640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 0x7fb2581f1050 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb24800e8d0 tx=0x7fb24800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.026+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248017070 con 0x7fb25806dc60 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.026+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2581f1590 con 0x7fb25806dc60 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.026+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24800c940 con 0x7fb25806dc60 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.026+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25815a750 con 0x7fb25806dc60 2024-04-26T22:07:37.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.027+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248017070 con 0x7fb25806dc60 2024-04-26T22:07:37.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.028+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2580507a0 con 0x7fb25806dc60 2024-04-26T22:07:37.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.028+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb24801a070 con 0x7fb25806dc60 2024-04-26T22:07:37.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.029+0000 7fb23ffff640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb24406caf0 0x7fb24406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:37.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.029+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 7526+0+0 (secure 0 0 0) 0x7fb24808bdc0 con 0x7fb25806dc60 2024-04-26T22:07:37.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.029+0000 7fb256ffd640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb24406caf0 0x7fb24406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:37.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.030+0000 7fb256ffd640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb24406caf0 0x7fb24406efb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb24c034000 tx=0x7fb24c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:37.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.035+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2581f1590 con 0x7fb25806dc60 2024-04-26T22:07:37.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.154+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4a014b140 con 0x7fa4a00c4160 2024-04-26T22:07:37.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.155+0000 7fa49cff9640 1 -- 172.21.15.135:0/3740905834 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5667 (secure 0 0 0) 0x7fa4900796c0 con 0x7fa4a00c4160 2024-04-26T22:07:37.155 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:37.156 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:32.835333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:37.156 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 59 2024-04-26T22:07:37.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa48c06c8f0 msgr2=0x7fa48c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa48c06c8f0 0x7fa48c06edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00c3180 tx=0x7fa494009040 comp rx=0 tx=0).stop 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 msgr2=0x7fa4a00c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 0x7fa4a00c2910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa49002a140 tx=0x7fa4900046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 shutdown_connections 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa48c06c8f0 0x7fa48c06edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4a00acda0 0x7fa4a00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a01510c0 0x7fa4a00aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 --2- 172.21.15.135:0/3740905834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4a00c4160 0x7fa4a00c2910 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.157+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 >> 172.21.15.135:0/3740905834 conn(0x7fa4a00c5ce0 msgr2=0x7fa4a0153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.158+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 shutdown_connections 2024-04-26T22:07:37.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.158+0000 7fa4a6a47640 1 -- 172.21.15.135:0/3740905834 wait complete. 2024-04-26T22:07:37.173 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:07:37.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.329+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb258076650 con 0x7fb25806dc60 2024-04-26T22:07:37.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.330+0000 7fb23ffff640 1 -- 172.21.15.135:0/1357203054 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5667 (secure 0 0 0) 0x7fb248059490 con 0x7fb25806dc60 2024-04-26T22:07:37.331 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:37.331 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:32.835333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:14.544129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5419":{"gid":5419,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":11,"addr":"172.21.15.135:6835/1037298395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1037298395},{"type":"v1","addr":"172.21.15.135:6835","nonce":1037298395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5419,"qdb_cluster":[5419]},"id":2}]} 2024-04-26T22:07:37.331 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 59 2024-04-26T22:07:37.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb24406caf0 msgr2=0x7fb24406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb24406caf0 0x7fb24406efb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb24c034000 tx=0x7fb24c02e000 comp rx=0 tx=0).stop 2024-04-26T22:07:37.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 msgr2=0x7fb2581f1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:37.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 0x7fb2581f1050 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb24800e8d0 tx=0x7fb24800ed90 comp rx=0 tx=0).stop 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 shutdown_connections 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb24406caf0 0x7fb24406efb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2580ad750 0x7fb258159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb258158830 0x7fb2580707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 --2- 172.21.15.135:0/1357203054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb25806dc60 0x7fb2581f1050 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 >> 172.21.15.135:0/1357203054 conn(0x7fb2580c6d80 msgr2=0x7fb25815d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 shutdown_connections 2024-04-26T22:07:37.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:37.332+0000 7fb25e02a640 1 -- 172.21.15.135:0/1357203054 wait complete. 2024-04-26T22:07:37.346 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:07:39.174 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:39.347 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:39.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.371+0000 7f4188dbf640 1 Processor -- start 2024-04-26T22:07:39.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.371+0000 7f4188dbf640 1 -- start start 2024-04-26T22:07:39.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.381+0000 7f4188dbf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f41840ab6a0 0x7f41841533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.381+0000 7f4188dbf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418406ff20 con 0x7f41840c2cd0 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.381+0000 7f4188dbf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418406f4e0 con 0x7f41840c8dc0 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.381+0000 7f4188dbf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f41840c4bd0 con 0x7f41840ab6a0 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.382+0000 7f4181d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f41840c8dc0 0x7f41840ac140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:36388/0 (socket says 172.21.15.135:36388) 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.382+0000 7f4181d9b640 1 -- 172.21.15.135:0/3453283714 learned_addr learned my addr 172.21.15.135:0/3453283714 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.382+0000 7f4182d9d640 1 --2- 172.21.15.135:0/3453283714 >> v2:172.21.15.135:3300/0 conn(0x7f41840ab6a0 0x7f41841533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.383+0000 7f4182d9d640 1 -- 172.21.15.135:0/3453283714 >> v1:172.21.15.139:6789/0 conn(0x7f41840c8dc0 legacy=0x7f41840ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.383+0000 7f4182d9d640 1 -- 172.21.15.135:0/3453283714 >> v1:172.21.15.135:6789/0 conn(0x7f41840c2cd0 legacy=0x7f4184150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.383+0000 7f4182d9d640 1 -- 172.21.15.135:0/3453283714 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41840c48f0 con 0x7f41840ab6a0 2024-04-26T22:07:39.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.384+0000 7f4182d9d640 1 --2- 172.21.15.135:0/3453283714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840ab6a0 0x7f41841533b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b050 tx=0x7f417c02a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a92a302b7d011f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:39.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.385+0000 7f418159a640 1 -- 172.21.15.135:0/3453283714 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c031660 con 0x7f41840ab6a0 2024-04-26T22:07:39.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.385+0000 7f418159a640 1 -- 172.21.15.135:0/3453283714 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c036070 con 0x7f41840ab6a0 2024-04-26T22:07:39.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.385+0000 7f418159a640 1 -- 172.21.15.135:0/3453283714 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c031a20 con 0x7f41840ab6a0 2024-04-26T22:07:39.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.386+0000 7f4188dbf640 1 -- 172.21.15.135:0/3453283714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840ab6a0 msgr2=0x7f41841533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.386+0000 7f4188dbf640 1 --2- 172.21.15.135:0/3453283714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840ab6a0 0x7f41841533b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b050 tx=0x7f417c02a8d0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.389+0000 7f4188dbf640 1 -- 172.21.15.135:0/3453283714 shutdown_connections 2024-04-26T22:07:39.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.389+0000 7f4188dbf640 1 --2- 172.21.15.135:0/3453283714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840ab6a0 0x7f41841533b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.389+0000 7f4188dbf640 1 -- 172.21.15.135:0/3453283714 >> 172.21.15.135:0/3453283714 conn(0x7f41840c82d0 msgr2=0x7f41840c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.391+0000 7f4188dbf640 1 -- 172.21.15.135:0/3453283714 shutdown_connections 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.391+0000 7f4188dbf640 1 -- 172.21.15.135:0/3453283714 wait complete. 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.393+0000 7f4188dbf640 1 Processor -- start 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.393+0000 7f4188dbf640 1 -- start start 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.394+0000 7f4188dbf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840ab6a0 0x7f4184156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.394+0000 7f4188dbf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 0x7f4184155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.394+0000 7f4188dbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840c8dc0 0x7f41841f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.394+0000 7f4188dbf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41840ac9d0 con 0x7f41840c2cd0 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.394+0000 7f4188dbf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41840ac850 con 0x7f41840c8dc0 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.394+0000 7f4188dbf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f41840abbf0 con 0x7f41840ab6a0 2024-04-26T22:07:39.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f418259c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840ab6a0 0x7f4184156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 0x7f4184155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 0x7f4184155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49686/0 (socket says 172.21.15.135:49686) 2024-04-26T22:07:39.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 -- 172.21.15.135:0/380419280 learned_addr learned my addr 172.21.15.135:0/380419280 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:39.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4182d9d640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840c8dc0 0x7f41841f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 -- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840ab6a0 msgr2=0x7f4184156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840ab6a0 0x7f4184156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 -- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840c8dc0 msgr2=0x7f41841f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.395+0000 7f4181d9b640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840c8dc0 0x7f41841f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.396+0000 7f4181d9b640 1 -- 172.21.15.135:0/380419280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41841528a0 con 0x7f41840c2cd0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.396+0000 7f4181d9b640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 0x7f4184155310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f417800c3f0 tx=0x7f417800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.397+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178017070 con 0x7f41840c2cd0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.397+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4178005ce0 con 0x7f41840c2cd0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.397+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780072d0 con 0x7f41840c2cd0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.397+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41841f4c40 con 0x7f41840c2cd0 2024-04-26T22:07:39.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.397+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418414d230 con 0x7f41840c2cd0 2024-04-26T22:07:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.400+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f417801a070 con 0x7f41840c2cd0 2024-04-26T22:07:39.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.400+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4168005290 con 0x7f41840c2cd0 2024-04-26T22:07:39.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.400+0000 7f4172ffd640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416406c9c0 0x7f416406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.400+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f417808b580 con 0x7f41840c2cd0 2024-04-26T22:07:39.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.409+0000 7f418259c640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416406c9c0 0x7f416406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.409+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4178058bd0 con 0x7f41840c2cd0 2024-04-26T22:07:39.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.410+0000 7f418259c640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416406c9c0 0x7f416406ee80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4174001780 tx=0x7f4174009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:39.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.534+0000 7f2d0c8f0640 1 Processor -- start 2024-04-26T22:07:39.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.534+0000 7f2d0c8f0640 1 -- start start 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0c8f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d0004b110 0x7f2d00006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0759e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2d000073e0 0x7f2d000d6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38144/0 (socket says 172.21.15.135:38144) 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0759e640 1 -- 172.21.15.135:0/2962883381 learned_addr learned my addr 172.21.15.135:0/2962883381 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0c8f0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000d9c60 con 0x7f2d000073e0 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0659c640 1 --2- 172.21.15.135:0/2962883381 >> v2:172.21.15.139:3300/0 conn(0x7f2d0004b110 0x7f2d00006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000d9f40 con 0x7f2d000d3b40 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2d000da220 con 0x7f2d0004b110 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928977943 0 0) 0x7f2d000d9c60 con 0x7f2d000073e0 2024-04-26T22:07:39.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.536+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cf0003520 con 0x7f2d000073e0 2024-04-26T22:07:39.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1634968183 0 0) 0x7f2cf0003520 con 0x7f2d000073e0 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 >> v2:172.21.15.139:3300/0 conn(0x7f2d0004b110 msgr2=0x7f2d00006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 --2- 172.21.15.135:0/2962883381 >> v2:172.21.15.139:3300/0 conn(0x7f2d0004b110 0x7f2d00006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 >> v1:172.21.15.139:6789/0 conn(0x7f2d000d3b40 legacy=0x7f2d00046b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000da500 con 0x7f2d000073e0 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2cf8001670 con 0x7f2d000073e0 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 >> v1:172.21.15.135:6789/0 conn(0x7f2d000073e0 legacy=0x7f2d000d6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00039b0 0x7f2cf0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cf0004800 0x7f2cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000da220 con 0x7f2cf00039b0 2024-04-26T22:07:39.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf00148f0 con 0x7f2cf00051c0 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf0014af0 con 0x7f2cf0004800 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.537+0000 7f2d0659c640 1 --2- 172.21.15.135:0/2962883381 >> v2:172.21.15.139:3300/0 conn(0x7f2d0004b110 0x7f2d00006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.538+0000 7f2d0759e640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.538+0000 7f2d0759e640 1 -- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cf0004800 msgr2=0x7f2cf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.538+0000 7f2d0759e640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cf0004800 0x7f2cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.538+0000 7f2d0759e640 1 -- 172.21.15.135:0/2962883381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00039b0 msgr2=0x7f2cf0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.538+0000 7f2d0759e640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00039b0 0x7f2cf0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.538+0000 7f2d0759e640 1 -- 172.21.15.135:0/2962883381 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000da500 con 0x7f2cf00051c0 2024-04-26T22:07:39.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.539+0000 7f2d0759e640 1 -- 172.21.15.135:0/2962883381 reap_dead start 2024-04-26T22:07:39.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.540+0000 7f2d0759e640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c260 tx=0x7f2cf0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eefc103e114a19b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:39.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.540+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8036070 con 0x7f2cf00051c0 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.540+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8005ad0 con 0x7f2cf00051c0 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.540+0000 7f2d05d9b640 1 -- 172.21.15.135:0/2962883381 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8007d50 con 0x7f2cf00051c0 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.541+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf00051c0 msgr2=0x7f2cf000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.541+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c260 tx=0x7f2cf0005830 comp rx=0 tx=0).stop 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.542+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 shutdown_connections 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.542+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2962883381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cf00051c0 0x7f2cf000b9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.542+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 >> 172.21.15.135:0/2962883381 conn(0x7f2d00048250 msgr2=0x7f2d00048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:39.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.547+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 shutdown_connections 2024-04-26T22:07:39.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.547+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2962883381 wait complete. 2024-04-26T22:07:39.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.549+0000 7f2d0c8f0640 1 Processor -- start 2024-04-26T22:07:39.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.550+0000 7f2d0c8f0640 1 -- start start 2024-04-26T22:07:39.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.551+0000 7f2d0c8f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 0x7f2d00043cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.551+0000 7f2d0c8f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d000456a0 0x7f2d000d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.551+0000 7f2d0c8f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d0016ee90 0x7f2d000d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.551+0000 7f2d0c8f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000d3f20 con 0x7f2cf00051c0 2024-04-26T22:07:39.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.551+0000 7f2d0c8f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000d9930 con 0x7f2d0016ee90 2024-04-26T22:07:39.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.551+0000 7f2d0c8f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0016f790 con 0x7f2d000456a0 2024-04-26T22:07:39.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.552+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d000456a0 0x7f2d000d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.552+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 0x7f2d00043cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.552+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 0x7f2d00043cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49710/0 (socket says 172.21.15.135:49710) 2024-04-26T22:07:39.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.552+0000 7f2d06d9d640 1 -- 172.21.15.135:0/2562339824 learned_addr learned my addr 172.21.15.135:0/2562339824 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:39.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.553+0000 7f2d06d9d640 1 -- 172.21.15.135:0/2562339824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d000456a0 msgr2=0x7f2d000d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.553+0000 7f2d06d9d640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d000456a0 0x7f2d000d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.553+0000 7f2d06d9d640 1 -- 172.21.15.135:0/2562339824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d0016ee90 msgr2=0x7f2d000d4ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:07:39.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.553+0000 7f2d06d9d640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d0016ee90 0x7f2d000d4ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.553+0000 7f2d06d9d640 1 -- 172.21.15.135:0/2562339824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0016fa70 con 0x7f2cf00051c0 2024-04-26T22:07:39.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.553+0000 7f2d06d9d640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 0x7f2d00043cc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000b60 tx=0x7f2d08069f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:39.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.554+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0806f840 con 0x7f2cf00051c0 2024-04-26T22:07:39.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.554+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d000d42d0 con 0x7f2cf00051c0 2024-04-26T22:07:39.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.554+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d000d13a0 con 0x7f2cf00051c0 2024-04-26T22:07:39.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.554+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0806f9e0 con 0x7f2cf00051c0 2024-04-26T22:07:39.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.554+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0806fe10 con 0x7f2cf00051c0 2024-04-26T22:07:39.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.557+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d08095060 con 0x7f2cf00051c0 2024-04-26T22:07:39.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.561+0000 7f2cef7fe640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2cf406c9f0 0x7f2cf406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:39.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.561+0000 7f2d0659c640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2cf406c9f0 0x7f2cf406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:39.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.562+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f2d08107d20 con 0x7f2cf00051c0 2024-04-26T22:07:39.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.562+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce8005290 con 0x7f2cf00051c0 2024-04-26T22:07:39.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.562+0000 7f2d0659c640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2cf406c9f0 0x7f2cf406eeb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d000d6390 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:39.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.571+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d080d59c0 con 0x7f2cf00051c0 2024-04-26T22:07:39.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.865+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4168005580 con 0x7f41840c2cd0 2024-04-26T22:07:39.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.866+0000 7f4172ffd640 1 -- 172.21.15.135:0/380419280 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+4832 (secure 0 0 0) 0x7f417805ced0 con 0x7f41840c2cd0 2024-04-26T22:07:39.866 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:39.866 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:32.835333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:37.836710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:39.867 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 60 2024-04-26T22:07:39.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416406c9c0 msgr2=0x7f416406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416406c9c0 0x7f416406ee80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4174001780 tx=0x7f4174009040 comp rx=0 tx=0).stop 2024-04-26T22:07:39.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 msgr2=0x7f4184155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:39.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 0x7f4184155310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f417800c3f0 tx=0x7f417800c8b0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 shutdown_connections 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416406c9c0 0x7f416406ee80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840c8dc0 0x7f41841f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41840c2cd0 0x7f4184155310 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 --2- 172.21.15.135:0/380419280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840ab6a0 0x7f4184156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 >> 172.21.15.135:0/380419280 conn(0x7f41840c82d0 msgr2=0x7f418414e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 shutdown_connections 2024-04-26T22:07:39.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:39.870+0000 7f4188dbf640 1 -- 172.21.15.135:0/380419280 wait complete. 2024-04-26T22:07:39.887 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:07:40.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.010+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ce8005580 con 0x7f2cf00051c0 2024-04-26T22:07:40.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.011+0000 7f2cef7fe640 1 -- 172.21.15.135:0/2562339824 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+4835 (secure 0 0 0) 0x7f2d080d52c0 con 0x7f2cf00051c0 2024-04-26T22:07:40.010 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:40.011 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:39.216368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:37.836710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:40.011 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 61 2024-04-26T22:07:40.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.013+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2cf406c9f0 msgr2=0x7f2cf406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.013+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2cf406c9f0 0x7f2cf406eeb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d000d6390 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.013+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 msgr2=0x7f2d00043cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.013+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 0x7f2d00043cc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000b60 tx=0x7f2d08069f90 comp rx=0 tx=0).stop 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 shutdown_connections 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2cf406c9f0 0x7f2cf406eeb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d0016ee90 0x7f2d000d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d000456a0 0x7f2d000d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 --2- 172.21.15.135:0/2562339824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cf00051c0 0x7f2d00043cc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 >> 172.21.15.135:0/2562339824 conn(0x7f2d00048250 msgr2=0x7f2d000d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 shutdown_connections 2024-04-26T22:07:40.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.014+0000 7f2d0c8f0640 1 -- 172.21.15.135:0/2562339824 wait complete. 2024-04-26T22:07:40.029 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-04-26T22:07:40.029 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:07:40.029 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:40.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.159+0000 7f84b3b22640 1 Processor -- start 2024-04-26T22:07:40.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.159+0000 7f84b3b22640 1 -- start start 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b3b22640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f84ac0c36b0 0x7f84ac0c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b3b22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b3b22640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f84ac0c5620 0x7f84ac15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b3b22640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f84ac157b20 con 0x7f84ac0c36b0 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b3b22640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f84ac0c48d0 con 0x7f84ac0c5620 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b3b22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac0aca00 con 0x7f84ac0ac3e0 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b18be640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f84ac0c36b0 0x7f84ac0c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b18be640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f84ac0c36b0 0x7f84ac0c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49712/0 (socket says 172.21.15.135:49712) 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.160+0000 7f84b18be640 1 -- 172.21.15.135:0/519405834 learned_addr learned my addr 172.21.15.135:0/519405834 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b10bd640 1 --2- 172.21.15.135:0/519405834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b20bf640 1 --2- 172.21.15.135:0/519405834 >> v2:172.21.15.139:3300/0 conn(0x7f84ac0c5620 0x7f84ac15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b18be640 1 -- 172.21.15.135:0/519405834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 msgr2=0x7f84ac1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:40.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b18be640 1 --2- 172.21.15.135:0/519405834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b18be640 1 -- 172.21.15.135:0/519405834 >> v2:172.21.15.139:3300/0 conn(0x7f84ac0c5620 msgr2=0x7f84ac15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:40.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b18be640 1 --2- 172.21.15.135:0/519405834 >> v2:172.21.15.139:3300/0 conn(0x7f84ac0c5620 0x7f84ac15a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.161+0000 7f84b18be640 1 -- 172.21.15.135:0/519405834 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac0acce0 con 0x7f84ac0c36b0 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.162+0000 7f84b18be640 1 --2- 172.21.15.135:0/519405834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0c3a90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f84ac068c90 tx=0x7f84a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433b5a9ace88575 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.162+0000 7f84b08bc640 1 -- 172.21.15.135:0/519405834 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8037070 con 0x7f84ac0c36b0 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.162+0000 7f84b08bc640 1 -- 172.21.15.135:0/519405834 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a8032440 con 0x7f84ac0c36b0 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.162+0000 7f84b08bc640 1 -- 172.21.15.135:0/519405834 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8032810 con 0x7f84ac0c36b0 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.162+0000 7f84b3b22640 1 -- 172.21.15.135:0/519405834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c36b0 msgr2=0x7f84ac0c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.162+0000 7f84b3b22640 1 --2- 172.21.15.135:0/519405834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0c3a90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f84ac068c90 tx=0x7f84a802a270 comp rx=0 tx=0).stop 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 -- 172.21.15.135:0/519405834 shutdown_connections 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 --2- 172.21.15.135:0/519405834 >> v2:172.21.15.139:3300/0 conn(0x7f84ac0c5620 0x7f84ac15a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 --2- 172.21.15.135:0/519405834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 --2- 172.21.15.135:0/519405834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0c3a90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 -- 172.21.15.135:0/519405834 >> 172.21.15.135:0/519405834 conn(0x7f84ac0c6d80 msgr2=0x7f84ac0c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:40.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 -- 172.21.15.135:0/519405834 shutdown_connections 2024-04-26T22:07:40.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.163+0000 7f84b3b22640 1 -- 172.21.15.135:0/519405834 wait complete. 2024-04-26T22:07:40.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 Processor -- start 2024-04-26T22:07:40.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 -- start start 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 0x7f84ac0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac15a700 con 0x7f84ac0c5620 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.165+0000 7f84b3b22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac1570c0 con 0x7f84ac0c36b0 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b3b22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac0ab690 con 0x7f84ac0ac3e0 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 0x7f84ac0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 0x7f84ac0c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49720/0 (socket says 172.21.15.135:49720) 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 -- 172.21.15.135:0/1595959422 learned_addr learned my addr 172.21.15.135:0/1595959422 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b10bd640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 -- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 msgr2=0x7f84ac1ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 -- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac0c36b0 msgr2=0x7f84ac0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 -- 172.21.15.135:0/1595959422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac0ad810 con 0x7f84ac0c5620 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.166+0000 7f84b20bf640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 0x7f84ac0c82f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f84a4007e90 tx=0x7f84a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.167+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4014420 con 0x7f84ac0c5620 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.167+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a4019070 con 0x7f84ac0c5620 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.167+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4014800 con 0x7f84ac0c5620 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.167+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac0aee90 con 0x7f84ac0c5620 2024-04-26T22:07:40.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.167+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac0af450 con 0x7f84ac0c5620 2024-04-26T22:07:40.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.168+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac0507a0 con 0x7f84ac0c5620 2024-04-26T22:07:40.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.169+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84a40149a0 con 0x7f84ac0c5620 2024-04-26T22:07:40.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.169+0000 7f84a27fc640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f849006c8f0 0x7f849006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:40.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.169+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f84a408cb10 con 0x7f84ac0c5620 2024-04-26T22:07:40.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.169+0000 7f84b18be640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f849006c8f0 0x7f849006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:40.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.170+0000 7f84b18be640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f849006c8f0 0x7f849006edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84a8034000 tx=0x7f84a802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:40.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.176+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84a405a7e0 con 0x7f84ac0c5620 2024-04-26T22:07:40.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.476+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84ac0c4620 con 0x7f84ac0c5620 2024-04-26T22:07:40.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.477+0000 7f84a27fc640 1 -- 172.21.15.135:0/1595959422 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+4835 (secure 0 0 0) 0x7f84a405a0e0 con 0x7f84ac0c5620 2024-04-26T22:07:40.477 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:40.477 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:39.216368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:37.836710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:40.477 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 61 2024-04-26T22:07:40.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f849006c8f0 msgr2=0x7f849006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f849006c8f0 0x7f849006edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84a8034000 tx=0x7f84a802c000 comp rx=0 tx=0).stop 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 msgr2=0x7f84ac0c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 0x7f84ac0c82f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f84a4007e90 tx=0x7f84a400e4b0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 shutdown_connections 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f849006c8f0 0x7f849006edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84ac0c5620 0x7f84ac0c82f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84ac0c36b0 0x7f84ac0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 --2- 172.21.15.135:0/1595959422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84ac0ac3e0 0x7f84ac1ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 >> 172.21.15.135:0/1595959422 conn(0x7f84ac0c6d80 msgr2=0x7f84ac154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 shutdown_connections 2024-04-26T22:07:40.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:40.479+0000 7f84b3b22640 1 -- 172.21.15.135:0/1595959422 wait complete. 2024-04-26T22:07:40.491 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 61, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 61, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5210": { "addr": "172.21.15.135:6837/3969385858", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 3969385858, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 3969385858, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5210, "incarnation": 59, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:07:39.216368+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5210 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 60, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4928": { "addr": "172.21.15.139:6833/3664037452", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 3664037452, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 3664037452, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4928, "incarnation": 60, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:07:37.836710+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4928 } } } ], "standbys": [] } 2024-04-26T22:07:41.888 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:42.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.037+0000 7fe1f64f8640 1 Processor -- start 2024-04-26T22:07:42.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.037+0000 7fe1f64f8640 1 -- start start 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1f64f8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1f00c30a0 0x7fe1f00c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1f64f8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f00c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1f64f8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f00c39b0 con 0x7fe1f01520d0 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1f64f8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f00720a0 con 0x7fe1f00c30a0 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1f64f8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f0071e80 con 0x7fe1f01524b0 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1ef7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1f00c30a0 0x7fe1f00c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1f4a95640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f00c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1ef7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1f00c30a0 0x7fe1f00c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:39944/0 (socket says 172.21.15.135:39944) 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.038+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/247096539 learned_addr learned my addr 172.21.15.135:0/247096539 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.039+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 msgr2=0x7fe1f00c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:42.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.039+0000 7fe1ef7fe640 1 --2- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f00c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.039+0000 7fe1eeffd640 1 -- 172.21.15.135:0/247096539 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362202613 0 0) 0x7fe1f00c39b0 con 0x7fe1f01520d0 2024-04-26T22:07:42.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.039+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/247096539 >> v1:172.21.15.135:6789/0 conn(0x7fe1f01520d0 legacy=0x7fe1f0073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.039+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/247096539 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f00c3c50 con 0x7fe1f00c30a0 2024-04-26T22:07:42.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.040+0000 7fe1ef7fe640 1 --2- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f00c30a0 0x7fe1f00c3f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009870 tx=0x7fe1e402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af32b28b962df56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:42.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.041+0000 7fe1eeffd640 1 -- 172.21.15.135:0/247096539 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4037070 con 0x7fe1f00c30a0 2024-04-26T22:07:42.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.041+0000 7fe1eeffd640 1 -- 172.21.15.135:0/247096539 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e4032440 con 0x7fe1f00c30a0 2024-04-26T22:07:42.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.041+0000 7fe1eeffd640 1 -- 172.21.15.135:0/247096539 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4032810 con 0x7fe1f00c30a0 2024-04-26T22:07:42.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 -- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f00c30a0 msgr2=0x7fe1f00c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f00c30a0 0x7fe1f00c3f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009870 tx=0x7fe1e402a270 comp rx=0 tx=0).stop 2024-04-26T22:07:42.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 -- 172.21.15.135:0/247096539 shutdown_connections 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f00c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/247096539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f00c30a0 0x7fe1f00c3f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 -- 172.21.15.135:0/247096539 >> 172.21.15.135:0/247096539 conn(0x7fe1f00c82d0 msgr2=0x7fe1f00c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 -- 172.21.15.135:0/247096539 shutdown_connections 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.042+0000 7fe1f64f8640 1 -- 172.21.15.135:0/247096539 wait complete. 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.043+0000 7fe1f64f8640 1 Processor -- start 2024-04-26T22:07:42.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.044+0000 7fe1f64f8640 1 -- start start 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f64f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1f01520d0 0x7fe1f00ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f64f8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f0154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f64f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0158270 0x7fe1f01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f64f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0072530 con 0x7fe1f01520d0 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f64f8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0155aa0 con 0x7fe1f0158270 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f64f8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f0156010 con 0x7fe1f01524b0 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1effff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1f01520d0 0x7fe1f00ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f0154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f0154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:35630/0 (socket says 172.21.15.135:35630) 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f4a95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0158270 0x7fe1f01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/2976193472 learned_addr learned my addr 172.21.15.135:0/2976193472 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0158270 msgr2=0x7fe1f01564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0158270 0x7fe1f01564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1f01520d0 msgr2=0x7fe1f00ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1f01520d0 0x7fe1f00ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1ef7fe640 1 -- 172.21.15.135:0/2976193472 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f00aa620 con 0x7fe1f01524b0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.045+0000 7fe1f4a95640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0158270 0x7fe1f01564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1effff640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1f01520d0 0x7fe1f00ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1ef7fe640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f0154400 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40090b0 tx=0x7fe1e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4041020 con 0x7fe1f01524b0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f00ad170 con 0x7fe1f01524b0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f0154940 con 0x7fe1f01524b0 2024-04-26T22:07:42.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e4030700 con 0x7fe1f01524b0 2024-04-26T22:07:42.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.046+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4037040 con 0x7fe1f01524b0 2024-04-26T22:07:42.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.048+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d0005290 con 0x7fe1f01524b0 2024-04-26T22:07:42.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.048+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1e4044020 con 0x7fe1f01524b0 2024-04-26T22:07:42.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.049+0000 7fe1ecff9640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe1dc06c8f0 0x7fe1dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.049+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7fe1e40ac940 con 0x7fe1f01524b0 2024-04-26T22:07:42.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.049+0000 7fe1effff640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe1dc06c8f0 0x7fe1dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.049+0000 7fe1effff640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe1dc06c8f0 0x7fe1dc06edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0007fd0 tx=0x7fe1e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:42.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.055+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1e404b030 con 0x7fe1f01524b0 2024-04-26T22:07:42.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:42.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.498+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1d0005580 con 0x7fe1f01524b0 2024-04-26T22:07:42.499 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:42.499 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:40.910431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:rejoin","state_seq":67,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:07:42.499 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 63 2024-04-26T22:07:42.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.499+0000 7fe1ecff9640 1 -- 172.21.15.135:0/2976193472 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+4840 (secure 0 0 0) 0x7fe1e407a5f0 con 0x7fe1f01524b0 2024-04-26T22:07:42.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe1dc06c8f0 msgr2=0x7fe1dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe1dc06c8f0 0x7fe1dc06edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0007fd0 tx=0x7fe1e0009040 comp rx=0 tx=0).stop 2024-04-26T22:07:42.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 msgr2=0x7fe1f0154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f0154400 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40090b0 tx=0x7fe1e40046c0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 shutdown_connections 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe1dc06c8f0 0x7fe1dc06edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f0158270 0x7fe1f01564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f01524b0 0x7fe1f0154400 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 --2- 172.21.15.135:0/2976193472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1f01520d0 0x7fe1f00ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 >> 172.21.15.135:0/2976193472 conn(0x7fe1f00c82d0 msgr2=0x7fe1f006e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:42.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.503+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 shutdown_connections 2024-04-26T22:07:42.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.506+0000 7fe1f64f8640 1 -- 172.21.15.135:0/2976193472 wait complete. 2024-04-26T22:07:42.530 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=5105 2024-04-26T22:07:42.531 INFO:tasks.mds_thrash.fs.[a]:waiting for 3 secs before reviving mds.b 2024-04-26T22:07:42.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.728+0000 7f7fe099e640 1 Processor -- start 2024-04-26T22:07:42.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.728+0000 7f7fe099e640 1 -- start start 2024-04-26T22:07:42.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.729+0000 7f7fe099e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7fd40dbee0 0x7f7fd40df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.729+0000 7f7fe099e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7fd40431d0 0x7f7fd4007210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7fd40431d0 0x7f7fd4007210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7fd40431d0 0x7f7fd4007210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49738/0 (socket says 172.21.15.135:49738) 2024-04-26T22:07:42.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fe099e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd4007750 0x7f7fd400e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fe099e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd40df820 con 0x7f7fd40431d0 2024-04-26T22:07:42.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fe099e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd40dc2c0 con 0x7f7fd40dbee0 2024-04-26T22:07:42.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fe099e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd404a700 con 0x7f7fd4007750 2024-04-26T22:07:42.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 -- 172.21.15.135:0/4029559872 learned_addr learned my addr 172.21.15.135:0/4029559872 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:42.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 -- 172.21.15.135:0/4029559872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd4007750 msgr2=0x7f7fd400e540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:42.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 --2- 172.21.15.135:0/4029559872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd4007750 0x7f7fd400e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 -- 172.21.15.135:0/4029559872 >> v2:172.21.15.139:3300/0 conn(0x7f7fd40dbee0 msgr2=0x7f7fd40df2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 --2- 172.21.15.135:0/4029559872 >> v2:172.21.15.139:3300/0 conn(0x7f7fd40dbee0 0x7f7fd40df2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.730+0000 7f7fdaffd640 1 -- 172.21.15.135:0/4029559872 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd404a9e0 con 0x7f7fd40431d0 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.731+0000 7f7fdaffd640 1 --2- 172.21.15.135:0/4029559872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd4007210 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc065e50 tx=0x7f7fdc0b8400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2210a37e5c7962a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.732+0000 7f7fda7fc640 1 -- 172.21.15.135:0/4029559872 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc06e650 con 0x7f7fd40431d0 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.732+0000 7f7fda7fc640 1 -- 172.21.15.135:0/4029559872 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc06e7f0 con 0x7f7fd40431d0 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.732+0000 7f7fda7fc640 1 -- 172.21.15.135:0/4029559872 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc06ec60 con 0x7f7fd40431d0 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 -- 172.21.15.135:0/4029559872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 msgr2=0x7f7fd4007210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 --2- 172.21.15.135:0/4029559872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd4007210 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc065e50 tx=0x7f7fdc0b8400 comp rx=0 tx=0).stop 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 -- 172.21.15.135:0/4029559872 shutdown_connections 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 --2- 172.21.15.135:0/4029559872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd4007210 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 --2- 172.21.15.135:0/4029559872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd4007750 0x7f7fd400e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 --2- 172.21.15.135:0/4029559872 >> v2:172.21.15.139:3300/0 conn(0x7f7fd40dbee0 0x7f7fd40df2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.733+0000 7f7fe099e640 1 -- 172.21.15.135:0/4029559872 >> 172.21.15.135:0/4029559872 conn(0x7f7fd4031730 msgr2=0x7f7fd4031b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:42.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.738+0000 7f7fe099e640 1 -- 172.21.15.135:0/4029559872 shutdown_connections 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.739+0000 7f7fe099e640 1 -- 172.21.15.135:0/4029559872 wait complete. 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.741+0000 7f7fe099e640 1 Processor -- start 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.741+0000 7f7fe099e640 1 -- start start 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fe099e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fd4007750 0x7f7fd40d3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fe099e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd40d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fdaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd40d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fdaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd40d3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49744/0 (socket says 172.21.15.135:49744) 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fdaffd640 1 -- 172.21.15.135:0/2993747907 learned_addr learned my addr 172.21.15.135:0/2993747907 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd418e880 0x7f7fd4044ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400e020 con 0x7f7fd40431d0 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd404afa0 con 0x7f7fd4007750 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.742+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd400ea80 con 0x7f7fd418e880 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdb7fe640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fd4007750 0x7f7fd40d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdaffd640 1 -- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd418e880 msgr2=0x7f7fd4044ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdaffd640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd418e880 0x7f7fd4044ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdaffd640 1 -- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fd4007750 msgr2=0x7f7fd40d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdaffd640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fd4007750 0x7f7fd40d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdaffd640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd4046690 con 0x7f7fd40431d0 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.743+0000 7f7fdaffd640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd40d3ac0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc065e50 tx=0x7f7fdc06b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.744+0000 7f7fdb7fe640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fd4007750 0x7f7fd40d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:42.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.745+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0bfaf0 con 0x7f7fd40431d0 2024-04-26T22:07:42.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.745+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd4043800 con 0x7f7fd40431d0 2024-04-26T22:07:42.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.748+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd4046170 con 0x7f7fd40431d0 2024-04-26T22:07:42.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.750+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc0bfc90 con 0x7f7fd40431d0 2024-04-26T22:07:42.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.750+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc087610 con 0x7f7fd40431d0 2024-04-26T22:07:42.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.750+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fdc0863c0 con 0x7f7fd40431d0 2024-04-26T22:07:42.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.750+0000 7f7fc7fff640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7fc006ca20 0x7f7fc006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:42.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.751+0000 7f7fc5ffb640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd40057e0 con 0x7f7fd40431d0 2024-04-26T22:07:42.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.751+0000 7f7fdb7fe640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7fc006ca20 0x7f7fc006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:42.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.752+0000 7f7fdb7fe640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7fc006ca20 0x7f7fc006eee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4008490 tx=0x7f7fc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:42.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.762+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f7fdc129970 con 0x7f7fd40431d0 2024-04-26T22:07:42.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:42.762+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7fdc133410 con 0x7f7fd40431d0 2024-04-26T22:07:43.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.274+0000 7f7fc5ffb640 1 -- 172.21.15.135:0/2993747907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fd40d47c0 con 0x7f7fd40431d0 2024-04-26T22:07:43.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.279+0000 7f7fc7fff640 1 -- 172.21.15.135:0/2993747907 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+4847 (secure 0 0 0) 0x7f7fdc0f7700 con 0x7f7fd40431d0 2024-04-26T22:07:43.279 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:43.279 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:07:43.280 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 64 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7fc006ca20 msgr2=0x7f7fc006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7fc006ca20 0x7f7fc006eee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4008490 tx=0x7f7fc8008040 comp rx=0 tx=0).stop 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 msgr2=0x7f7fd40d3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd40d3ac0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc065e50 tx=0x7f7fdc06b850 comp rx=0 tx=0).stop 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 shutdown_connections 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7fc006ca20 0x7f7fc006eee0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fd418e880 0x7f7fd4044ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd40431d0 0x7f7fd40d3ac0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 --2- 172.21.15.135:0/2993747907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fd4007750 0x7f7fd40d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:43.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.284+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 >> 172.21.15.135:0/2993747907 conn(0x7f7fd4031730 msgr2=0x7f7fd400ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:43.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.285+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 shutdown_connections 2024-04-26T22:07:43.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:43.285+0000 7f7fe099e640 1 -- 172.21.15.135:0/2993747907 wait complete. 2024-04-26T22:07:43.316 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=5419 2024-04-26T22:07:43.316 INFO:tasks.mds_thrash.fs.[b]:waiting for 9 secs before reviving mds.c 2024-04-26T22:07:45.532 INFO:tasks.mds_thrash.fs.[a]:reviving mds.b 2024-04-26T22:07:45.532 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T22:07:45.532 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T22:07:45.535 INFO:tasks.ceph.mds.b:Started 2024-04-26T22:07:45.536 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:45.690 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-04-26T22:07:45.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.721+0000 7f9f1459d640 1 Processor -- start 2024-04-26T22:07:45.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.723+0000 7f9f1459d640 1 -- start start 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f1459d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9f0c150200 0x7f9f0c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f1459d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f0c0c2cd0 0x7f9f0c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f1459d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f1459d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9f0c0c4bd0 con 0x7f9f0c0c2cd0 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f1459d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9f0c06ff20 con 0x7f9f0c150200 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f1459d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0c06f4e0 con 0x7f9f0c0ac680 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f11b38640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f0c0c2cd0 0x7f9f0c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f11b38640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f0c0c2cd0 0x7f9f0c1533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49752/0 (socket says 172.21.15.135:49752) 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.725+0000 7f9f11b38640 1 -- 172.21.15.135:0/3447876732 learned_addr learned my addr 172.21.15.135:0/3447876732 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.726+0000 7f9f11b38640 1 -- 172.21.15.135:0/3447876732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 msgr2=0x7f9f0c1538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.726+0000 7f9f11b38640 1 --2- 172.21.15.135:0/3447876732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.726+0000 7f9f11b38640 1 -- 172.21.15.135:0/3447876732 >> v2:172.21.15.139:3300/0 conn(0x7f9f0c150200 msgr2=0x7f9f0c0ac140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.726+0000 7f9f11b38640 1 --2- 172.21.15.135:0/3447876732 >> v2:172.21.15.139:3300/0 conn(0x7f9f0c150200 0x7f9f0c0ac140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.726+0000 7f9f11b38640 1 -- 172.21.15.135:0/3447876732 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c0c48f0 con 0x7f9f0c0c2cd0 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.727+0000 7f9f11b38640 1 --2- 172.21.15.135:0/3447876732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c2cd0 0x7f9f0c1533b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c071320 tx=0x7f9f0402cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b486cfcf2ed7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.728+0000 7f9f11337640 1 -- 172.21.15.135:0/3447876732 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04034070 con 0x7f9f0c0c2cd0 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.728+0000 7f9f11337640 1 -- 172.21.15.135:0/3447876732 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f04030460 con 0x7f9f0c0c2cd0 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.728+0000 7f9f11337640 1 -- 172.21.15.135:0/3447876732 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04030810 con 0x7f9f0c0c2cd0 2024-04-26T22:07:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 -- 172.21.15.135:0/3447876732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c2cd0 msgr2=0x7f9f0c1533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 --2- 172.21.15.135:0/3447876732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c2cd0 0x7f9f0c1533b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c071320 tx=0x7f9f0402cab0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 -- 172.21.15.135:0/3447876732 shutdown_connections 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 --2- 172.21.15.135:0/3447876732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c2cd0 0x7f9f0c1533b0 secure :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c071320 tx=0x7f9f0402cab0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 --2- 172.21.15.135:0/3447876732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c1538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 --2- 172.21.15.135:0/3447876732 >> v2:172.21.15.139:3300/0 conn(0x7f9f0c150200 0x7f9f0c0ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.729+0000 7f9f1459d640 1 -- 172.21.15.135:0/3447876732 >> 172.21.15.135:0/3447876732 conn(0x7f9f0c0c82d0 msgr2=0x7f9f0c0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.731+0000 7f9f1459d640 1 -- 172.21.15.135:0/3447876732 shutdown_connections 2024-04-26T22:07:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.734+0000 7f9f1459d640 1 -- 172.21.15.135:0/3447876732 wait complete. 2024-04-26T22:07:45.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.736+0000 7f9f1459d640 1 Processor -- start 2024-04-26T22:07:45.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.736+0000 7f9f1459d640 1 -- start start 2024-04-26T22:07:45.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f1459d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f1459d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f0c150200 0x7f9f0c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f1459d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c7210 0x7f9f0c152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f1459d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c150760 con 0x7f9f0c0c7210 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f1459d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1505e0 con 0x7f9f0c150200 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f1459d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0c0c8dc0 con 0x7f9f0c0ac680 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.737+0000 7f9f12b3a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c7210 0x7f9f0c152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.738+0000 7f9f12b3a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c7210 0x7f9f0c152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49762/0 (socket says 172.21.15.135:49762) 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.738+0000 7f9f12b3a640 1 -- 172.21.15.135:0/2038384920 learned_addr learned my addr 172.21.15.135:0/2038384920 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.741+0000 7f9f12339640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:45.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.741+0000 7f9f11b38640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f0c150200 0x7f9f0c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:45.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f11b38640 1 -- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 msgr2=0x7f9f0c148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:45.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f11b38640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f11b38640 1 -- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c7210 msgr2=0x7f9f0c152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:45.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f11b38640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c7210 0x7f9f0c152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:45.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f11b38640 1 -- 172.21.15.135:0/2038384920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c13f660 con 0x7f9f0c150200 2024-04-26T22:07:45.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f11b38640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f0c150200 0x7f9f0c151b20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f04033000 tx=0x7f9f04004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:45.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.742+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04046070 con 0x7f9f0c150200 2024-04-26T22:07:45.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.743+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c13f920 con 0x7f9f0c150200 2024-04-26T22:07:45.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.743+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c13fee0 con 0x7f9f0c150200 2024-04-26T22:07:45.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.743+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f04030460 con 0x7f9f0c150200 2024-04-26T22:07:45.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.743+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04034070 con 0x7f9f0c150200 2024-04-26T22:07:45.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.746+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f04049070 con 0x7f9f0c150200 2024-04-26T22:07:45.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.748+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef4005290 con 0x7f9f0c150200 2024-04-26T22:07:45.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.750+0000 7f9f02ffd640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ef006c8f0 0x7f9ef006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:45.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.750+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f9f040acd10 con 0x7f9f0c150200 2024-04-26T22:07:45.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.751+0000 7f9f12339640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ef006c8f0 0x7f9ef006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:45.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.751+0000 7f9f12339640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ef006c8f0 0x7f9ef006edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c14d740 tx=0x7f9efc009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:45.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:45.760+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f0403e070 con 0x7f9f0c150200 2024-04-26T22:07:46.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.259+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ef4005580 con 0x7f9f0c150200 2024-04-26T22:07:46.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.260+0000 7f9f02ffd640 1 -- 172.21.15.135:0/2038384920 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5674 (secure 0 0 0) 0x7f9f0407a370 con 0x7f9f0c150200 2024-04-26T22:07:46.260 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:46.261 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:07:46.262 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 65 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ef006c8f0 msgr2=0x7f9ef006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ef006c8f0 0x7f9ef006edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c14d740 tx=0x7f9efc009060 comp rx=0 tx=0).stop 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f0c150200 msgr2=0x7f9f0c151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f0c150200 0x7f9f0c151b20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f04033000 tx=0x7f9f04004a40 comp rx=0 tx=0).stop 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 shutdown_connections 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ef006c8f0 0x7f9ef006edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f0c0c7210 0x7f9f0c152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f0c150200 0x7f9f0c151b20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 --2- 172.21.15.135:0/2038384920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f0c0ac680 0x7f9f0c148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:46.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.266+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 >> 172.21.15.135:0/2038384920 conn(0x7f9f0c0c82d0 msgr2=0x7f9f0c14ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:46.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.267+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 shutdown_connections 2024-04-26T22:07:46.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:46.267+0000 7f9f1459d640 1 -- 172.21.15.135:0/2038384920 wait complete. 2024-04-26T22:07:46.287 INFO:tasks.mds_thrash.fs.[a]:mds.b reported in up:standby state 2024-04-26T22:07:46.287 INFO:tasks.mds_thrash.fs.[a]:waiting for 110 secs before thrashing 2024-04-26T22:07:46.623 INFO:tasks.ceph.mds.a.smithi135.stderr:2024-04-26T22:07:46.624+0000 7f7bec906640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:07:47.282 INFO:tasks.ceph.mds.d.smithi139.stderr:2024-04-26T22:07:47.282+0000 7fe374fbf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:07:52.317 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-04-26T22:07:52.317 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:07:52.317 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:07:52.320 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:07:52.320 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:52.513 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-26T22:07:52.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.523+0000 7f9c3523e640 1 Processor -- start 2024-04-26T22:07:52.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.525+0000 7f9c3523e640 1 -- start start 2024-04-26T22:07:52.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.526+0000 7f9c3523e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c280dbea0 0x7f9c2800f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:52.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.526+0000 7f9c3523e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c2800e2e0 0x7f9c2800e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:52.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.526+0000 7f9c3523e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c280dc280 con 0x7f9c2804e110 2024-04-26T22:07:52.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.526+0000 7f9c3523e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9c280dea40 con 0x7f9c2800e2e0 2024-04-26T22:07:52.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.526+0000 7f9c3523e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9c280ded20 con 0x7f9c280dbea0 2024-04-26T22:07:52.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.527+0000 7f9c2ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c280dbea0 0x7f9c2800f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:52.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.527+0000 7f9c2ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c280dbea0 0x7f9c2800f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:43068/0 (socket says 172.21.15.135:43068) 2024-04-26T22:07:52.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.527+0000 7f9c2ffff640 1 -- 172.21.15.135:0/3164352681 learned_addr learned my addr 172.21.15.135:0/3164352681 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:52.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.528+0000 7f9c34a3d640 1 --2- 172.21.15.135:0/3164352681 >> v2:172.21.15.135:3300/0 conn(0x7f9c2800e2e0 0x7f9c2800e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:52.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.528+0000 7f9c34a3d640 1 -- 172.21.15.135:0/3164352681 >> v2:172.21.15.139:3300/0 conn(0x7f9c280dbea0 msgr2=0x7f9c2800f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:52.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.528+0000 7f9c34a3d640 1 --2- 172.21.15.135:0/3164352681 >> v2:172.21.15.139:3300/0 conn(0x7f9c280dbea0 0x7f9c2800f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:52.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.528+0000 7f9c34a3d640 1 -- 172.21.15.135:0/3164352681 >> v1:172.21.15.139:6789/0 conn(0x7f9c2804e110 legacy=0x7f9c28007250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:07:52.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.528+0000 7f9c34a3d640 1 -- 172.21.15.135:0/3164352681 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c280df000 con 0x7f9c2800e2e0 2024-04-26T22:07:52.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.530+0000 7f9c34a3d640 1 --2- 172.21.15.135:0/3164352681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2800e2e0 0x7f9c2800e720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9c2804d700 tx=0x7f9c2402a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f7aa5897c5df8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:52.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.530+0000 7f9c2effd640 1 -- 172.21.15.135:0/3164352681 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24037070 con 0x7f9c2800e2e0 2024-04-26T22:07:52.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.531+0000 7f9c2effd640 1 -- 172.21.15.135:0/3164352681 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2402ad40 con 0x7f9c2800e2e0 2024-04-26T22:07:52.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.531+0000 7f9c2effd640 1 -- 172.21.15.135:0/3164352681 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24032660 con 0x7f9c2800e2e0 2024-04-26T22:07:52.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.534+0000 7f9c3523e640 1 -- 172.21.15.135:0/3164352681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2800e2e0 msgr2=0x7f9c2800e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:52.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.534+0000 7f9c3523e640 1 --2- 172.21.15.135:0/3164352681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2800e2e0 0x7f9c2800e720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9c2804d700 tx=0x7f9c2402a030 comp rx=0 tx=0).stop 2024-04-26T22:07:52.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.534+0000 7f9c3523e640 1 -- 172.21.15.135:0/3164352681 shutdown_connections 2024-04-26T22:07:52.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.534+0000 7f9c3523e640 1 --2- 172.21.15.135:0/3164352681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2800e2e0 0x7f9c2800e720 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:52.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.534+0000 7f9c3523e640 1 --2- 172.21.15.135:0/3164352681 >> v2:172.21.15.139:3300/0 conn(0x7f9c280dbea0 0x7f9c2800f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:52.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.534+0000 7f9c3523e640 1 -- 172.21.15.135:0/3164352681 >> 172.21.15.135:0/3164352681 conn(0x7f9c2804c600 msgr2=0x7f9c2804ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:52.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.539+0000 7f9c3523e640 1 -- 172.21.15.135:0/3164352681 shutdown_connections 2024-04-26T22:07:52.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.540+0000 7f9c3523e640 1 -- 172.21.15.135:0/3164352681 wait complete. 2024-04-26T22:07:52.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.541+0000 7f9c3523e640 1 Processor -- start 2024-04-26T22:07:52.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.541+0000 7f9c3523e640 1 -- start start 2024-04-26T22:07:52.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.542+0000 7f9c3523e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c2800e2e0 0x7f9c280dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:52.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.542+0000 7f9c3523e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2804e110 0x7f9c28047600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.542+0000 7f9c3523e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c280dbea0 0x7f9c28045670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.542+0000 7f9c3523e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28007960 con 0x7f9c2804e110 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.542+0000 7f9c3523e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c280df5c0 con 0x7f9c280dbea0 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.542+0000 7f9c3523e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9c28007ae0 con 0x7f9c2800e2e0 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.543+0000 7f9c2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2804e110 0x7f9c28047600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.543+0000 7f9c34a3d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c280dbea0 0x7f9c28045670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.543+0000 7f9c34a3d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c280dbea0 0x7f9c28045670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:43074/0 (socket says 172.21.15.135:43074) 2024-04-26T22:07:52.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.543+0000 7f9c34a3d640 1 -- 172.21.15.135:0/2395760535 learned_addr learned my addr 172.21.15.135:0/2395760535 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:07:52.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2f7fe640 1 -- 172.21.15.135:0/2395760535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c2800e2e0 msgr2=0x7f9c280dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:52.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2f7fe640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c2800e2e0 0x7f9c280dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:52.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2f7fe640 1 -- 172.21.15.135:0/2395760535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c280dbea0 msgr2=0x7f9c28045670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:52.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2f7fe640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c280dbea0 0x7f9c28045670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:52.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2f7fe640 1 -- 172.21.15.135:0/2395760535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c280daa30 con 0x7f9c2804e110 2024-04-26T22:07:52.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2f7fe640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2804e110 0x7f9c28047600 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b3e0 tx=0x7f9c1c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:52.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c015020 con 0x7f9c2804e110 2024-04-26T22:07:52.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1c011830 con 0x7f9c2804e110 2024-04-26T22:07:52.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.544+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c011c20 con 0x7f9c2804e110 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.546+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c280d2e10 con 0x7f9c2804e110 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.546+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c280db7b0 con 0x7f9c2804e110 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.549+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c280db2c0 con 0x7f9c2804e110 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.551+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c1c018020 con 0x7f9c2804e110 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.552+0000 7f9c2cff9640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c1406c9c0 0x7f9c1406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.552+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f9c1c08be10 con 0x7f9c2804e110 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.553+0000 7f9c2ffff640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c1406c9c0 0x7f9c1406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:52.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.554+0000 7f9c2ffff640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c1406c9c0 0x7f9c1406ee80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c30075230 tx=0x7f9c3006b630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:52.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:52.559+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c1c01f030 con 0x7f9c2804e110 2024-04-26T22:07:53.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.011+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c2804da80 con 0x7f9c2804e110 2024-04-26T22:07:53.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.013+0000 7f9c2cff9640 1 -- 172.21.15.135:0/2395760535 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7f9c1c059b40 con 0x7f9c2804e110 2024-04-26T22:07:53.016 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:07:53.016 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c1406c9c0 msgr2=0x7f9c1406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c1406c9c0 0x7f9c1406ee80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c30075230 tx=0x7f9c3006b630 comp rx=0 tx=0).stop 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2804e110 msgr2=0x7f9c28047600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2804e110 0x7f9c28047600 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b3e0 tx=0x7f9c1c00b8a0 comp rx=0 tx=0).stop 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 shutdown_connections 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c1406c9c0 0x7f9c1406ee80 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c30075230 tx=0x7f9c3006b630 comp rx=0 tx=0).stop 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c280dbea0 0x7f9c28045670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c2804e110 0x7f9c28047600 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:53.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 --2- 172.21.15.135:0/2395760535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c2800e2e0 0x7f9c280dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:53.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.017+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 >> 172.21.15.135:0/2395760535 conn(0x7f9c2804c600 msgr2=0x7f9c2802fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:53.019 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.020+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 shutdown_connections 2024-04-26T22:07:53.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:07:53.020+0000 7f9c3523e640 1 -- 172.21.15.135:0/2395760535 wait complete. 2024-04-26T22:07:53.043 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-04-26T22:07:53.043 INFO:tasks.mds_thrash.fs.[b]:waiting for 50 secs before thrashing 2024-04-26T22:08:00.700 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:08:00.702 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:08:30.835 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:08:30.838 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:08:43.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:43.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.167+0000 7fad6a647640 1 Processor -- start 2024-04-26T22:08:43.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.167+0000 7fad6a647640 1 -- start start 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad6a647640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad64154160 0x7fad64156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad6a647640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad640ca5a0 0x7fad6415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad6a647640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640c6e00 0x7fad640c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad6a647640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fad6415a6e0 con 0x7fad640ca5a0 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad6a647640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fad640c9100 con 0x7fad64154160 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad6a647640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad640c8df0 con 0x7fad640c6e00 2024-04-26T22:08:43.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad640ca5a0 0x7fad6415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad640ca5a0 0x7fad6415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45350/0 (socket says 172.21.15.135:45350) 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 -- 172.21.15.135:0/3122900435 learned_addr learned my addr 172.21.15.135:0/3122900435 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad63fff640 1 --2- 172.21.15.135:0/3122900435 >> v2:172.21.15.139:3300/0 conn(0x7fad64154160 0x7fad64156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad68be4640 1 --2- 172.21.15.135:0/3122900435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640c6e00 0x7fad640c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 -- 172.21.15.135:0/3122900435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640c6e00 msgr2=0x7fad640c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 --2- 172.21.15.135:0/3122900435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640c6e00 0x7fad640c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 -- 172.21.15.135:0/3122900435 >> v2:172.21.15.139:3300/0 conn(0x7fad64154160 msgr2=0x7fad64156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 --2- 172.21.15.135:0/3122900435 >> v2:172.21.15.139:3300/0 conn(0x7fad64154160 0x7fad64156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.168+0000 7fad637fe640 1 -- 172.21.15.135:0/3122900435 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad64154540 con 0x7fad640ca5a0 2024-04-26T22:08:43.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.169+0000 7fad637fe640 1 --2- 172.21.15.135:0/3122900435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640ca5a0 0x7fad6415a1a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad6406da80 tx=0x7fad5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103dc3eacd1c353e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:43.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.169+0000 7fad62ffd640 1 -- 172.21.15.135:0/3122900435 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58037070 con 0x7fad640ca5a0 2024-04-26T22:08:43.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.169+0000 7fad62ffd640 1 -- 172.21.15.135:0/3122900435 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad58032440 con 0x7fad640ca5a0 2024-04-26T22:08:43.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.169+0000 7fad62ffd640 1 -- 172.21.15.135:0/3122900435 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58032810 con 0x7fad640ca5a0 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 -- 172.21.15.135:0/3122900435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640ca5a0 msgr2=0x7fad6415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 --2- 172.21.15.135:0/3122900435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640ca5a0 0x7fad6415a1a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad6406da80 tx=0x7fad5802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 -- 172.21.15.135:0/3122900435 shutdown_connections 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 --2- 172.21.15.135:0/3122900435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640ca5a0 0x7fad6415a1a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 --2- 172.21.15.135:0/3122900435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640c6e00 0x7fad640c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 --2- 172.21.15.135:0/3122900435 >> v2:172.21.15.139:3300/0 conn(0x7fad64154160 0x7fad64156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 -- 172.21.15.135:0/3122900435 >> 172.21.15.135:0/3122900435 conn(0x7fad640ae010 msgr2=0x7fad640ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 -- 172.21.15.135:0/3122900435 shutdown_connections 2024-04-26T22:08:43.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.170+0000 7fad6a647640 1 -- 172.21.15.135:0/3122900435 wait complete. 2024-04-26T22:08:43.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.171+0000 7fad6a647640 1 Processor -- start 2024-04-26T22:08:43.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.171+0000 7fad6a647640 1 -- start start 2024-04-26T22:08:43.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad6a647640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 0x7fad640b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad6a647640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640ca5a0 0x7fad640cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad6a647640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad64154160 0x7fad640cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad6a647640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6415aed0 con 0x7fad640c6e00 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad6a647640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad6415abf0 con 0x7fad64154160 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad6a647640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad640c9590 con 0x7fad640ca5a0 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 0x7fad640b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 0x7fad640b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45358/0 (socket says 172.21.15.135:45358) 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 -- 172.21.15.135:0/3171195487 learned_addr learned my addr 172.21.15.135:0/3171195487 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad68be4640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad64154160 0x7fad640cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad637fe640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640ca5a0 0x7fad640cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 -- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640ca5a0 msgr2=0x7fad640cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640ca5a0 0x7fad640cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 -- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad64154160 msgr2=0x7fad640cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad64154160 0x7fad640cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 -- 172.21.15.135:0/3171195487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad64158100 con 0x7fad640c6e00 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.172+0000 7fad63fff640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 0x7fad640b1bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e870 tx=0x7fad5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.173+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54017070 con 0x7fad640c6e00 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.173+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5400c940 con 0x7fad640c6e00 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.173+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5400cd30 con 0x7fad640c6e00 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.173+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad641f0230 con 0x7fad640c6e00 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.173+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641ed080 con 0x7fad640c6e00 2024-04-26T22:08:43.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.174+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad641ecb90 con 0x7fad640c6e00 2024-04-26T22:08:43.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.174+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad5401a070 con 0x7fad640c6e00 2024-04-26T22:08:43.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.175+0000 7fad60ff9640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad5006c8f0 0x7fad5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.175+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7fad5408bf60 con 0x7fad640c6e00 2024-04-26T22:08:43.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.175+0000 7fad637fe640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad5006c8f0 0x7fad5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.175+0000 7fad637fe640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad5006c8f0 0x7fad5006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad58034000 tx=0x7fad5802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:43.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.179+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad64158100 con 0x7fad640c6e00 2024-04-26T22:08:43.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.538+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad64076650 con 0x7fad640c6e00 2024-04-26T22:08:43.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.539+0000 7fad60ff9640 1 -- 172.21.15.135:0/3171195487 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7fad54059660 con 0x7fad640c6e00 2024-04-26T22:08:43.539 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:43.539 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:43.540 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad5006c8f0 msgr2=0x7fad5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad5006c8f0 0x7fad5006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad58034000 tx=0x7fad5802e000 comp rx=0 tx=0).stop 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 msgr2=0x7fad640b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 0x7fad640b1bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e870 tx=0x7fad5400ed30 comp rx=0 tx=0).stop 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 shutdown_connections 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fad5006c8f0 0x7fad5006edb0 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad58034000 tx=0x7fad5802e000 comp rx=0 tx=0).stop 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad64154160 0x7fad640cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad640ca5a0 0x7fad640cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 --2- 172.21.15.135:0/3171195487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad640c6e00 0x7fad640b1bf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 >> 172.21.15.135:0/3171195487 conn(0x7fad640ae010 msgr2=0x7fad6414ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 shutdown_connections 2024-04-26T22:08:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.543+0000 7fad6a647640 1 -- 172.21.15.135:0/3171195487 wait complete. 2024-04-26T22:08:43.561 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-04-26T22:08:43.561 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T22:08:43.561 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:08:43.582 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T22:08:43.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:43.703 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.703+0000 7f30fd732640 1 Processor -- start 2024-04-26T22:08:43.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.704+0000 7f30fd732640 1 -- start start 2024-04-26T22:08:43.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30fd732640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30f80ae3d0 0x7f30f80c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30fd732640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80a80a0 0x7f30f80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30fd732640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f8070750 con 0x7f30f80c5480 2024-04-26T22:08:43.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30fd732640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f30f81565b0 con 0x7f30f80ae3d0 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30fd732640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f30f8156800 con 0x7f30f80a80a0 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30f80ae3d0 0x7f30f80c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f30f80c5480 0x7f30f806ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:60438/0 (socket says 172.21.15.135:60438) 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f77fe640 1 -- 172.21.15.135:0/3438793590 learned_addr learned my addr 172.21.15.135:0/3438793590 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f6ffd640 1 -- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80a80a0 msgr2=0x7f30f80ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f6ffd640 1 --2- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80a80a0 0x7f30f80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f6ffd640 1 -- 172.21.15.135:0/3438793590 >> v1:172.21.15.139:6789/0 conn(0x7f30f80c5480 legacy=0x7f30f806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.705+0000 7f30f6ffd640 1 -- 172.21.15.135:0/3438793590 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f8156ae0 con 0x7f30f80ae3d0 2024-04-26T22:08:43.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.706+0000 7f30f6ffd640 1 --2- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c8250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097a0 tx=0x7f30e802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2493cd4e3845d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:43.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30f5ffb640 1 -- 172.21.15.135:0/3438793590 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8037070 con 0x7f30f80ae3d0 2024-04-26T22:08:43.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30f5ffb640 1 -- 172.21.15.135:0/3438793590 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8032440 con 0x7f30f80ae3d0 2024-04-26T22:08:43.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30f5ffb640 1 -- 172.21.15.135:0/3438793590 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8032810 con 0x7f30f80ae3d0 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 -- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 msgr2=0x7f30f80c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 --2- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c8250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097a0 tx=0x7f30e802a180 comp rx=0 tx=0).stop 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 -- 172.21.15.135:0/3438793590 shutdown_connections 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 --2- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80a80a0 0x7f30f80ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 --2- 172.21.15.135:0/3438793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c8250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 -- 172.21.15.135:0/3438793590 >> 172.21.15.135:0/3438793590 conn(0x7f30f80c89f0 msgr2=0x7f30f80c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 -- 172.21.15.135:0/3438793590 shutdown_connections 2024-04-26T22:08:43.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.707+0000 7f30fd732640 1 -- 172.21.15.135:0/3438793590 wait complete. 2024-04-26T22:08:43.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 Processor -- start 2024-04-26T22:08:43.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 -- start start 2024-04-26T22:08:43.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 0x7f30f81445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80c5480 0x7f30f814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80c5890 con 0x7f30f80a80a0 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f30f81570a0 con 0x7f30f80ae3d0 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.709+0000 7f30fd732640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f30f80c5a10 con 0x7f30f80c5480 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 0x7f30f81445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 0x7f30f81445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45362/0 (socket says 172.21.15.135:45362) 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 -- 172.21.15.135:0/2673461959 learned_addr learned my addr 172.21.15.135:0/2673461959 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f67fc640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f77fe640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80c5480 0x7f30f814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 -- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80c5480 msgr2=0x7f30f814a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80c5480 0x7f30f814a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 -- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 msgr2=0x7f30f80c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 -- 172.21.15.135:0/2673461959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f814eed0 con 0x7f30f80a80a0 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30f6ffd640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 0x7f30f81445d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097a0 tx=0x7f30e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8041070 con 0x7f30f80a80a0 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e803c890 con 0x7f30f80a80a0 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f81efdc0 con 0x7f30f80a80a0 2024-04-26T22:08:43.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.710+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81f0380 con 0x7f30f80a80a0 2024-04-26T22:08:43.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.711+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8037040 con 0x7f30f80a80a0 2024-04-26T22:08:43.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.712+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f80765e0 con 0x7f30f80a80a0 2024-04-26T22:08:43.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.712+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30e8044070 con 0x7f30f80a80a0 2024-04-26T22:08:43.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.712+0000 7f30df7fe640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30e406c920 0x7f30e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:43.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.712+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f30e80acd10 con 0x7f30f80a80a0 2024-04-26T22:08:43.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.713+0000 7f30f67fc640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30e406c920 0x7f30e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:43.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.713+0000 7f30f67fc640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30e406c920 0x7f30e406ede0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30f80c2990 tx=0x7f30ec009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:43.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:43.718+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30e807a530 con 0x7f30f80a80a0 2024-04-26T22:08:44.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.136+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f80507a0 con 0x7f30f80a80a0 2024-04-26T22:08:44.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.138+0000 7f30df7fe640 1 -- 172.21.15.135:0/2673461959 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7f30e807e830 con 0x7f30f80a80a0 2024-04-26T22:08:44.139 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:44.139 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:44.140 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30e406c920 msgr2=0x7f30e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30e406c920 0x7f30e406ede0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30f80c2990 tx=0x7f30ec009110 comp rx=0 tx=0).stop 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 msgr2=0x7f30f81445d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 0x7f30f81445d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097a0 tx=0x7f30e80046c0 comp rx=0 tx=0).stop 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 shutdown_connections 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f30e406c920 0x7f30e406ede0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30f80c5480 0x7f30f814a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30f80ae3d0 0x7f30f80c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 --2- 172.21.15.135:0/2673461959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30f80a80a0 0x7f30f81445d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 >> 172.21.15.135:0/2673461959 conn(0x7f30f80c89f0 msgr2=0x7f30f80ac7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 shutdown_connections 2024-04-26T22:08:44.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:44.143+0000 7f30fd732640 1 -- 172.21.15.135:0/2673461959 wait complete. 2024-04-26T22:08:44.164 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:08:46.166 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:46.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.340+0000 7fdf227a7640 1 Processor -- start 2024-04-26T22:08:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.341+0000 7fdf227a7640 1 -- start start 2024-04-26T22:08:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.342+0000 7fdf227a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 0x7fdf1c15f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.342+0000 7fdf227a7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdf1c0cb830 0x7fdf1c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.342+0000 7fdf227a7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0cbd10 con 0x7fdf1c0cc5f0 2024-04-26T22:08:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.342+0000 7fdf227a7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c8c40 con 0x7fdf1c0cb830 2024-04-26T22:08:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.342+0000 7fdf227a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0cac10 con 0x7fdf1c158980 2024-04-26T22:08:46.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.343+0000 7fdf20fa4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 0x7fdf1c15f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:46.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.343+0000 7fdf20fa4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 0x7fdf1c15f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:52810/0 (socket says 172.21.15.135:52810) 2024-04-26T22:08:46.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.343+0000 7fdf20fa4640 1 -- 172.21.15.135:0/2583156378 learned_addr learned my addr 172.21.15.135:0/2583156378 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:46.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.344+0000 7fdf20fa4640 1 -- 172.21.15.135:0/2583156378 >> v2:172.21.15.135:3300/0 conn(0x7fdf1c0cb830 msgr2=0x7fdf1c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:46.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.344+0000 7fdf20fa4640 1 --2- 172.21.15.135:0/2583156378 >> v2:172.21.15.135:3300/0 conn(0x7fdf1c0cb830 0x7fdf1c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.344+0000 7fdf20fa4640 1 -- 172.21.15.135:0/2583156378 >> v1:172.21.15.135:6789/0 conn(0x7fdf1c0cc5f0 legacy=0x7fdf1c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.344+0000 7fdf20fa4640 1 -- 172.21.15.135:0/2583156378 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c0caef0 con 0x7fdf1c158980 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.345+0000 7fdf20fa4640 1 --2- 172.21.15.135:0/2583156378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 0x7fdf1c15f010 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea4beaa613088873 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.347+0000 7fdf13fff640 1 -- 172.21.15.135:0/2583156378 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18037070 con 0x7fdf1c158980 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.347+0000 7fdf13fff640 1 -- 172.21.15.135:0/2583156378 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf18032440 con 0x7fdf1c158980 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.347+0000 7fdf13fff640 1 -- 172.21.15.135:0/2583156378 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18032810 con 0x7fdf1c158980 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.348+0000 7fdf227a7640 1 -- 172.21.15.135:0/2583156378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 msgr2=0x7fdf1c15f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:46.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.348+0000 7fdf227a7640 1 --2- 172.21.15.135:0/2583156378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 0x7fdf1c15f010 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802a200 comp rx=0 tx=0).stop 2024-04-26T22:08:46.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.351+0000 7fdf227a7640 1 -- 172.21.15.135:0/2583156378 shutdown_connections 2024-04-26T22:08:46.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.351+0000 7fdf227a7640 1 --2- 172.21.15.135:0/2583156378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c158980 0x7fdf1c15f010 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.351+0000 7fdf227a7640 1 --2- 172.21.15.135:0/2583156378 >> v2:172.21.15.135:3300/0 conn(0x7fdf1c0cb830 0x7fdf1c0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.351+0000 7fdf227a7640 1 -- 172.21.15.135:0/2583156378 >> 172.21.15.135:0/2583156378 conn(0x7fdf1c0ae180 msgr2=0x7fdf1c0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:46.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.356+0000 7fdf227a7640 1 -- 172.21.15.135:0/2583156378 shutdown_connections 2024-04-26T22:08:46.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.359+0000 7fdf227a7640 1 -- 172.21.15.135:0/2583156378 wait complete. 2024-04-26T22:08:46.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.360+0000 7fdf227a7640 1 Processor -- start 2024-04-26T22:08:46.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.362+0000 7fdf227a7640 1 -- start start 2024-04-26T22:08:46.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf227a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 0x7fdf1c150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf227a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c0cc5f0 0x7fdf1c1ec190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf227a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c158980 0x7fdf1c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf227a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c0bfc50 con 0x7fdf1c0cb830 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf227a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c0bfad0 con 0x7fdf1c158980 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf227a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0cc9d0 con 0x7fdf1c0cc5f0 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 0x7fdf1c150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf21fa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c158980 0x7fdf1c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 0x7fdf1c150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:45390/0 (socket says 172.21.15.135:45390) 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 -- 172.21.15.135:0/1248363334 learned_addr learned my addr 172.21.15.135:0/1248363334 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf20fa4640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c0cc5f0 0x7fdf1c1ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 -- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c0cc5f0 msgr2=0x7fdf1c1ec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c0cc5f0 0x7fdf1c1ec190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 -- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c158980 msgr2=0x7fdf1c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:46.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.363+0000 7fdf217a5640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c158980 0x7fdf1c15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.364+0000 7fdf217a5640 1 -- 172.21.15.135:0/1248363334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c15bd30 con 0x7fdf1c0cb830 2024-04-26T22:08:46.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.364+0000 7fdf217a5640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 0x7fdf1c150bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c450 tx=0x7fdf0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:46.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.367+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0052d0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.367+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c005470 con 0x7fdf1c0cb830 2024-04-26T22:08:46.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.367+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c012850 con 0x7fdf1c0cb830 2024-04-26T22:08:46.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.369+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1511d0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.370+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c15aff0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.376+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c069bc0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.376+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf0c0129f0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.377+0000 7fdf11ffb640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c9c0 0x7fdf0006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:46.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.377+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7fdf0c08bdd0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.377+0000 7fdf20fa4640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c9c0 0x7fdf0006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:46.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.378+0000 7fdf20fa4640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c9c0 0x7fdf0006ee80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf18034000 tx=0x7fdf1802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:46.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.387+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf1c069bc0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.815+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf1c0506c0 con 0x7fdf1c0cb830 2024-04-26T22:08:46.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.818+0000 7fdf11ffb640 1 -- 172.21.15.135:0/1248363334 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7fdf0c059b00 con 0x7fdf1c0cb830 2024-04-26T22:08:46.818 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:46.819 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:46.819 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:46.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.824+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c9c0 msgr2=0x7fdf0006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.824+0000 7fdf227a7640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c9c0 0x7fdf0006ee80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf18034000 tx=0x7fdf1802e000 comp rx=0 tx=0).stop 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.825+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 msgr2=0x7fdf1c150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.825+0000 7fdf227a7640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 0x7fdf1c150bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c450 tx=0x7fdf0c00c910 comp rx=0 tx=0).stop 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 shutdown_connections 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c9c0 0x7fdf0006ee80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1c158980 0x7fdf1c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1c0cc5f0 0x7fdf1c1ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 --2- 172.21.15.135:0/1248363334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf1c0cb830 0x7fdf1c150bf0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 >> 172.21.15.135:0/1248363334 conn(0x7fdf1c0ae180 msgr2=0x7fdf1c0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 shutdown_connections 2024-04-26T22:08:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:46.826+0000 7fdf227a7640 1 -- 172.21.15.135:0/1248363334 wait complete. 2024-04-26T22:08:46.846 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:08:48.847 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:48.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.960+0000 7f3b65d67640 1 Processor -- start 2024-04-26T22:08:48.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.960+0000 7f3b65d67640 1 -- start start 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.960+0000 7f3b65d67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 0x7f3b600c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.960+0000 7f3b65d67640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b600c3fc0 con 0x7f3b601541d0 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b65d67640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b600c3500 con 0x7f3b6015a3c0 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b65d67640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b600c37e0 con 0x7f3b60070490 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3b601541d0 0x7f3b600c7420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:58356/0 (socket says 172.21.15.135:58356) 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/3091627433 learned_addr learned my addr 172.21.15.135:0/3091627433 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5effd640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 0x7f3b600c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253913715 0 0) 0x7f3b600c3fc0 con 0x7f3b601541d0 2024-04-26T22:08:48.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b48003520 con 0x7f3b601541d0 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3844931268 0 0) 0x7f3b48003520 con 0x7f3b601541d0 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 msgr2=0x7f3b600c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 0x7f3b600c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 >> v1:172.21.15.139:6789/0 conn(0x7f3b6015a3c0 legacy=0x7f3b6015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600c3ac0 con 0x7f3b601541d0 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f3b50001670 con 0x7f3b601541d0 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 >> v1:172.21.15.135:6789/0 conn(0x7f3b601541d0 legacy=0x7f3b600c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 0x7f3b48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b48004d40 0x7f3b480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600c37e0 con 0x7f3b480039e0 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4800ba70 con 0x7f3b48004d40 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.961+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4800bc70 con 0x7f3b60070490 2024-04-26T22:08:48.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5effd640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 0x7f3b48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 msgr2=0x7f3b48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:48.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b60070490 0x7f3b48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:48.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b48004d40 msgr2=0x7f3b480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:48.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b48004d40 0x7f3b480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:48.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.962+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/3091627433 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600c3ac0 con 0x7f3b480039e0 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.963+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b48003de0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000c370 tx=0x7f3b48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8835cbfb66f10076 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.963+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50007e20 con 0x7f3b480039e0 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.963+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b50034070 con 0x7f3b480039e0 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.963+0000 7f3b5e7fc640 1 -- 172.21.15.135:0/3091627433 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50005cf0 con 0x7f3b480039e0 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b65d67640 1 -- 172.21.15.135:0/3091627433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 msgr2=0x7f3b48003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b65d67640 1 --2- 172.21.15.135:0/3091627433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b48003de0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000c370 tx=0x7f3b48005830 comp rx=0 tx=0).stop 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b5ffff640 1 -- 172.21.15.135:0/3091627433 reap_dead start 2024-04-26T22:08:48.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b65d67640 1 -- 172.21.15.135:0/3091627433 shutdown_connections 2024-04-26T22:08:48.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b65d67640 1 -- 172.21.15.135:0/3091627433 >> 172.21.15.135:0/3091627433 conn(0x7f3b600c8c40 msgr2=0x7f3b600c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:48.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b65d67640 1 -- 172.21.15.135:0/3091627433 shutdown_connections 2024-04-26T22:08:48.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.964+0000 7f3b65d67640 1 -- 172.21.15.135:0/3091627433 wait complete. 2024-04-26T22:08:48.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.966+0000 7f3b65d67640 1 Processor -- start 2024-04-26T22:08:48.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.966+0000 7f3b65d67640 1 -- start start 2024-04-26T22:08:48.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b65d67640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b600ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b65d67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6006f160 0x7f3b601ec710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b65d67640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6006f620 0x7f3b601ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b65d67640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600c7b30 con 0x7f3b480039e0 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b65d67640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b601545b0 con 0x7f3b6006f620 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b65d67640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b600c7cb0 con 0x7f3b6006f160 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b5f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b600ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b5f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b600ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57876/0 (socket says 172.21.15.135:57876) 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/135109540 learned_addr learned my addr 172.21.15.135:0/135109540 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b5ffff640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6006f620 0x7f3b601ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.967+0000 7f3b5effd640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6006f160 0x7f3b601ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6006f160 msgr2=0x7f3b601ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6006f160 0x7f3b601ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6006f620 msgr2=0x7f3b601ecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6006f620 0x7f3b601ecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b5f7fe640 1 -- 172.21.15.135:0/135109540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b60074b60 con 0x7f3b480039e0 2024-04-26T22:08:48.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b5f7fe640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b600ad450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000c490 tx=0x7f3b50001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:48.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50007e20 con 0x7f3b480039e0 2024-04-26T22:08:48.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b50007480 con 0x7f3b480039e0 2024-04-26T22:08:48.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50044440 con 0x7f3b480039e0 2024-04-26T22:08:48.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.968+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b601efdb0 con 0x7f3b480039e0 2024-04-26T22:08:48.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.969+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b601f0370 con 0x7f3b480039e0 2024-04-26T22:08:48.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.970+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b600507a0 con 0x7f3b480039e0 2024-04-26T22:08:48.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.970+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b5002e070 con 0x7f3b480039e0 2024-04-26T22:08:48.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.971+0000 7f3b47fff640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b4c06c970 0x7f3b4c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:48.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.971+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f3b500abd50 con 0x7f3b480039e0 2024-04-26T22:08:48.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.972+0000 7f3b5effd640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b4c06c970 0x7f3b4c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:48.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.972+0000 7f3b5effd640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b4c06c970 0x7f3b4c06ee30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3b6006e0d0 tx=0x7f3b54009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:48.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:48.979+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b5002c040 con 0x7f3b480039e0 2024-04-26T22:08:49.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.371+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b600c6aa0 con 0x7f3b480039e0 2024-04-26T22:08:49.372 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.372+0000 7f3b47fff640 1 -- 172.21.15.135:0/135109540 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7f3b5002c040 con 0x7f3b480039e0 2024-04-26T22:08:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:49.372 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:49.372 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b4c06c970 msgr2=0x7f3b4c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b4c06c970 0x7f3b4c06ee30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3b6006e0d0 tx=0x7f3b54009190 comp rx=0 tx=0).stop 2024-04-26T22:08:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 msgr2=0x7f3b600ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b600ad450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000c490 tx=0x7f3b50001a40 comp rx=0 tx=0).stop 2024-04-26T22:08:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 shutdown_connections 2024-04-26T22:08:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b4c06c970 0x7f3b4c06ee30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:49.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6006f620 0x7f3b601ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:49.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6006f160 0x7f3b601ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:49.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 --2- 172.21.15.135:0/135109540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b480039e0 0x7f3b600ad450 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:49.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 >> 172.21.15.135:0/135109540 conn(0x7f3b600c8c40 msgr2=0x7f3b600c7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:49.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 shutdown_connections 2024-04-26T22:08:49.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:49.374+0000 7f3b65d67640 1 -- 172.21.15.135:0/135109540 wait complete. 2024-04-26T22:08:49.394 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:08:51.395 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:51.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.544+0000 7f4120967640 1 Processor -- start 2024-04-26T22:08:51.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.544+0000 7f4120967640 1 -- start start 2024-04-26T22:08:51.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f4120967640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f411c154160 0x7f411c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f4120967640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c15a1a0 0x7f411c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f4120967640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0c8df0 con 0x7f411c0ca5a0 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f4120967640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f411c0c9100 con 0x7f411c154160 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f4120967640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f411c154540 con 0x7f411c15a1a0 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f411a59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f411c154160 0x7f411c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f411a59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f411c154160 0x7f411c156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57890/0 (socket says 172.21.15.135:57890) 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f411a59c640 1 -- 172.21.15.135:0/113646994 learned_addr learned my addr 172.21.15.135:0/113646994 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.545+0000 7f411ad9d640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c15a1a0 0x7f411c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411a59c640 1 -- 172.21.15.135:0/113646994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c15a1a0 msgr2=0x7f411c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411a59c640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c15a1a0 0x7f411c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411a59c640 1 -- 172.21.15.135:0/113646994 >> v1:172.21.15.139:6789/0 conn(0x7f411c0ca5a0 legacy=0x7f411c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:51.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411a59c640 1 -- 172.21.15.135:0/113646994 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c15a910 con 0x7f411c154160 2024-04-26T22:08:51.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411ad9d640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c15a1a0 0x7f411c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411a59c640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c156bb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f410c009870 tx=0x7f410c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38b84f2cb5f8c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.546+0000 7f411959a640 1 -- 172.21.15.135:0/113646994 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c037070 con 0x7f411c154160 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f411959a640 1 -- 172.21.15.135:0/113646994 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f410c032440 con 0x7f411c154160 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f411959a640 1 -- 172.21.15.135:0/113646994 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c032810 con 0x7f411c154160 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 -- 172.21.15.135:0/113646994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 msgr2=0x7f411c156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c156bb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f410c009870 tx=0x7f410c02a210 comp rx=0 tx=0).stop 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 -- 172.21.15.135:0/113646994 shutdown_connections 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c15a1a0 0x7f411c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 --2- 172.21.15.135:0/113646994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c156bb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 -- 172.21.15.135:0/113646994 >> 172.21.15.135:0/113646994 conn(0x7f411c0ae010 msgr2=0x7f411c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 -- 172.21.15.135:0/113646994 shutdown_connections 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.547+0000 7f4120967640 1 -- 172.21.15.135:0/113646994 wait complete. 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.548+0000 7f4120967640 1 Processor -- start 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.548+0000 7f4120967640 1 -- start start 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4120967640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c0ca5a0 0x7f411c1ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4120967640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c1ed530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4120967640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f411c15a1a0 0x7f411c0c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4120967640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f411c15a5f0 con 0x7f411c154160 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4120967640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f411c15aed0 con 0x7f411c15a1a0 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4120967640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0c7510 con 0x7f411c0ca5a0 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c1ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c1ed530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57892/0 (socket says 172.21.15.135:57892) 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 -- 172.21.15.135:0/1797387872 learned_addr learned my addr 172.21.15.135:0/1797387872 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:51.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f411ad9d640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f411c15a1a0 0x7f411c0c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f411a59c640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c0ca5a0 0x7f411c1ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 -- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c0ca5a0 msgr2=0x7f411c1ecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c0ca5a0 0x7f411c1ecff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 -- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f411c15a1a0 msgr2=0x7f411c0c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f411c15a1a0 0x7f411c0c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.549+0000 7f4119d9b640 1 -- 172.21.15.135:0/1797387872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0c4390 con 0x7f411c154160 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.550+0000 7f4119d9b640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c1ed530 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f411000c460 tx=0x7f411000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.550+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110016020 con 0x7f411c154160 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.550+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4110012870 con 0x7f411c154160 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.550+0000 7f4120967640 1 -- 172.21.15.135:0/1797387872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c1589b0 con 0x7f411c154160 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.550+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110012c60 con 0x7f411c154160 2024-04-26T22:08:51.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.550+0000 7f4120967640 1 -- 172.21.15.135:0/1797387872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c158c70 con 0x7f411c154160 2024-04-26T22:08:51.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.551+0000 7f4120967640 1 -- 172.21.15.135:0/1797387872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c159440 con 0x7f411c154160 2024-04-26T22:08:51.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.551+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4110019020 con 0x7f411c154160 2024-04-26T22:08:51.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.552+0000 7f410affd640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f40fc06c920 0x7f40fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:51.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.552+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f411008bef0 con 0x7f411c154160 2024-04-26T22:08:51.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.552+0000 7f411a59c640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f40fc06c920 0x7f40fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:51.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.552+0000 7f411a59c640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f40fc06c920 0x7f40fc06ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:51.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.556+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4110020030 con 0x7f411c154160 2024-04-26T22:08:51.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.851+0000 7f4120967640 1 -- 172.21.15.135:0/1797387872 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f411c076650 con 0x7f411c154160 2024-04-26T22:08:51.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.852+0000 7f410affd640 1 -- 172.21.15.135:0/1797387872 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7f4110059bb0 con 0x7f411c154160 2024-04-26T22:08:51.855 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:51.855 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:51.855 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:51.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.861+0000 7f4108ff9640 1 -- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f40fc06c920 msgr2=0x7f40fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:51.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.861+0000 7f4108ff9640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f40fc06c920 0x7f40fc06ede0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c02e000 comp rx=0 tx=0).stop 2024-04-26T22:08:51.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.861+0000 7f4108ff9640 1 -- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 msgr2=0x7f411c1ed530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:51.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.861+0000 7f4108ff9640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c1ed530 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f411000c460 tx=0x7f411000c920 comp rx=0 tx=0).stop 2024-04-26T22:08:51.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.865+0000 7f4108ff9640 1 -- 172.21.15.135:0/1797387872 shutdown_connections 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.865+0000 7f4108ff9640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f40fc06c920 0x7f40fc06ede0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.865+0000 7f4108ff9640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f411c15a1a0 0x7f411c0c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.865+0000 7f4108ff9640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f411c154160 0x7f411c1ed530 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.865+0000 7f4108ff9640 1 --2- 172.21.15.135:0/1797387872 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f411c0ca5a0 0x7f411c1ecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.865+0000 7f4108ff9640 1 -- 172.21.15.135:0/1797387872 >> 172.21.15.135:0/1797387872 conn(0x7f411c0ae010 msgr2=0x7f411c14f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.866+0000 7f4108ff9640 1 -- 172.21.15.135:0/1797387872 shutdown_connections 2024-04-26T22:08:51.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:51.866+0000 7f4108ff9640 1 -- 172.21.15.135:0/1797387872 wait complete. 2024-04-26T22:08:51.886 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:08:53.887 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:54.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.091+0000 7fefa3fa4640 1 Processor -- start 2024-04-26T22:08:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.094+0000 7fefa3fa4640 1 -- start start 2024-04-26T22:08:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa3fa4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef9c1588c0 0x7fef9c0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa3fa4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 0x7fef9c0ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa3fa4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9c0c3830 con 0x7fef9c15e660 2024-04-26T22:08:54.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa3fa4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fef9c074710 con 0x7fef9c1588c0 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa3fa4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef9c0c4680 con 0x7fef9c0ac090 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa2541640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fef9c15e660 0x7fef9c0c5300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:58362/0 (socket says 172.21.15.135:58362) 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.095+0000 7fefa2541640 1 -- 172.21.15.135:0/3421473838 learned_addr learned my addr 172.21.15.135:0/3421473838 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.096+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119507283 0 0) 0x7fef9c0c3830 con 0x7fef9c15e660 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.096+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef88003520 con 0x7fef9c15e660 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa153f640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 0x7fef9c0ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:54.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3977168064 0 0) 0x7fef88003520 con 0x7fef9c15e660 2024-04-26T22:08:54.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 msgr2=0x7fef9c0ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:54.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 0x7fef9c0ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 >> v2:172.21.15.139:3300/0 conn(0x7fef9c1588c0 msgr2=0x7fef9c0c4b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:08:54.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 --2- 172.21.15.135:0/3421473838 >> v2:172.21.15.139:3300/0 conn(0x7fef9c1588c0 0x7fef9c0c4b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c0c3b10 con 0x7fef9c15e660 2024-04-26T22:08:54.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.097+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fef98001670 con 0x7fef9c15e660 2024-04-26T22:08:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 >> v1:172.21.15.135:6789/0 conn(0x7fef9c15e660 legacy=0x7fef9c0c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef880039b0 0x7fef88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef88004800 0x7fef88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 0x7fef880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c074710 con 0x7fef880039b0 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c0c4680 con 0x7fef88004800 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef8800bb40 con 0x7fef9c0ac090 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa1d40640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef880039b0 0x7fef88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.098+0000 7fefa2541640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 0x7fef880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.099+0000 7fefa1d40640 1 -- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 msgr2=0x7fef880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.099+0000 7fefa1d40640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c0ac090 0x7fef880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.099+0000 7fefa1d40640 1 -- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef88004800 msgr2=0x7fef88004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.099+0000 7fefa1d40640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef88004800 0x7fef88004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.099+0000 7fefa1d40640 1 -- 172.21.15.135:0/3421473838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c0c3b10 con 0x7fef880039b0 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.100+0000 7fefa1d40640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef880039b0 0x7fef88003dd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009860 tx=0x7fef88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18692e9ee14331a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.100+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c037070 con 0x7fef880039b0 2024-04-26T22:08:54.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.100+0000 7fefa0d3e640 1 -- 172.21.15.135:0/3421473838 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c02aa30 con 0x7fef880039b0 2024-04-26T22:08:54.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.101+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3421473838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef880039b0 msgr2=0x7fef88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:54.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.101+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3421473838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef880039b0 0x7fef88003dd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009860 tx=0x7fef88005830 comp rx=0 tx=0).stop 2024-04-26T22:08:54.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.104+0000 7fefa2541640 1 -- 172.21.15.135:0/3421473838 reap_dead start 2024-04-26T22:08:54.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.105+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3421473838 shutdown_connections 2024-04-26T22:08:54.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.105+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3421473838 >> 172.21.15.135:0/3421473838 conn(0x7fef9c0c89b0 msgr2=0x7fef9c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:54.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.107+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3421473838 shutdown_connections 2024-04-26T22:08:54.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.107+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3421473838 wait complete. 2024-04-26T22:08:54.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.109+0000 7fefa3fa4640 1 Processor -- start 2024-04-26T22:08:54.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.109+0000 7fefa3fa4640 1 -- start start 2024-04-26T22:08:54.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.110+0000 7fefa3fa4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c1374f0 0x7fef9c158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.110+0000 7fefa3fa4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c1378d0 0x7fef9c176740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.110+0000 7fefa3fa4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 0x7fef9c176c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.110+0000 7fefa3fa4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c158ca0 con 0x7fef9c137d30 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.110+0000 7fefa3fa4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c0c3df0 con 0x7fef9c1374f0 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.110+0000 7fefa3fa4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef9c0c5a10 con 0x7fef9c1378d0 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 0x7fef9c176c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 0x7fef9c176c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57910/0 (socket says 172.21.15.135:57910) 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 -- 172.21.15.135:0/3027494718 learned_addr learned my addr 172.21.15.135:0/3027494718 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 -- 172.21.15.135:0/3027494718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c1378d0 msgr2=0x7fef9c176740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c1378d0 0x7fef9c176740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 -- 172.21.15.135:0/3027494718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c1374f0 msgr2=0x7fef9c158030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c1374f0 0x7fef9c158030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.111+0000 7fefa2541640 1 -- 172.21.15.135:0/3027494718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1595e0 con 0x7fef9c137d30 2024-04-26T22:08:54.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.112+0000 7fefa2541640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 0x7fef9c176c80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fef9c06d550 tx=0x7fef9800deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:54.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.113+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98004aa0 con 0x7fef9c137d30 2024-04-26T22:08:54.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.113+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1598c0 con 0x7fef9c137d30 2024-04-26T22:08:54.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.113+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c159e80 con 0x7fef9c137d30 2024-04-26T22:08:54.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.113+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef98004c40 con 0x7fef9c137d30 2024-04-26T22:08:54.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.113+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98007070 con 0x7fef9c137d30 2024-04-26T22:08:54.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.116+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef9801b020 con 0x7fef9c137d30 2024-04-26T22:08:54.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.117+0000 7fef927fc640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef8006c9f0 0x7fef8006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:54.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.117+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7fef9808bdd0 con 0x7fef9c137d30 2024-04-26T22:08:54.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.118+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef84005290 con 0x7fef9c137d30 2024-04-26T22:08:54.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.119+0000 7fefa1d40640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef8006c9f0 0x7fef8006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:54.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.120+0000 7fefa1d40640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef8006c9f0 0x7fef8006eeb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fef8c034000 tx=0x7fef8c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:54.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.129+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef98059b00 con 0x7fef9c137d30 2024-04-26T22:08:54.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.611+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef84005580 con 0x7fef9c137d30 2024-04-26T22:08:54.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.613+0000 7fef927fc640 1 -- 172.21.15.135:0/3027494718 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7fef98059400 con 0x7fef9c137d30 2024-04-26T22:08:54.616 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:54.616 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:54.616 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef8006c9f0 msgr2=0x7fef8006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef8006c9f0 0x7fef8006eeb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fef8c034000 tx=0x7fef8c02e000 comp rx=0 tx=0).stop 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 msgr2=0x7fef9c176c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 0x7fef9c176c80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fef9c06d550 tx=0x7fef9800deb0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 shutdown_connections 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef8006c9f0 0x7fef8006eeb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c137d30 0x7fef9c176c80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9c1378d0 0x7fef9c176740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 --2- 172.21.15.135:0/3027494718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9c1374f0 0x7fef9c158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:54.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.617+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 >> 172.21.15.135:0/3027494718 conn(0x7fef9c0c89b0 msgr2=0x7fef9c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:54.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.621+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 shutdown_connections 2024-04-26T22:08:54.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:54.621+0000 7fefa3fa4640 1 -- 172.21.15.135:0/3027494718 wait complete. 2024-04-26T22:08:54.640 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout:Syncing()...59 sec 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout:FFSB benchmark finished at: Fri Apr 26 22:08:56 2024 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout:Results: 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout:Benchmark took 365.97 sec 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout:Total Results 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout:=============== 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout: ======= ============ ========= ======= =========== ========== 2024-04-26T22:08:56.506 INFO:tasks.workunit.client.0.smithi135.stdout: write : 21331200 58287.43 100.000% 100.000% 228MB/sec 2024-04-26T22:08:56.507 INFO:tasks.workunit.client.0.smithi135.stdout:- 2024-04-26T22:08:56.507 INFO:tasks.workunit.client.0.smithi135.stdout:58287.43 Transactions per Second 2024-04-26T22:08:56.507 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.507 INFO:tasks.workunit.client.0.smithi135.stdout:Throughput Results 2024-04-26T22:08:56.507 INFO:tasks.workunit.client.0.smithi135.stdout:=================== 2024-04-26T22:08:56.507 INFO:tasks.workunit.client.0.smithi135.stdout:Write Throughput: 228MB/sec 2024-04-26T22:08:56.615 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.615 INFO:tasks.workunit.client.0.smithi135.stdout:System Call Latency statistics in millisecs 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout:===== 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout: Min Avg Max Total Calls 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout: ======== ======== ======== ============ 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout:[ open] 0.004000 58.088816 24310.640625 16665 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[0] 5000.000000 - 10000.000000 : 22 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[1] 2000.000000 - 5000.000000 : 67 2024-04-26T22:08:56.616 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[3] 500.000000 - 1000.000000 : 2 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[4] 200.000000 - 500.000000 : 13 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[5] 100.000000 - 200.000000 : 138 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[6] 50.000000 - 100.000000 : 126 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[7] 20.000000 - 50.000000 : 103 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[8] 10.000000 - 20.000000 : 39 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[9] 5.000000 - 10.000000 : 50 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[10] 2.000000 - 5.000000 : 52 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[11] 1.000000 - 2.000000 : 26 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[12] 0.500000 - 1.000000 : 9 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[13] 0.200000 - 0.500000 : 15 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[14] 0.100000 - 0.200000 : 18 2024-04-26T22:08:56.624 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[15] 0.050000 - 0.100000 : 58 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[16] 0.020000 - 0.050000 : 3691 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[17] 0.010000 - 0.020000 : 11547 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[18] 0.000000 - 0.010000 : 655 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout:[ write] 0.001000 0.404773 25075.246094 21331200 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[0] 5000.000000 - 10000.000000 : 155 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[1] 2000.000000 - 5000.000000 : 831 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[2] 1000.000000 - 2000.000000 : 654 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[3] 500.000000 - 1000.000000 : 607 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[4] 200.000000 - 500.000000 : 1047 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[5] 100.000000 - 200.000000 : 3712 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[6] 50.000000 - 100.000000 : 20616 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[7] 20.000000 - 50.000000 : 20083 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[8] 10.000000 - 20.000000 : 1270 2024-04-26T22:08:56.625 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[9] 5.000000 - 10.000000 : 1527 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[10] 2.000000 - 5.000000 : 2281 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[11] 1.000000 - 2.000000 : 1721 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[12] 0.500000 - 1.000000 : 1672 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[13] 0.200000 - 0.500000 : 2382 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[14] 0.100000 - 0.200000 : 2608 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[15] 0.050000 - 0.100000 : 5513 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[16] 0.020000 - 0.050000 : 27572 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[17] 0.010000 - 0.020000 : 90809 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[18] 0.000000 - 0.010000 : 21146099 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout:[ lseek] 0.000000 0.000605 118.833000 21331200 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-26T22:08:56.626 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[6] 50.000000 - 100.000000 : 3 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[7] 20.000000 - 50.000000 : 33 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[8] 10.000000 - 20.000000 : 53 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[9] 5.000000 - 10.000000 : 87 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[10] 2.000000 - 5.000000 : 175 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[11] 1.000000 - 2.000000 : 156 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[12] 0.500000 - 1.000000 : 169 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[13] 0.200000 - 0.500000 : 238 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[14] 0.100000 - 0.200000 : 276 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[15] 0.050000 - 0.100000 : 517 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[16] 0.020000 - 0.050000 : 2395 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[17] 0.010000 - 0.020000 : 5774 2024-04-26T22:08:56.627 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[18] 0.000000 - 0.010000 : 21321323 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout:[ close] 0.001000 0.010576 39.674000 16665 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[7] 20.000000 - 50.000000 : 2 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[11] 1.000000 - 2.000000 : 2 2024-04-26T22:08:56.628 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[12] 0.500000 - 1.000000 : 3 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[13] 0.200000 - 0.500000 : 4 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[14] 0.100000 - 0.200000 : 5 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[15] 0.050000 - 0.100000 : 9 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[16] 0.020000 - 0.050000 : 41 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[17] 0.010000 - 0.020000 : 197 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: msec_range[18] 0.000000 - 0.010000 : 16400 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout: 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout:3.0% User Time 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout:19.0% System Time 2024-04-26T22:08:56.629 INFO:tasks.workunit.client.0.smithi135.stdout:22.0% CPU Utilization 2024-04-26T22:08:56.639 INFO:tasks.workunit.client.0.smithi135.stderr:+ cd .. 2024-04-26T22:08:56.641 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:56.644 INFO:tasks.workunit.client.0.smithi135.stderr:+ rm -r tmp ffsb 2024-04-26T22:08:56.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.804+0000 7f14c2012640 1 Processor -- start 2024-04-26T22:08:56.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.804+0000 7f14c2012640 1 -- start start 2024-04-26T22:08:56.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14c2012640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c90c0 0x7f14bc15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:56.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14c2012640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14bc0c6e00 0x7f14bc158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:56.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14c2012640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14bc0cbde0 con 0x7f14bc0bfbb0 2024-04-26T22:08:56.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14c2012640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f14bc15c150 con 0x7f14bc0c6e00 2024-04-26T22:08:56.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14c2012640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc0ca380 con 0x7f14bc0c90c0 2024-04-26T22:08:56.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14bc0c6e00 0x7f14bc158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14bc0c6e00 0x7f14bc158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:32770/0 (socket says 172.21.15.135:32770) 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 -- 172.21.15.135:0/3700354888 learned_addr learned my addr 172.21.15.135:0/3700354888 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bb7fe640 1 --2- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c90c0 0x7f14bc15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 -- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c90c0 msgr2=0x7f14bc15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 --2- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c90c0 0x7f14bc15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 -- 172.21.15.135:0/3700354888 >> v1:172.21.15.139:6789/0 conn(0x7f14bc0bfbb0 legacy=0x7f14bc0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:08:56.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.806+0000 7f14bbfff640 1 -- 172.21.15.135:0/3700354888 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc0cc0c0 con 0x7f14bc0c6e00 2024-04-26T22:08:56.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.810+0000 7f14bbfff640 1 --2- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0c6e00 0x7f14bc158510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f14bc08e170 tx=0x7f14b402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679653e9de66707e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:56.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.810+0000 7f14ba7fc640 1 -- 172.21.15.135:0/3700354888 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4036070 con 0x7f14bc0c6e00 2024-04-26T22:08:56.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.810+0000 7f14ba7fc640 1 -- 172.21.15.135:0/3700354888 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4031600 con 0x7f14bc0c6e00 2024-04-26T22:08:56.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.810+0000 7f14ba7fc640 1 -- 172.21.15.135:0/3700354888 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4031a10 con 0x7f14bc0c6e00 2024-04-26T22:08:56.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.812+0000 7f14c2012640 1 -- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0c6e00 msgr2=0x7f14bc158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:56.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.812+0000 7f14c2012640 1 --2- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0c6e00 0x7f14bc158510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f14bc08e170 tx=0x7f14b402a5b0 comp rx=0 tx=0).stop 2024-04-26T22:08:56.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.813+0000 7f14c2012640 1 -- 172.21.15.135:0/3700354888 shutdown_connections 2024-04-26T22:08:56.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.813+0000 7f14c2012640 1 --2- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0c6e00 0x7f14bc158510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:56.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.813+0000 7f14c2012640 1 --2- 172.21.15.135:0/3700354888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c90c0 0x7f14bc15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:56.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.813+0000 7f14c2012640 1 -- 172.21.15.135:0/3700354888 >> 172.21.15.135:0/3700354888 conn(0x7f14bc0ae010 msgr2=0x7f14bc0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:56.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.813+0000 7f14c2012640 1 -- 172.21.15.135:0/3700354888 shutdown_connections 2024-04-26T22:08:56.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.813+0000 7f14c2012640 1 -- 172.21.15.135:0/3700354888 wait complete. 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.815+0000 7f14c2012640 1 Processor -- start 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.815+0000 7f14c2012640 1 -- start start 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.816+0000 7f14c2012640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0bfbb0 0x7f14bc1491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14c2012640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c6e00 0x7f14bc08d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14c2012640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 0x7f14bc0c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 0x7f14bc0c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 0x7f14bc0c0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57912/0 (socket says 172.21.15.135:57912) 2024-04-26T22:08:56.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14bbfff640 1 -- 172.21.15.135:0/774779478 learned_addr learned my addr 172.21.15.135:0/774779478 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc0c94a0 con 0x7f14bc0c90c0 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc0cb4e0 con 0x7f14bc0bfbb0 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc0c9620 con 0x7f14bc0c6e00 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14baffd640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c6e00 0x7f14bc08d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.817+0000 7f14bb7fe640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0bfbb0 0x7f14bc1491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.818+0000 7f14bbfff640 1 -- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c6e00 msgr2=0x7f14bc08d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.818+0000 7f14bbfff640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c6e00 0x7f14bc08d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.818+0000 7f14bbfff640 1 -- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0bfbb0 msgr2=0x7f14bc1491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.818+0000 7f14bbfff640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0bfbb0 0x7f14bc1491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.818+0000 7f14bbfff640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc0c2a60 con 0x7f14bc0c90c0 2024-04-26T22:08:56.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.818+0000 7f14bbfff640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 0x7f14bc0c0f20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009f40 tx=0x7f14b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:56.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.819+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4036050 con 0x7f14bc0c90c0 2024-04-26T22:08:56.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.820+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4042070 con 0x7f14bc0c90c0 2024-04-26T22:08:56.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.821+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b403d450 con 0x7f14bc0c90c0 2024-04-26T22:08:56.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.821+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc0c0a40 con 0x7f14bc0c90c0 2024-04-26T22:08:56.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.821+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc149af0 con 0x7f14bc0c90c0 2024-04-26T22:08:56.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.823+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc076650 con 0x7f14bc0c90c0 2024-04-26T22:08:56.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.835+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14b4031d10 con 0x7f14bc0c90c0 2024-04-26T22:08:56.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.836+0000 7f14a3fff640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a806c9f0 0x7f14a806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:56.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.836+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 7612+0+0 (secure 0 0 0) 0x7f14b40ad580 con 0x7f14bc0c90c0 2024-04-26T22:08:56.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.836+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14b404b020 con 0x7f14bc0c90c0 2024-04-26T22:08:56.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.840+0000 7f14bb7fe640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a806c9f0 0x7f14a806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:56.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:56.841+0000 7f14bb7fe640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a806c9f0 0x7f14a806eeb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009970 tx=0x7f14b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:57.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.334+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14bc0507a0 con 0x7f14bc0c90c0 2024-04-26T22:08:57.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.335+0000 7f14a3fff640 1 -- 172.21.15.135:0/774779478 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+6502 (secure 0 0 0) 0x7f14b407b3d0 con 0x7f14bc0c90c0 2024-04-26T22:08:57.335 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:08:57.335 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5970,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:07:42.906062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4928":{"gid":4928,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":68,"addr":"172.21.15.139:6833/3664037452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3664037452},{"type":"v1","addr":"172.21.15.139:6833","nonce":3664037452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4928,"qdb_cluster":[4928]},"id":2}]} 2024-04-26T22:08:57.335 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 66 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a806c9f0 msgr2=0x7f14a806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a806c9f0 0x7f14a806eeb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009970 tx=0x7f14b0008040 comp rx=0 tx=0).stop 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 msgr2=0x7f14bc0c0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 0x7f14bc0c0f20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009f40 tx=0x7f14b4004a40 comp rx=0 tx=0).stop 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 shutdown_connections 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a806c9f0 0x7f14a806eeb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:57.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc0c90c0 0x7f14bc0c0f20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:57.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14bc0c6e00 0x7f14bc08d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:57.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 --2- 172.21.15.135:0/774779478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14bc0bfbb0 0x7f14bc1491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:57.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 >> 172.21.15.135:0/774779478 conn(0x7f14bc0ae010 msgr2=0x7f14bc0cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:57.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 shutdown_connections 2024-04-26T22:08:57.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:57.344+0000 7f14c2012640 1 -- 172.21.15.135:0/774779478 wait complete. 2024-04-26T22:08:57.370 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-04-26T22:08:59.373 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:08:59.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.549+0000 7f613b17f640 1 Processor -- start 2024-04-26T22:08:59.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.549+0000 7f613b17f640 1 -- start start 2024-04-26T22:08:59.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.550+0000 7f613b17f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f613406dc60 0x7f613415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:59.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f613b17f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61340c4960 con 0x7f61340ac870 2024-04-26T22:08:59.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f613b17f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613415c040 con 0x7f6134158a00 2024-04-26T22:08:59.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f613b17f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f613415b7a0 con 0x7f613406dc60 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f6133fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f61340ac870 0x7f61341582f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47200/0 (socket says 172.21.15.135:47200) 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f6133fff640 1 -- 172.21.15.135:0/3352401141 learned_addr learned my addr 172.21.15.135:0/3352401141 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f6138f1b640 1 --2- 172.21.15.135:0/3352401141 >> v2:172.21.15.135:3300/0 conn(0x7f613406dc60 0x7f613415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f61337fe640 1 -- 172.21.15.135:0/3352401141 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75267673 0 0) 0x7f613415c040 con 0x7f6134158a00 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f61337fe640 1 -- 172.21.15.135:0/3352401141 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f611c003520 con 0x7f6134158a00 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f6138f1b640 1 -- 172.21.15.135:0/3352401141 >> v1:172.21.15.139:6789/0 conn(0x7f6134158a00 legacy=0x7f61340aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.551+0000 7f61337fe640 1 -- 172.21.15.135:0/3352401141 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396262549 0 0) 0x7f61340c4960 con 0x7f61340ac870 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.552+0000 7f6138f1b640 1 -- 172.21.15.135:0/3352401141 >> v1:172.21.15.135:6789/0 conn(0x7f61340ac870 legacy=0x7f61341582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:59.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.552+0000 7f6138f1b640 1 -- 172.21.15.135:0/3352401141 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613415ba80 con 0x7f613406dc60 2024-04-26T22:08:59.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.553+0000 7f6138f1b640 1 --2- 172.21.15.135:0/3352401141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613415b0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f612c009d90 tx=0x7f612c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261c6053897f3d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:59.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.553+0000 7f61337fe640 1 -- 172.21.15.135:0/3352401141 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c031600 con 0x7f613406dc60 2024-04-26T22:08:59.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.553+0000 7f61337fe640 1 -- 172.21.15.135:0/3352401141 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612c0317a0 con 0x7f613406dc60 2024-04-26T22:08:59.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.553+0000 7f61337fe640 1 -- 172.21.15.135:0/3352401141 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c031b50 con 0x7f613406dc60 2024-04-26T22:08:59.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.555+0000 7f613b17f640 1 -- 172.21.15.135:0/3352401141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 msgr2=0x7f613415b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:59.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.555+0000 7f613b17f640 1 --2- 172.21.15.135:0/3352401141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613415b0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f612c009d90 tx=0x7f612c02a5d0 comp rx=0 tx=0).stop 2024-04-26T22:08:59.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.555+0000 7f613b17f640 1 -- 172.21.15.135:0/3352401141 shutdown_connections 2024-04-26T22:08:59.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.555+0000 7f613b17f640 1 --2- 172.21.15.135:0/3352401141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613415b0a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:59.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.555+0000 7f613b17f640 1 -- 172.21.15.135:0/3352401141 >> 172.21.15.135:0/3352401141 conn(0x7f61340c6d80 msgr2=0x7f61340c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:59.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.556+0000 7f613b17f640 1 -- 172.21.15.135:0/3352401141 shutdown_connections 2024-04-26T22:08:59.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.556+0000 7f613b17f640 1 -- 172.21.15.135:0/3352401141 wait complete. 2024-04-26T22:08:59.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.558+0000 7f613b17f640 1 Processor -- start 2024-04-26T22:08:59.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.558+0000 7f613b17f640 1 -- start start 2024-04-26T22:08:59.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.559+0000 7f613b17f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613414dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.559+0000 7f613b17f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61340ac870 0x7f6134070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.559+0000 7f613b17f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6134158a00 0x7f61340ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.559+0000 7f613b17f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f613406e040 con 0x7f613406dc60 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.560+0000 7f6138f1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613414dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.560+0000 7f6138f1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613414dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36596/0 (socket says 172.21.15.135:36596) 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.560+0000 7f6138f1b640 1 -- 172.21.15.135:0/2014816801 learned_addr learned my addr 172.21.15.135:0/2014816801 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:08:59.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.560+0000 7f6133fff640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61340ac870 0x7f6134070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:59.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.561+0000 7f613971c640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6134158a00 0x7f61340ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:59.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.561+0000 7f613b17f640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f61340ab510 con 0x7f61340ac870 2024-04-26T22:08:59.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.561+0000 7f613b17f640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f613406e1c0 con 0x7f6134158a00 2024-04-26T22:08:59.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613971c640 1 -- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61340ac870 msgr2=0x7f6134070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:59.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613971c640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61340ac870 0x7f6134070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613971c640 1 -- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 msgr2=0x7f613414dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613971c640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613414dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613971c640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613414c3c0 con 0x7f6134158a00 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f6138f1b640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613971c640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6134158a00 0x7f61340ad800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f612800c3c0 tx=0x7f612800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128017070 con 0x7f6134158a00 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f613b17f640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613406ff40 con 0x7f6134158a00 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.562+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61280052d0 con 0x7f6134158a00 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.563+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280037f0 con 0x7f6134158a00 2024-04-26T22:08:59.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.563+0000 7f613b17f640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f613414d860 con 0x7f6134158a00 2024-04-26T22:08:59.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.565+0000 7f613b17f640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613406cf10 con 0x7f6134158a00 2024-04-26T22:08:59.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.565+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f612801a070 con 0x7f6134158a00 2024-04-26T22:08:59.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.565+0000 7f61317fa640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f612006c920 0x7f612006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:59.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.566+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f612808b3d0 con 0x7f6134158a00 2024-04-26T22:08:59.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.566+0000 7f6138f1b640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f612006c920 0x7f612006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:59.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.566+0000 7f6138f1b640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f612006c920 0x7f612006ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f612c009bd0 tx=0x7f612c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:59.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:08:59.574+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f613406cf10 con 0x7f6134158a00 2024-04-26T22:09:00.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.072+0000 7f613b17f640 1 -- 172.21.15.135:0/2014816801 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6134076650 con 0x7f6134158a00 2024-04-26T22:09:00.073 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.073+0000 7f61317fa640 1 -- 172.21.15.135:0/2014816801 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5666 (secure 0 0 0) 0x7f61280589f0 con 0x7f6134158a00 2024-04-26T22:09:00.074 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:00.075 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:08:57.853978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:09:00.075 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 67 2024-04-26T22:09:00.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.079+0000 7f611affd640 1 -- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f612006c920 msgr2=0x7f612006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.079+0000 7f611affd640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f612006c920 0x7f612006ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f612c009bd0 tx=0x7f612c02f000 comp rx=0 tx=0).stop 2024-04-26T22:09:00.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.079+0000 7f611affd640 1 -- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6134158a00 msgr2=0x7f61340ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.080+0000 7f611affd640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6134158a00 0x7f61340ad800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f612800c3c0 tx=0x7f612800c880 comp rx=0 tx=0).stop 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 -- 172.21.15.135:0/2014816801 shutdown_connections 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f612006c920 0x7f612006ede0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6134158a00 0x7f61340ad800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61340ac870 0x7f6134070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 --2- 172.21.15.135:0/2014816801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613406dc60 0x7f613414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 -- 172.21.15.135:0/2014816801 >> 172.21.15.135:0/2014816801 conn(0x7f61340c6d80 msgr2=0x7f61340c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 -- 172.21.15.135:0/2014816801 shutdown_connections 2024-04-26T22:09:00.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.089+0000 7f611affd640 1 -- 172.21.15.135:0/2014816801 wait complete. 2024-04-26T22:09:00.110 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-04-26T22:09:00.110 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:09:00.111 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:00.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.280+0000 7f2dc7e50640 1 Processor -- start 2024-04-26T22:09:00.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.280+0000 7f2dc7e50640 1 -- start start 2024-04-26T22:09:00.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.281+0000 7f2dc7e50640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2db80315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.281+0000 7f2dc7e50640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db8031b20 con 0x7f2db80cd2b0 2024-04-26T22:09:00.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.281+0000 7f2dc7e50640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db80cd690 con 0x7f2db803f740 2024-04-26T22:09:00.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.281+0000 7f2dc7e50640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2db800b800 con 0x7f2db8049f40 2024-04-26T22:09:00.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.282+0000 7f2dc5bec640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2db80cd2b0 0x7f2db80469c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47202/0 (socket says 172.21.15.135:47202) 2024-04-26T22:09:00.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.282+0000 7f2dc5bec640 1 -- 172.21.15.135:0/1899800350 learned_addr learned my addr 172.21.15.135:0/1899800350 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.283+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1737922732 0 0) 0x7f2db80cd690 con 0x7f2db803f740 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.283+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dac003520 con 0x7f2db803f740 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.283+0000 7f2dc63ed640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2db80315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.283+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89746299 0 0) 0x7f2db8031b20 con 0x7f2db80cd2b0 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.283+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dac003880 con 0x7f2db80cd2b0 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2880161915 0 0) 0x7f2dac003520 con 0x7f2db803f740 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 msgr2=0x7f2db80315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:00.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2db80315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 >> v1:172.21.15.135:6789/0 conn(0x7f2db80cd2b0 legacy=0x7f2db80469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db80d7720 con 0x7f2db803f740 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f2db0001650 con 0x7f2db803f740 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 >> v1:172.21.15.139:6789/0 conn(0x7f2db803f740 legacy=0x7f2db8030ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2dac003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dac0042a0 0x7f2dac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dac004e60 0x7f2dac0079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2db800b800 con 0x7f2dac004e60 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac005680 con 0x7f2dac0042a0 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac007f60 con 0x7f2db8049f40 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc63ed640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dac004e60 0x7f2dac0079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc5bec640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2dac003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.284+0000 7f2dc53eb640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dac0042a0 0x7f2dac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.285+0000 7f2dc63ed640 1 -- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 msgr2=0x7f2dac003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:00.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.285+0000 7f2dc63ed640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2dac003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.285+0000 7f2dc63ed640 1 -- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dac0042a0 msgr2=0x7f2dac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:00.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.285+0000 7f2dc63ed640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dac0042a0 0x7f2dac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.285+0000 7f2dc63ed640 1 -- 172.21.15.135:0/1899800350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac003520 con 0x7f2dac004e60 2024-04-26T22:09:00.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.286+0000 7f2dc53eb640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dac0042a0 0x7f2dac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:00.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.286+0000 7f2dc5bec640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db8049f40 0x7f2dac003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:00.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.286+0000 7f2dc63ed640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dac004e60 0x7f2dac0079a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0075190 tx=0x7f2dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc89c64bc04fa8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:00.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.288+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00b5070 con 0x7f2dac004e60 2024-04-26T22:09:00.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.288+0000 7f2dc4bea640 1 -- 172.21.15.135:0/1899800350 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc00b0410 con 0x7f2dac004e60 2024-04-26T22:09:00.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.289+0000 7f2dc7e50640 1 -- 172.21.15.135:0/1899800350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dac004e60 msgr2=0x7f2dac0079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.289+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/1899800350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dac004e60 0x7f2dac0079a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0075190 tx=0x7f2dac005830 comp rx=0 tx=0).stop 2024-04-26T22:09:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.289+0000 7f2dc63ed640 1 -- 172.21.15.135:0/1899800350 reap_dead start 2024-04-26T22:09:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.289+0000 7f2dc7e50640 1 -- 172.21.15.135:0/1899800350 shutdown_connections 2024-04-26T22:09:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.289+0000 7f2dc7e50640 1 -- 172.21.15.135:0/1899800350 >> 172.21.15.135:0/1899800350 conn(0x7f2db8048ac0 msgr2=0x7f2db8048ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.290+0000 7f2dc7e50640 1 -- 172.21.15.135:0/1899800350 shutdown_connections 2024-04-26T22:09:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.290+0000 7f2dc7e50640 1 -- 172.21.15.135:0/1899800350 wait complete. 2024-04-26T22:09:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.292+0000 7f2dc7e50640 1 Processor -- start 2024-04-26T22:09:00.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.292+0000 7f2dc7e50640 1 -- start start 2024-04-26T22:09:00.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.293+0000 7f2dc7e50640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 0x7f2db816c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.293+0000 7f2dc7e50640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db816f690 0x7f2db802ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.293+0000 7f2dc7e50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2db802f440 0x7f2db80d3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.293+0000 7f2dc7e50640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2db800b490 con 0x7f2db8006700 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.293+0000 7f2dc7e50640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2db800b1b0 con 0x7f2db802f440 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.293+0000 7f2dc7e50640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2db800c220 con 0x7f2db816f690 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.294+0000 7f2dc5bec640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 0x7f2db816c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.294+0000 7f2dc5bec640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 0x7f2db816c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36614/0 (socket says 172.21.15.135:36614) 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.294+0000 7f2dc5bec640 1 -- 172.21.15.135:0/82973841 learned_addr learned my addr 172.21.15.135:0/82973841 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.294+0000 7f2dc53eb640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db816f690 0x7f2db802ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.295+0000 7f2dc5bec640 1 -- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db816f690 msgr2=0x7f2db802ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.295+0000 7f2dc5bec640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db816f690 0x7f2db802ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.295+0000 7f2dc5bec640 1 -- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2db802f440 msgr2=0x7f2db80d3910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:09:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.295+0000 7f2dc5bec640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2db802f440 0x7f2db80d3910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.295+0000 7f2dc5bec640 1 -- 172.21.15.135:0/82973841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db816b710 con 0x7f2db8006700 2024-04-26T22:09:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.295+0000 7f2dc5bec640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 0x7f2db816c1d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc007360 tx=0x7f2dbc007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.296+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc00f960 con 0x7f2db8006700 2024-04-26T22:09:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.296+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db8041ea0 con 0x7f2db8006700 2024-04-26T22:09:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.296+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db816b9d0 con 0x7f2db8006700 2024-04-26T22:09:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.297+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dbc018070 con 0x7f2db8006700 2024-04-26T22:09:00.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.300+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc00fd00 con 0x7f2db8006700 2024-04-26T22:09:00.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.301+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dbc01d460 con 0x7f2db8006700 2024-04-26T22:09:00.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.301+0000 7f2db67fc640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2da406ca20 0x7f2da406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:00.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.302+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da8005290 con 0x7f2db8006700 2024-04-26T22:09:00.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.302+0000 7f2dc53eb640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2da406ca20 0x7f2da406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:00.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.302+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f2dbc08c350 con 0x7f2db8006700 2024-04-26T22:09:00.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.311+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2dbc05a020 con 0x7f2db8006700 2024-04-26T22:09:00.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.303+0000 7f2dc53eb640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2da406ca20 0x7f2da406eee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2db0000c00 tx=0x7f2db0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:00.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.824+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2da8005580 con 0x7f2db8006700 2024-04-26T22:09:00.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.825+0000 7f2db67fc640 1 -- 172.21.15.135:0/82973841 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5666 (secure 0 0 0) 0x7f2dbc059920 con 0x7f2db8006700 2024-04-26T22:09:00.828 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:00.828 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:08:57.853978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:09:00.828 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 67 2024-04-26T22:09:00.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.831+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2da406ca20 msgr2=0x7f2da406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.831+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2da406ca20 0x7f2da406eee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2db0000c00 tx=0x7f2db0009040 comp rx=0 tx=0).stop 2024-04-26T22:09:00.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.831+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 msgr2=0x7f2db816c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:00.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.831+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 0x7f2db816c1d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc007360 tx=0x7f2dbc007760 comp rx=0 tx=0).stop 2024-04-26T22:09:00.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 shutdown_connections 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2da406ca20 0x7f2da406eee0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2db802f440 0x7f2db80d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2db816f690 0x7f2db802ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 --2- 172.21.15.135:0/82973841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2db8006700 0x7f2db816c1d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 >> 172.21.15.135:0/82973841 conn(0x7f2db8048ac0 msgr2=0x7f2db80d9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.832+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 shutdown_connections 2024-04-26T22:09:00.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:00.833+0000 7f2dc7e50640 1 -- 172.21.15.135:0/82973841 wait complete. 2024-04-26T22:09:00.852 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 67, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 63, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5210": { "addr": "172.21.15.135:6837/3969385858", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6836", "nonce": 3969385858, "type": "v2" }, { "addr": "172.21.15.135:6837", "nonce": 3969385858, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5210, "incarnation": 59, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:07:41.896787+0000", "qdb_cluster": [ 5210 ], "qdb_leader": 5210, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5210 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 67, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5970": { "addr": "172.21.15.135:6835/3800113814", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 3800113814, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 3800113814, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5970, "incarnation": 67, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:08:57.853978+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5970 } } } ], "standbys": [ { "addr": "172.21.15.139:6835/1717329133", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 1717329133, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 1717329133, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 65, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5291, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:09:00.949 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T22:09:00.949 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T22:09:01.056 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-04-26T22:09:01.056 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T22:09:01.058 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:09:01.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:09:01.581 DEBUG:teuthology.parallel:result is None 2024-04-26T22:09:02.854 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:03.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.026+0000 7f1ce4946640 1 Processor -- start 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.026+0000 7f1ce4946640 1 -- start start 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.027+0000 7f1ce4946640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1ce01510c0 0x7f1ce0072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.027+0000 7f1ce4946640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce00734d0 0x7f1ce006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.027+0000 7f1ce4946640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce00c4b50 con 0x7f1ce00c4160 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.027+0000 7f1ce4946640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce00c50d0 con 0x7f1ce01510c0 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.027+0000 7f1ce4946640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce01514a0 con 0x7f1ce00734d0 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.028+0000 7f1cded9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1ce00c4160 0x7f1ce0157070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:36154/0 (socket says 172.21.15.135:36154) 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.028+0000 7f1cded9d640 1 -- 172.21.15.135:0/1316810879 learned_addr learned my addr 172.21.15.135:0/1316810879 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.028+0000 7f1cd7fff640 1 --2- 172.21.15.135:0/1316810879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce00734d0 0x7f1ce006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.029+0000 7f1cd7fff640 1 -- 172.21.15.135:0/1316810879 >> v2:172.21.15.139:3300/0 conn(0x7f1ce01510c0 msgr2=0x7f1ce0072f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.029+0000 7f1cd7fff640 1 --2- 172.21.15.135:0/1316810879 >> v2:172.21.15.139:3300/0 conn(0x7f1ce01510c0 0x7f1ce0072f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.029+0000 7f1cd7fff640 1 -- 172.21.15.135:0/1316810879 >> v1:172.21.15.139:6789/0 conn(0x7f1ce00c4160 legacy=0x7f1ce0157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:03.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.029+0000 7f1cd7fff640 1 -- 172.21.15.135:0/1316810879 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce0154920 con 0x7f1ce00734d0 2024-04-26T22:09:03.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.030+0000 7f1cd7fff640 1 --2- 172.21.15.135:0/1316810879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce00734d0 0x7f1ce006d320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00c5250 tx=0x7f1cc802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fdf33646dd3b815 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:03.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.031+0000 7f1cddd9b640 1 -- 172.21.15.135:0/1316810879 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8037070 con 0x7f1ce00734d0 2024-04-26T22:09:03.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.031+0000 7f1cddd9b640 1 -- 172.21.15.135:0/1316810879 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc802add0 con 0x7f1ce00734d0 2024-04-26T22:09:03.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.031+0000 7f1cddd9b640 1 -- 172.21.15.135:0/1316810879 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8032660 con 0x7f1ce00734d0 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 -- 172.21.15.135:0/1316810879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce00734d0 msgr2=0x7f1ce006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1316810879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce00734d0 0x7f1ce006d320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00c5250 tx=0x7f1cc802a080 comp rx=0 tx=0).stop 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 -- 172.21.15.135:0/1316810879 shutdown_connections 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1316810879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce00734d0 0x7f1ce006d320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1316810879 >> v2:172.21.15.139:3300/0 conn(0x7f1ce01510c0 0x7f1ce0072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 -- 172.21.15.135:0/1316810879 >> 172.21.15.135:0/1316810879 conn(0x7f1ce00c5ce0 msgr2=0x7f1ce00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.032+0000 7f1ce4946640 1 -- 172.21.15.135:0/1316810879 shutdown_connections 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.033+0000 7f1ce4946640 1 -- 172.21.15.135:0/1316810879 wait complete. 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.034+0000 7f1ce4946640 1 Processor -- start 2024-04-26T22:09:03.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.034+0000 7f1ce4946640 1 -- start start 2024-04-26T22:09:03.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.035+0000 7f1ce4946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 0x7f1ce00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:03.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.035+0000 7f1ce4946640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ce01510c0 0x7f1ce00c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:03.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.035+0000 7f1ce4946640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce01560b0 0x7f1ce0156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.035+0000 7f1ce4946640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00c5920 con 0x7f1ce00c4160 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.035+0000 7f1ce4946640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00c5400 con 0x7f1ce01510c0 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.035+0000 7f1ce4946640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce006f500 con 0x7f1ce01560b0 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 0x7f1ce00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 0x7f1ce00ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36622/0 (socket says 172.21.15.135:36622) 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 -- 172.21.15.135:0/1971916989 learned_addr learned my addr 172.21.15.135:0/1971916989 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cd7fff640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ce01510c0 0x7f1ce00c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cded9d640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce01560b0 0x7f1ce0156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 -- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce01560b0 msgr2=0x7f1ce0156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce01560b0 0x7f1ce0156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 -- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ce01510c0 msgr2=0x7f1ce00c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ce01510c0 0x7f1ce00c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.036+0000 7f1cde59c640 1 -- 172.21.15.135:0/1971916989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce00c8c20 con 0x7f1ce00c4160 2024-04-26T22:09:03.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.037+0000 7f1cde59c640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 0x7f1ce00ac8c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000b3e0 tx=0x7f1cd000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:03.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.037+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0015020 con 0x7f1ce00c4160 2024-04-26T22:09:03.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.037+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd0011850 con 0x7f1ce00c4160 2024-04-26T22:09:03.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.038+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0011c40 con 0x7f1ce00c4160 2024-04-26T22:09:03.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.041+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce00acea0 con 0x7f1ce00c4160 2024-04-26T22:09:03.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.042+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce00aeb90 con 0x7f1ce00c4160 2024-04-26T22:09:03.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.044+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce00507a0 con 0x7f1ce00c4160 2024-04-26T22:09:03.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.045+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cd0018020 con 0x7f1ce00c4160 2024-04-26T22:09:03.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.046+0000 7f1cd6ffd640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1cc006c9c0 0x7f1cc006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:03.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.046+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f1cd008c240 con 0x7f1ce00c4160 2024-04-26T22:09:03.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.047+0000 7f1cd7fff640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1cc006c9c0 0x7f1cc006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:03.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.048+0000 7f1cd7fff640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1cc006c9c0 0x7f1cc006ee80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:03.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.053+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cd001f020 con 0x7f1ce00c4160 2024-04-26T22:09:03.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.448+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ce00c3ed0 con 0x7f1ce00c4160 2024-04-26T22:09:03.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.456+0000 7f1cd6ffd640 1 -- 172.21.15.135:0/1971916989 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5666 (secure 0 0 0) 0x7f1cd0018310 con 0x7f1ce00c4160 2024-04-26T22:09:03.458 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:03.458 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:08:57.853978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:09:03.459 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 67 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.463+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1cc006c9c0 msgr2=0x7f1cc006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.463+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1cc006c9c0 0x7f1cc006ee80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc802e000 comp rx=0 tx=0).stop 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.463+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 msgr2=0x7f1ce00ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.463+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 0x7f1ce00ac8c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000b3e0 tx=0x7f1cd000b8a0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 shutdown_connections 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1cc006c9c0 0x7f1cc006ee80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ce01560b0 0x7f1ce0156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ce01510c0 0x7f1ce00c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 --2- 172.21.15.135:0/1971916989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ce00c4160 0x7f1ce00ac8c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 >> 172.21.15.135:0/1971916989 conn(0x7f1ce00c5ce0 msgr2=0x7f1ce014b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:03.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 shutdown_connections 2024-04-26T22:09:03.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:03.464+0000 7f1ce4946640 1 -- 172.21.15.135:0/1971916989 wait complete. 2024-04-26T22:09:03.485 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:09:05.486 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:05.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.613+0000 7fdf19f11640 1 Processor -- start 2024-04-26T22:09:05.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.613+0000 7fdf19f11640 1 -- start start 2024-04-26T22:09:05.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf19f11640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf14156bb0 0x7fdf1415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:05.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf19f11640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf140c8df0 con 0x7fdf14154160 2024-04-26T22:09:05.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf19f11640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf14154540 con 0x7fdf140ca5a0 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf19f11640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1415a6e0 con 0x7fdf14156bb0 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf137fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fdf14154160 0x7fdf140c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47206/0 (socket says 172.21.15.135:47206) 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf137fe640 1 -- 172.21.15.135:0/4051262367 learned_addr learned my addr 172.21.15.135:0/4051262367 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf13fff640 1 --2- 172.21.15.135:0/4051262367 >> v2:172.21.15.139:3300/0 conn(0x7fdf14156bb0 0x7fdf1415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf127fc640 1 -- 172.21.15.135:0/4051262367 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366506006 0 0) 0x7fdf04001110 con 0x7fdf14154160 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.614+0000 7fdf127fc640 1 -- 172.21.15.135:0/4051262367 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdefc003520 con 0x7fdf14154160 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.615+0000 7fdf13fff640 1 -- 172.21.15.135:0/4051262367 >> v1:172.21.15.139:6789/0 conn(0x7fdf140ca5a0 legacy=0x7fdf140c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.615+0000 7fdf13fff640 1 -- 172.21.15.135:0/4051262367 >> v1:172.21.15.135:6789/0 conn(0x7fdf14154160 legacy=0x7fdf140c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:05.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.615+0000 7fdf13fff640 1 -- 172.21.15.135:0/4051262367 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1415a910 con 0x7fdf14156bb0 2024-04-26T22:09:05.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.615+0000 7fdf13fff640 1 --2- 172.21.15.135:0/4051262367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14156bb0 0x7fdf1415a1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009d90 tx=0x7fdf0c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f738854963aeae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:05.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.616+0000 7fdf127fc640 1 -- 172.21.15.135:0/4051262367 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c007740 con 0x7fdf14156bb0 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.616+0000 7fdf127fc640 1 -- 172.21.15.135:0/4051262367 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c0078e0 con 0x7fdf14156bb0 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.616+0000 7fdf127fc640 1 -- 172.21.15.135:0/4051262367 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c007cb0 con 0x7fdf14156bb0 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 -- 172.21.15.135:0/4051262367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14156bb0 msgr2=0x7fdf1415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 --2- 172.21.15.135:0/4051262367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14156bb0 0x7fdf1415a1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009d90 tx=0x7fdf0c02a6f0 comp rx=0 tx=0).stop 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 -- 172.21.15.135:0/4051262367 shutdown_connections 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 --2- 172.21.15.135:0/4051262367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14156bb0 0x7fdf1415a1a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 -- 172.21.15.135:0/4051262367 >> 172.21.15.135:0/4051262367 conn(0x7fdf140ae010 msgr2=0x7fdf140ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 -- 172.21.15.135:0/4051262367 shutdown_connections 2024-04-26T22:09:05.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.617+0000 7fdf19f11640 1 -- 172.21.15.135:0/4051262367 wait complete. 2024-04-26T22:09:05.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.618+0000 7fdf19f11640 1 Processor -- start 2024-04-26T22:09:05.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 -- start start 2024-04-26T22:09:05.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140ca5a0 0x7fdf1414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:05.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 0x7fdf14148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:05.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf14156bb0 0x7fdf141492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1415aed0 con 0x7fdf14156bb0 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf140c9100 con 0x7fdf14154160 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.619+0000 7fdf19f11640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf140c7510 con 0x7fdf140ca5a0 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 0x7fdf14148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 0x7fdf14148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:53984/0 (socket says 172.21.15.135:53984) 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 -- 172.21.15.135:0/2247092705 learned_addr learned my addr 172.21.15.135:0/2247092705 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf13fff640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf14156bb0 0x7fdf141492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 -- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140ca5a0 msgr2=0x7fdf1414b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140ca5a0 0x7fdf1414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 -- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf14156bb0 msgr2=0x7fdf141492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf14156bb0 0x7fdf141492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf12ffd640 1 -- 172.21.15.135:0/2247092705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf14158120 con 0x7fdf14154160 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.620+0000 7fdf13fff640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf14156bb0 0x7fdf141492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:05.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.621+0000 7fdf12ffd640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 0x7fdf14148d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c4d0 tx=0x7fdf0800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:05.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.621+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080052d0 con 0x7fdf14154160 2024-04-26T22:09:05.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.621+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08005470 con 0x7fdf14154160 2024-04-26T22:09:05.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.621+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08012850 con 0x7fdf14154160 2024-04-26T22:09:05.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.621+0000 7fdf19f11640 1 -- 172.21.15.135:0/2247092705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf141583e0 con 0x7fdf14154160 2024-04-26T22:09:05.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.622+0000 7fdf19f11640 1 -- 172.21.15.135:0/2247092705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1414bee0 con 0x7fdf14154160 2024-04-26T22:09:05.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.623+0000 7fdf19f11640 1 -- 172.21.15.135:0/2247092705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf14076650 con 0x7fdf14154160 2024-04-26T22:09:05.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.623+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf080129f0 con 0x7fdf14154160 2024-04-26T22:09:05.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.624+0000 7fdefbfff640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c920 0x7fdf0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:05.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.624+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7fdf0808c080 con 0x7fdf14154160 2024-04-26T22:09:05.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.624+0000 7fdf137fe640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c920 0x7fdf0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:05.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.625+0000 7fdf137fe640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c920 0x7fdf0006ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf04007fd0 tx=0x7fdf04001150 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:05.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:05.629+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf08059d50 con 0x7fdf14154160 2024-04-26T22:09:06.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.117+0000 7fdf19f11640 1 -- 172.21.15.135:0/2247092705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf14156200 con 0x7fdf14154160 2024-04-26T22:09:06.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.119+0000 7fdefbfff640 1 -- 172.21.15.135:0/2247092705 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5674 (secure 0 0 0) 0x7fdf08059650 con 0x7fdf14154160 2024-04-26T22:09:06.120 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:06.120 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:06.120 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 70 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 -- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c920 msgr2=0x7fdf0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c920 0x7fdf0006ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf04007fd0 tx=0x7fdf04001150 comp rx=0 tx=0).stop 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 -- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 msgr2=0x7fdf14148d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 0x7fdf14148d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c4d0 tx=0x7fdf0800c990 comp rx=0 tx=0).stop 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 -- 172.21.15.135:0/2247092705 shutdown_connections 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdf0006c920 0x7fdf0006ede0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf14156bb0 0x7fdf141492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf14154160 0x7fdf14148d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:06.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 --2- 172.21.15.135:0/2247092705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140ca5a0 0x7fdf1414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:06.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 -- 172.21.15.135:0/2247092705 >> 172.21.15.135:0/2247092705 conn(0x7fdf140ae010 msgr2=0x7fdf140ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:06.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.125+0000 7fdef9ffb640 1 -- 172.21.15.135:0/2247092705 shutdown_connections 2024-04-26T22:09:06.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:06.126+0000 7fdef9ffb640 1 -- 172.21.15.135:0/2247092705 wait complete. 2024-04-26T22:09:06.144 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=4928 2024-04-26T22:09:06.144 INFO:tasks.mds_thrash.fs.[b]:waiting for 9 secs before reviving mds.d 2024-04-26T22:09:07.989 INFO:tasks.ceph.mds.c.smithi135.stderr:2024-04-26T22:09:07.990+0000 7f317bc5a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:09:15.145 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-04-26T22:09:15.145 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T22:09:15.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T22:09:15.150 INFO:tasks.ceph.mds.d:Started 2024-04-26T22:09:15.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:15.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.321+0000 7f5f7634d640 1 Processor -- start 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.321+0000 7f5f7634d640 1 -- start start 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.322+0000 7f5f7634d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f7014d540 0x7f5f700cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.322+0000 7f5f7634d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f700ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.322+0000 7f5f7634d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7014d920 con 0x7f5f700cbb70 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.322+0000 7f5f7634d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5f701577f0 con 0x7f5f7014d540 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.322+0000 7f5f7634d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f70156be0 con 0x7f5f700ca300 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.323+0000 7f5f74b4a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5f700cbb70 0x7f5f70073940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57546/0 (socket says 172.21.15.135:57546) 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.323+0000 7f5f74b4a640 1 -- 172.21.15.135:0/2200554535 learned_addr learned my addr 172.21.15.135:0/2200554535 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:15.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.323+0000 7f5f7534b640 1 --2- 172.21.15.135:0/2200554535 >> v2:172.21.15.139:3300/0 conn(0x7f5f7014d540 0x7f5f700cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:15.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.326+0000 7f5f7534b640 1 -- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 msgr2=0x7f5f700ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:15.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.326+0000 7f5f67fff640 1 -- 172.21.15.135:0/2200554535 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647487067 0 0) 0x7f5f7014d920 con 0x7f5f700cbb70 2024-04-26T22:09:15.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.324+0000 7f5f75b4c640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f700ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:15.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.327+0000 7f5f7534b640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f700ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.327+0000 7f5f7534b640 1 -- 172.21.15.135:0/2200554535 >> v1:172.21.15.139:6789/0 conn(0x7f5f700cbb70 legacy=0x7f5f70073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:15.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.327+0000 7f5f7534b640 1 -- 172.21.15.135:0/2200554535 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f70156ec0 con 0x7f5f7014d540 2024-04-26T22:09:15.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.327+0000 7f5f75b4c640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f700ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:09:15.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.328+0000 7f5f7534b640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f700cb630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009dc0 tx=0x7f5f6802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4313c39c00d0aa7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:15.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.329+0000 7f5f67fff640 1 -- 172.21.15.135:0/2200554535 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6802ae60 con 0x7f5f7014d540 2024-04-26T22:09:15.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.329+0000 7f5f67fff640 1 -- 172.21.15.135:0/2200554535 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68007740 con 0x7f5f7014d540 2024-04-26T22:09:15.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.329+0000 7f5f67fff640 1 -- 172.21.15.135:0/2200554535 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68007b10 con 0x7f5f7014d540 2024-04-26T22:09:15.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.330+0000 7f5f7634d640 1 -- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 msgr2=0x7f5f700cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:15.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.330+0000 7f5f7634d640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f700cb630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009dc0 tx=0x7f5f6802a680 comp rx=0 tx=0).stop 2024-04-26T22:09:15.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.332+0000 7f5f7634d640 1 -- 172.21.15.135:0/2200554535 shutdown_connections 2024-04-26T22:09:15.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.332+0000 7f5f7634d640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f700ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.332+0000 7f5f7634d640 1 --2- 172.21.15.135:0/2200554535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f700cb630 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.332+0000 7f5f7634d640 1 -- 172.21.15.135:0/2200554535 >> 172.21.15.135:0/2200554535 conn(0x7f5f700c8da0 msgr2=0x7f5f700c91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.332+0000 7f5f7634d640 1 -- 172.21.15.135:0/2200554535 shutdown_connections 2024-04-26T22:09:15.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.333+0000 7f5f7634d640 1 -- 172.21.15.135:0/2200554535 wait complete. 2024-04-26T22:09:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.334+0000 7f5f7634d640 1 Processor -- start 2024-04-26T22:09:15.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.334+0000 7f5f7634d640 1 -- start start 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.335+0000 7f5f7634d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f701f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.335+0000 7f5f7634d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f7006ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f7634d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 0x7f5f701f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 0x7f5f701f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 0x7f5f701f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39552/0 (socket says 172.21.15.135:39552) 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f74b4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f7006ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 -- 172.21.15.135:0/4064723957 learned_addr learned my addr 172.21.15.135:0/4064723957 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70074050 con 0x7f5f7006f750 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70157480 con 0x7f5f7014d540 2024-04-26T22:09:15.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f700741d0 con 0x7f5f700ca300 2024-04-26T22:09:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 -- 172.21.15.135:0/4064723957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 msgr2=0x7f5f701f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f701f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 -- 172.21.15.135:0/4064723957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 msgr2=0x7f5f7006ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:15.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f7006ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.336+0000 7f5f75b4c640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f70073170 con 0x7f5f7006f750 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.337+0000 7f5f75b4c640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 0x7f5f701f2840 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c8e0 tx=0x7f5f6000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.337+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60015070 con 0x7f5f7006f750 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.337+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f70073450 con 0x7f5f7006f750 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.337+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f60007900 con 0x7f5f7006f750 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.337+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60007cf0 con 0x7f5f7006f750 2024-04-26T22:09:15.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.339+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f70070130 con 0x7f5f7006f750 2024-04-26T22:09:15.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.341+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f60018070 con 0x7f5f7006f750 2024-04-26T22:09:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.341+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f700506c0 con 0x7f5f7006f750 2024-04-26T22:09:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.342+0000 7f5f65ffb640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f5406c8f0 0x7f5f5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:15.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.342+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f5f60093350 con 0x7f5f7006f750 2024-04-26T22:09:15.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.343+0000 7f5f7534b640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f5406c8f0 0x7f5f5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:15.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.344+0000 7f5f7534b640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f5406c8f0 0x7f5f5406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f68034000 tx=0x7f5f6802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:15.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.349+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f600594c0 con 0x7f5f7006f750 2024-04-26T22:09:15.665 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-04-26T22:09:15.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.743+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f70073930 con 0x7f5f7006f750 2024-04-26T22:09:15.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.745+0000 7f5f65ffb640 1 -- 172.21.15.135:0/4064723957 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5674 (secure 0 0 0) 0x7f5f60058dc0 con 0x7f5f7006f750 2024-04-26T22:09:15.748 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:15.749 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:15.749 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 70 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f5406c8f0 msgr2=0x7f5f5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f5406c8f0 0x7f5f5406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f68034000 tx=0x7f5f6802e000 comp rx=0 tx=0).stop 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 msgr2=0x7f5f701f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 0x7f5f701f2840 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c8e0 tx=0x7f5f6000cda0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 shutdown_connections 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5f5406c8f0 0x7f5f5406edb0 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f68034000 tx=0x7f5f6802e000 comp rx=0 tx=0).stop 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f7006f750 0x7f5f701f2840 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f7014d540 0x7f5f7006ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 --2- 172.21.15.135:0/4064723957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f700ca300 0x7f5f701f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.751+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 >> 172.21.15.135:0/4064723957 conn(0x7f5f700c8da0 msgr2=0x7f5f700ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:15.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.752+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 shutdown_connections 2024-04-26T22:09:15.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:15.752+0000 7f5f7634d640 1 -- 172.21.15.135:0/4064723957 wait complete. 2024-04-26T22:09:15.773 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-04-26T22:09:17.774 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:17.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.965+0000 7f29dbdf0640 1 Processor -- start 2024-04-26T22:09:17.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.969+0000 7f29dbdf0640 1 -- start start 2024-04-26T22:09:17.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29dbdf0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc0dae30 0x7f29cc00ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29dbdf0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29cc007240 0x7f29cc0076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29dbdf0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29cc00f160 con 0x7f29cc035320 2024-04-26T22:09:17.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29dbdf0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f29cc00e790 con 0x7f29cc007240 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29dbdf0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc04a950 con 0x7f29cc0dae30 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29d9b8c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f29cc035320 0x7f29cc04daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:57552/0 (socket says 172.21.15.135:57552) 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29d9b8c640 1 -- 172.21.15.135:0/807703733 learned_addr learned my addr 172.21.15.135:0/807703733 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.970+0000 7f29d938b640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc0dae30 0x7f29cc00ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767823686 0 0) 0x7f29cc00f160 con 0x7f29cc035320 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29c0003520 con 0x7f29cc035320 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29da38d640 1 --2- 172.21.15.135:0/807703733 >> v2:172.21.15.135:3300/0 conn(0x7f29cc007240 0x7f29cc0076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3968341285 0 0) 0x7f29c0003520 con 0x7f29cc035320 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc0dae30 msgr2=0x7f29cc00ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc0dae30 0x7f29cc00ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 >> v2:172.21.15.135:3300/0 conn(0x7f29cc007240 msgr2=0x7f29cc0076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 --2- 172.21.15.135:0/807703733 >> v2:172.21.15.135:3300/0 conn(0x7f29cc007240 0x7f29cc0076d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc04ac30 con 0x7f29cc035320 2024-04-26T22:09:17.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.971+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f29d4050be0 con 0x7f29cc035320 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 >> v1:172.21.15.139:6789/0 conn(0x7f29cc035320 legacy=0x7f29cc04daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29c00039b0 0x7f29c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c0004350 0x7f29c0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29c0005070 0x7f29c002d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc00e790 con 0x7f29c0005070 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc04a950 con 0x7f29c0004350 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29c002e010 con 0x7f29c00039b0 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29da38d640 1 --2- 172.21.15.135:0/807703733 >> v2:172.21.15.135:3300/0 conn(0x7f29cc007240 0x7f29cc0076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d9b8c640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29c00039b0 0x7f29c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29da38d640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29c0005070 0x7f29c002d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d9b8c640 1 -- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c0004350 msgr2=0x7f29c0004820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d9b8c640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c0004350 0x7f29c0004820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d9b8c640 1 -- 172.21.15.135:0/807703733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29c0005070 msgr2=0x7f29c002d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d9b8c640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29c0005070 0x7f29c002d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:17.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.972+0000 7f29d9b8c640 1 -- 172.21.15.135:0/807703733 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c0003520 con 0x7f29c00039b0 2024-04-26T22:09:17.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.973+0000 7f29da38d640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29c0005070 0x7f29c002d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:09:17.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.976+0000 7f29d9b8c640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29c00039b0 0x7f29c0003dd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f29d4051fc0 tx=0x7f29c0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8050735bc24b7a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:17.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.977+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4050f10 con 0x7f29c00039b0 2024-04-26T22:09:17.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.977+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d409d600 con 0x7f29c00039b0 2024-04-26T22:09:17.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.977+0000 7f29d8b8a640 1 -- 172.21.15.135:0/807703733 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d409da30 con 0x7f29c00039b0 2024-04-26T22:09:17.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.980+0000 7f29dbdf0640 1 -- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29c00039b0 msgr2=0x7f29c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:17.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.980+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/807703733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29c00039b0 0x7f29c0003dd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f29d4051fc0 tx=0x7f29c0005830 comp rx=0 tx=0).stop 2024-04-26T22:09:17.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.980+0000 7f29da38d640 1 -- 172.21.15.135:0/807703733 reap_dead start 2024-04-26T22:09:17.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.990+0000 7f29dbdf0640 1 -- 172.21.15.135:0/807703733 shutdown_connections 2024-04-26T22:09:17.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.990+0000 7f29dbdf0640 1 -- 172.21.15.135:0/807703733 >> 172.21.15.135:0/807703733 conn(0x7f29cc031760 msgr2=0x7f29cc031b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:17.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.994+0000 7f29dbdf0640 1 -- 172.21.15.135:0/807703733 shutdown_connections 2024-04-26T22:09:17.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.994+0000 7f29dbdf0640 1 -- 172.21.15.135:0/807703733 wait complete. 2024-04-26T22:09:17.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.996+0000 7f29dbdf0640 1 Processor -- start 2024-04-26T22:09:17.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.996+0000 7f29dbdf0640 1 -- start start 2024-04-26T22:09:17.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.997+0000 7f29dbdf0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c00039b0 0x7f29cc02f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29dbdf0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 0x7f29cc046c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:17.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29d9b8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c00039b0 0x7f29cc02f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:17.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29d9b8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c00039b0 0x7f29cc02f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:48576/0 (socket says 172.21.15.135:48576) 2024-04-26T22:09:17.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29d938b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 0x7f29cc046c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:18.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29d938b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 0x7f29cc046c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39580/0 (socket says 172.21.15.135:39580) 2024-04-26T22:09:18.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29dbdf0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc047170 0x7f29cc0d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:18.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29dbdf0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc0db270 con 0x7f29cc1673a0 2024-04-26T22:09:18.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29dbdf0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc035700 con 0x7f29c00039b0 2024-04-26T22:09:18.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.998+0000 7f29dbdf0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc032300 con 0x7f29cc047170 2024-04-26T22:09:18.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:17.999+0000 7f29d9b8c640 1 -- 172.21.15.135:0/1360580108 learned_addr learned my addr 172.21.15.135:0/1360580108 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:18.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.000+0000 7f29d938b640 1 -- 172.21.15.135:0/1360580108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc047170 msgr2=0x7f29cc0d93a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:09:18.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.000+0000 7f29d938b640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc047170 0x7f29cc0d93a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:18.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.000+0000 7f29d938b640 1 -- 172.21.15.135:0/1360580108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c00039b0 msgr2=0x7f29cc02f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:18.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.000+0000 7f29d938b640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c00039b0 0x7f29cc02f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:18.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.000+0000 7f29d938b640 1 -- 172.21.15.135:0/1360580108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc0de5d0 con 0x7f29cc1673a0 2024-04-26T22:09:18.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.000+0000 7f29d938b640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 0x7f29cc046c30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f29cc04b4d0 tx=0x7f29c400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:18.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.006+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4004030 con 0x7f29cc1673a0 2024-04-26T22:09:18.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.006+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc049630 con 0x7f29cc1673a0 2024-04-26T22:09:18.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.009+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc166160 con 0x7f29cc1673a0 2024-04-26T22:09:18.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.010+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c40041d0 con 0x7f29cc1673a0 2024-04-26T22:09:18.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.010+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4003610 con 0x7f29cc1673a0 2024-04-26T22:09:18.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.011+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29c4003830 con 0x7f29cc1673a0 2024-04-26T22:09:18.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.014+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc005290 con 0x7f29cc1673a0 2024-04-26T22:09:18.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.024+0000 7f29ca7fc640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29b806ca20 0x7f29b806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:18.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.024+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f29c408f170 con 0x7f29cc1673a0 2024-04-26T22:09:18.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.024+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29c4015020 con 0x7f29cc1673a0 2024-04-26T22:09:18.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.027+0000 7f29d9b8c640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29b806ca20 0x7f29b806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:18.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.028+0000 7f29d9b8c640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29b806ca20 0x7f29b806eee0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f29d406acf0 tx=0x7f29d409b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:18.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.514+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29bc005850 con 0x7f29cc1673a0 2024-04-26T22:09:18.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.515+0000 7f29ca7fc640 1 -- 172.21.15.135:0/1360580108 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f29c40045c0 con 0x7f29cc1673a0 2024-04-26T22:09:18.516 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:18.516 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:18.517 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:18.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29b806ca20 msgr2=0x7f29b806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:18.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29b806ca20 0x7f29b806eee0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f29d406acf0 tx=0x7f29d409b000 comp rx=0 tx=0).stop 2024-04-26T22:09:18.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 msgr2=0x7f29cc046c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:18.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 0x7f29cc046c30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f29cc04b4d0 tx=0x7f29c400cc50 comp rx=0 tx=0).stop 2024-04-26T22:09:18.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 shutdown_connections 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29b806ca20 0x7f29b806eee0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29cc047170 0x7f29cc0d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29cc1673a0 0x7f29cc046c30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 --2- 172.21.15.135:0/1360580108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29c00039b0 0x7f29cc02f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 >> 172.21.15.135:0/1360580108 conn(0x7f29cc031760 msgr2=0x7f29cc033580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 shutdown_connections 2024-04-26T22:09:18.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:18.519+0000 7f29dbdf0640 1 -- 172.21.15.135:0/1360580108 wait complete. 2024-04-26T22:09:18.551 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-04-26T22:09:18.551 INFO:tasks.mds_thrash.fs.[b]:waiting for 30 secs before thrashing 2024-04-26T22:09:31.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:09:31.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:09:36.288 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:36.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.461+0000 7f8761772640 1 Processor -- start 2024-04-26T22:09:36.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.461+0000 7f8761772640 1 -- start start 2024-04-26T22:09:36.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.462+0000 7f8761772640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 0x7f875c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.462+0000 7f8761772640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f875c0c8dc0 0x7f875c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.462+0000 7f8761772640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f875c0ac680 0x7f875c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.462+0000 7f8761772640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f875c0c4bd0 con 0x7f875c0c8dc0 2024-04-26T22:09:36.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.462+0000 7f8761772640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f875c06f4e0 con 0x7f875c0ac680 2024-04-26T22:09:36.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.462+0000 7f8761772640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0c48f0 con 0x7f875c0c2cd0 2024-04-26T22:09:36.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.463+0000 7f875b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f875c0ac680 0x7f875c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:36.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.463+0000 7f875affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 0x7f875c0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:36.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.463+0000 7f875affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 0x7f875c0ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34206/0 (socket says 172.21.15.135:34206) 2024-04-26T22:09:36.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.463+0000 7f875b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f875c0ac680 0x7f875c1538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:49798/0 (socket says 172.21.15.135:49798) 2024-04-26T22:09:36.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.463+0000 7f875b7fe640 1 -- 172.21.15.135:0/2107730954 learned_addr learned my addr 172.21.15.135:0/2107730954 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:36.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.464+0000 7f875affd640 1 -- 172.21.15.135:0/2107730954 >> v2:172.21.15.139:3300/0 conn(0x7f875c0ac680 msgr2=0x7f875c1538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:36.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.464+0000 7f875affd640 1 --2- 172.21.15.135:0/2107730954 >> v2:172.21.15.139:3300/0 conn(0x7f875c0ac680 0x7f875c1538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.464+0000 7f875affd640 1 -- 172.21.15.135:0/2107730954 >> v2:172.21.15.135:3300/0 conn(0x7f875c0c8dc0 msgr2=0x7f875c1533b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:36.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.464+0000 7f875affd640 1 --2- 172.21.15.135:0/2107730954 >> v2:172.21.15.135:3300/0 conn(0x7f875c0c8dc0 0x7f875c1533b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.464+0000 7f875affd640 1 -- 172.21.15.135:0/2107730954 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c152e90 con 0x7f875c0c2cd0 2024-04-26T22:09:36.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.466+0000 7f875affd640 1 --2- 172.21.15.135:0/2107730954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 0x7f875c0ac140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f875c0767e0 tx=0x7f875402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e96692f2f4ecbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:36.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.467+0000 7f8759ffb640 1 -- 172.21.15.135:0/2107730954 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754007740 con 0x7f875c0c2cd0 2024-04-26T22:09:36.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.467+0000 7f8759ffb640 1 -- 172.21.15.135:0/2107730954 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87540078e0 con 0x7f875c0c2cd0 2024-04-26T22:09:36.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.468+0000 7f8761772640 1 -- 172.21.15.135:0/2107730954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 msgr2=0x7f875c0ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:36.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.468+0000 7f8761772640 1 --2- 172.21.15.135:0/2107730954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 0x7f875c0ac140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f875c0767e0 tx=0x7f875402a5f0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.471+0000 7f8761772640 1 -- 172.21.15.135:0/2107730954 shutdown_connections 2024-04-26T22:09:36.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.471+0000 7f8761772640 1 --2- 172.21.15.135:0/2107730954 >> v2:172.21.15.135:3300/0 conn(0x7f875c0c8dc0 0x7f875c1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.471+0000 7f8761772640 1 --2- 172.21.15.135:0/2107730954 >> v2:172.21.15.139:3300/0 conn(0x7f875c0ac680 0x7f875c1538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.471+0000 7f8761772640 1 --2- 172.21.15.135:0/2107730954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0c2cd0 0x7f875c0ac140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.471+0000 7f8761772640 1 -- 172.21.15.135:0/2107730954 >> 172.21.15.135:0/2107730954 conn(0x7f875c0c82d0 msgr2=0x7f875c0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:36.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.471+0000 7f8761772640 1 -- 172.21.15.135:0/2107730954 shutdown_connections 2024-04-26T22:09:36.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.472+0000 7f8761772640 1 -- 172.21.15.135:0/2107730954 wait complete. 2024-04-26T22:09:36.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.474+0000 7f8761772640 1 Processor -- start 2024-04-26T22:09:36.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.474+0000 7f8761772640 1 -- start start 2024-04-26T22:09:36.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.475+0000 7f8761772640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0ac680 0x7f875c1ed760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f8761772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f875c0c2cd0 0x7f875c142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f8761772640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 0x7f875c155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 0x7f875c155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:36.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 0x7f875c155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:41952/0 (socket says 172.21.15.135:41952) 2024-04-26T22:09:36.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f875b7fe640 1 -- 172.21.15.135:0/2713420603 learned_addr learned my addr 172.21.15.135:0/2713420603 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:36.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f875affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0ac680 0x7f875c1ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:36.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.476+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f875c0abbf0 con 0x7f875c0c8dc0 2024-04-26T22:09:36.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f875c0ab8f0 con 0x7f875c0c2cd0 2024-04-26T22:09:36.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0abd70 con 0x7f875c0ac680 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f875b7fe640 1 -- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0ac680 msgr2=0x7f875c1ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f875b7fe640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0ac680 0x7f875c1ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f875b7fe640 1 -- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f875c0c2cd0 msgr2=0x7f875c142b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f875b7fe640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f875c0c2cd0 0x7f875c142b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.477+0000 7f875b7fe640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c152920 con 0x7f875c0c8dc0 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.478+0000 7f875b7fe640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 0x7f875c155c70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e920 tx=0x7f874c00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:36.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.478+0000 7f875affd640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0ac680 0x7f875c1ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:09:36.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.479+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c017020 con 0x7f875c0c8dc0 2024-04-26T22:09:36.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.480+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c004230 con 0x7f875c0c8dc0 2024-04-26T22:09:36.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.480+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c00f040 con 0x7f875c0c8dc0 2024-04-26T22:09:36.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.480+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c1f0360 con 0x7f875c0c8dc0 2024-04-26T22:09:36.483 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.483+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c1430d0 con 0x7f875c0c8dc0 2024-04-26T22:09:36.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.484+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f874c01a020 con 0x7f875c0c8dc0 2024-04-26T22:09:36.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.484+0000 7f87437fe640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f874806c9f0 0x7f874806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:36.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.485+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f874c08f170 con 0x7f875c0c8dc0 2024-04-26T22:09:36.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.485+0000 7f875affd640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f874806c9f0 0x7f874806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:36.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.487+0000 7f875affd640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f874806c9f0 0x7f874806eeb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:36.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.487+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c0acb50 con 0x7f875c0c8dc0 2024-04-26T22:09:36.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.498+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f874c04ec00 con 0x7f875c0c8dc0 2024-04-26T22:09:36.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.934+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f875c0507a0 con 0x7f875c0c8dc0 2024-04-26T22:09:36.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.938+0000 7f87437fe640 1 -- 172.21.15.135:0/2713420603 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f874c059160 con 0x7f875c0c8dc0 2024-04-26T22:09:36.938 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:36.938 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:36.938 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f874806c9f0 msgr2=0x7f874806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f874806c9f0 0x7f874806eeb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402f000 comp rx=0 tx=0).stop 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 msgr2=0x7f875c155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 0x7f875c155c70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e920 tx=0x7f874c00ede0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 shutdown_connections 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f874806c9f0 0x7f874806eeb0 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402f000 comp rx=0 tx=0).stop 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f875c0c8dc0 0x7f875c155c70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f875c0c2cd0 0x7f875c142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 --2- 172.21.15.135:0/2713420603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f875c0ac680 0x7f875c1ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 >> 172.21.15.135:0/2713420603 conn(0x7f875c0c82d0 msgr2=0x7f875c1532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 shutdown_connections 2024-04-26T22:09:36.943 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:36.942+0000 7f8761772640 1 -- 172.21.15.135:0/2713420603 wait complete. 2024-04-26T22:09:36.964 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-04-26T22:09:36.964 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:09:36.964 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:09:36.988 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:09:36.988 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:37.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.205+0000 7f22c533b640 1 Processor -- start 2024-04-26T22:09:37.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.205+0000 7f22c533b640 1 -- start start 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.206+0000 7f22c533b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22c0166ad0 0x7f22c006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.206+0000 7f22c533b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f22c008d100 0x7f22c00c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.206+0000 7f22c533b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c01670b0 con 0x7f22c00ab9b0 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.206+0000 7f22c533b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f22c0167c80 con 0x7f22c008d100 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.206+0000 7f22c533b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f22c0167350 con 0x7f22c0166ad0 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22c0166ad0 0x7f22c006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f22c00ab9b0 0x7f22c01663c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:34216/0 (socket says 172.21.15.135:34216) 2024-04-26T22:09:37.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bffff640 1 -- 172.21.15.135:0/4174315401 learned_addr learned my addr 172.21.15.135:0/4174315401 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:37.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bf7fe640 1 -- 172.21.15.135:0/4174315401 >> v2:172.21.15.135:3300/0 conn(0x7f22c008d100 msgr2=0x7f22c00c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:37.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bf7fe640 1 --2- 172.21.15.135:0/4174315401 >> v2:172.21.15.135:3300/0 conn(0x7f22c008d100 0x7f22c00c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bf7fe640 1 -- 172.21.15.135:0/4174315401 >> v1:172.21.15.139:6789/0 conn(0x7f22c00ab9b0 legacy=0x7f22c01663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:37.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.207+0000 7f22bf7fe640 1 -- 172.21.15.135:0/4174315401 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c0167630 con 0x7f22c0166ad0 2024-04-26T22:09:37.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.208+0000 7f22bf7fe640 1 --2- 172.21.15.135:0/4174315401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c006de50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f22c00711c0 tx=0x7f22b402e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341669918680b149 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:37.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.210+0000 7f22beffd640 1 -- 172.21.15.135:0/4174315401 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4009780 con 0x7f22c0166ad0 2024-04-26T22:09:37.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.210+0000 7f22beffd640 1 -- 172.21.15.135:0/4174315401 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b4009920 con 0x7f22c0166ad0 2024-04-26T22:09:37.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.210+0000 7f22beffd640 1 -- 172.21.15.135:0/4174315401 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4009cf0 con 0x7f22c0166ad0 2024-04-26T22:09:37.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.211+0000 7f22c533b640 1 -- 172.21.15.135:0/4174315401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 msgr2=0x7f22c006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:37.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.211+0000 7f22c533b640 1 --2- 172.21.15.135:0/4174315401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c006de50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f22c00711c0 tx=0x7f22b402e780 comp rx=0 tx=0).stop 2024-04-26T22:09:37.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.212+0000 7f22c533b640 1 -- 172.21.15.135:0/4174315401 shutdown_connections 2024-04-26T22:09:37.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.212+0000 7f22c533b640 1 --2- 172.21.15.135:0/4174315401 >> v2:172.21.15.135:3300/0 conn(0x7f22c008d100 0x7f22c00c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.212+0000 7f22c533b640 1 --2- 172.21.15.135:0/4174315401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c006de50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.212+0000 7f22c533b640 1 -- 172.21.15.135:0/4174315401 >> 172.21.15.135:0/4174315401 conn(0x7f22c00c4cd0 msgr2=0x7f22c00c50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:37.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.214+0000 7f22c533b640 1 -- 172.21.15.135:0/4174315401 shutdown_connections 2024-04-26T22:09:37.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.215+0000 7f22c533b640 1 -- 172.21.15.135:0/4174315401 wait complete. 2024-04-26T22:09:37.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.216+0000 7f22c533b640 1 Processor -- start 2024-04-26T22:09:37.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.221+0000 7f22c533b640 1 -- start start 2024-04-26T22:09:37.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c533b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22c008d100 0x7f22c0085a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:37.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c533b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 0x7f22c02049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c533b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c0204f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c533b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00abd90 con 0x7f22c008d100 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c533b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00ac410 con 0x7f22c0166ad0 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c533b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22c00abf10 con 0x7f22c00ab9b0 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22bf7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 0x7f22c02049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22bf7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 0x7f22c02049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34228/0 (socket says 172.21.15.135:34228) 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22bf7fe640 1 -- 172.21.15.135:0/391302655 learned_addr learned my addr 172.21.15.135:0/391302655 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.222+0000 7f22c4b3a640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c0204f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bffff640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22c008d100 0x7f22c0085a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bf7fe640 1 -- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 msgr2=0x7f22c0204f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bf7fe640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c0204f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bf7fe640 1 -- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22c008d100 msgr2=0x7f22c0085a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:37.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bf7fe640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22c008d100 0x7f22c0085a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bf7fe640 1 -- 172.21.15.135:0/391302655 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c00c6ac0 con 0x7f22c00ab9b0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.223+0000 7f22bf7fe640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 0x7f22c02049d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22b4000c00 tx=0x7f22b40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.224+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4041020 con 0x7f22c00ab9b0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.224+0000 7f22c533b640 1 -- 172.21.15.135:0/391302655 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c0205450 con 0x7f22c00ab9b0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.224+0000 7f22c533b640 1 -- 172.21.15.135:0/391302655 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c00c7040 con 0x7f22c00ab9b0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.224+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b4010560 con 0x7f22c00ab9b0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.225+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4003850 con 0x7f22c00ab9b0 2024-04-26T22:09:37.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.227+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22b4044020 con 0x7f22c00ab9b0 2024-04-26T22:09:37.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.228+0000 7f22bcff9640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f22ac06c940 0x7f22ac06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:37.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.228+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f22b40ac980 con 0x7f22c00ab9b0 2024-04-26T22:09:37.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.228+0000 7f22c533b640 1 -- 172.21.15.135:0/391302655 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a0005290 con 0x7f22c00ab9b0 2024-04-26T22:09:37.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.228+0000 7f22bffff640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f22ac06c940 0x7f22ac06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:37.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.229+0000 7f22bffff640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f22ac06c940 0x7f22ac06ee00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22b00047c0 tx=0x7f22b000a120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:37.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.242+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22b404b030 con 0x7f22c00ab9b0 2024-04-26T22:09:37.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.683+0000 7f22c533b640 1 -- 172.21.15.135:0/391302655 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a0005850 con 0x7f22c00ab9b0 2024-04-26T22:09:37.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.684+0000 7f22bcff9640 1 -- 172.21.15.135:0/391302655 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f22b407a140 con 0x7f22c00ab9b0 2024-04-26T22:09:37.687 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:37.687 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:37.687 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:37.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 -- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f22ac06c940 msgr2=0x7f22ac06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f22ac06c940 0x7f22ac06ee00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22b00047c0 tx=0x7f22b000a120 comp rx=0 tx=0).stop 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 -- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 msgr2=0x7f22c02049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 0x7f22c02049d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22b4000c00 tx=0x7f22b40046c0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 -- 172.21.15.135:0/391302655 shutdown_connections 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f22ac06c940 0x7f22ac06ee00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0166ad0 0x7f22c0204f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22c00ab9b0 0x7f22c02049d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 --2- 172.21.15.135:0/391302655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22c008d100 0x7f22c0085a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 -- 172.21.15.135:0/391302655 >> 172.21.15.135:0/391302655 conn(0x7f22c00c4cd0 msgr2=0x7f22c01628c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 -- 172.21.15.135:0/391302655 shutdown_connections 2024-04-26T22:09:37.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:37.691+0000 7f22a67fc640 1 -- 172.21.15.135:0/391302655 wait complete. 2024-04-26T22:09:37.718 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:09:39.719 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:39.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.895+0000 7f27389f0640 1 Processor -- start 2024-04-26T22:09:39.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.895+0000 7f27389f0640 1 -- start start 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.896+0000 7f27389f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2734158c20 0x7f273415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.896+0000 7f27389f0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27340cbde0 con 0x7f27340c90c0 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.896+0000 7f27389f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27340cc0c0 con 0x7f27340bfbb0 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.896+0000 7f27389f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f273415c150 con 0x7f2734158c20 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.897+0000 7f2731d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f27340bfbb0 0x7f2734158510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:51514/0 (socket says 172.21.15.135:51514) 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.897+0000 7f2731d9b640 1 -- 172.21.15.135:0/2966660518 learned_addr learned my addr 172.21.15.135:0/2966660518 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:39.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.897+0000 7f2732d9d640 1 --2- 172.21.15.135:0/2966660518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2734158c20 0x7f273415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.898+0000 7f273159a640 1 -- 172.21.15.135:0/2966660518 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761553754 0 0) 0x7f27340cc0c0 con 0x7f27340bfbb0 2024-04-26T22:09:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.898+0000 7f273159a640 1 -- 172.21.15.135:0/2966660518 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f271c003520 con 0x7f27340bfbb0 2024-04-26T22:09:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.898+0000 7f2732d9d640 1 -- 172.21.15.135:0/2966660518 >> v1:172.21.15.139:6789/0 conn(0x7f27340bfbb0 legacy=0x7f2734158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.898+0000 7f2732d9d640 1 -- 172.21.15.135:0/2966660518 >> v1:172.21.15.135:6789/0 conn(0x7f27340c90c0 legacy=0x7f27340cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.898+0000 7f2732d9d640 1 -- 172.21.15.135:0/2966660518 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27340ca5d0 con 0x7f2734158c20 2024-04-26T22:09:39.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.900+0000 7f2732d9d640 1 --2- 172.21.15.135:0/2966660518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2734158c20 0x7f273415bc10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f272c009dc0 tx=0x7f272c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1b96f30a3a5473 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:39.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.901+0000 7f273159a640 1 -- 172.21.15.135:0/2966660518 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c031600 con 0x7f2734158c20 2024-04-26T22:09:39.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.901+0000 7f273159a640 1 -- 172.21.15.135:0/2966660518 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c0317a0 con 0x7f2734158c20 2024-04-26T22:09:39.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.902+0000 7f273159a640 1 -- 172.21.15.135:0/2966660518 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c031b50 con 0x7f2734158c20 2024-04-26T22:09:39.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.903+0000 7f27389f0640 1 -- 172.21.15.135:0/2966660518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2734158c20 msgr2=0x7f273415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:39.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.903+0000 7f27389f0640 1 --2- 172.21.15.135:0/2966660518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2734158c20 0x7f273415bc10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f272c009dc0 tx=0x7f272c02a5d0 comp rx=0 tx=0).stop 2024-04-26T22:09:39.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.903+0000 7f27389f0640 1 -- 172.21.15.135:0/2966660518 shutdown_connections 2024-04-26T22:09:39.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.903+0000 7f27389f0640 1 --2- 172.21.15.135:0/2966660518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2734158c20 0x7f273415bc10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:39.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.903+0000 7f27389f0640 1 -- 172.21.15.135:0/2966660518 >> 172.21.15.135:0/2966660518 conn(0x7f27340ae010 msgr2=0x7f27340ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:39.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.906+0000 7f27389f0640 1 -- 172.21.15.135:0/2966660518 shutdown_connections 2024-04-26T22:09:39.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.906+0000 7f27389f0640 1 -- 172.21.15.135:0/2966660518 wait complete. 2024-04-26T22:09:39.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.908+0000 7f27389f0640 1 Processor -- start 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.908+0000 7f27389f0640 1 -- start start 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.909+0000 7f27389f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27340bfbb0 0x7f27340b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.909+0000 7f27389f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27340c90c0 0x7f27340b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.909+0000 7f27389f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2734158c20 0x7f27341f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f27389f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27340c94a0 con 0x7f2734158c20 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f27389f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27340ca380 con 0x7f27340c90c0 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f27389f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f27340c9620 con 0x7f27340bfbb0 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f2731d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27340c90c0 0x7f27340b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:39.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f2731d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27340c90c0 0x7f27340b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:49774/0 (socket says 172.21.15.135:49774) 2024-04-26T22:09:39.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f2731d9b640 1 -- 172.21.15.135:0/2223641005 learned_addr learned my addr 172.21.15.135:0/2223641005 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:39.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f2732d9d640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2734158c20 0x7f27341f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:39.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.910+0000 7f273259c640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27340bfbb0 0x7f27340b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:39.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.911+0000 7f2732d9d640 1 -- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27340bfbb0 msgr2=0x7f27340b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:39.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.911+0000 7f2732d9d640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27340bfbb0 0x7f27340b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:39.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.911+0000 7f2732d9d640 1 -- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27340c90c0 msgr2=0x7f27340b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:39.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.911+0000 7f2732d9d640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27340c90c0 0x7f27340b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:39.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.911+0000 7f2732d9d640 1 -- 172.21.15.135:0/2223641005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341538a0 con 0x7f2734158c20 2024-04-26T22:09:39.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.911+0000 7f2732d9d640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2734158c20 0x7f27341f40e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f272c031000 tx=0x7f272c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:39.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.919+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c040070 con 0x7f2734158c20 2024-04-26T22:09:39.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.919+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c004240 con 0x7f2734158c20 2024-04-26T22:09:39.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.919+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c004610 con 0x7f2734158c20 2024-04-26T22:09:39.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.923+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273415ce40 con 0x7f2734158c20 2024-04-26T22:09:39.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.923+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27340c6560 con 0x7f2734158c20 2024-04-26T22:09:39.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.925+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2734076650 con 0x7f2734158c20 2024-04-26T22:09:39.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.929+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f272c039040 con 0x7f2734158c20 2024-04-26T22:09:39.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.930+0000 7f2722ffd640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f271406c9c0 0x7f271406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:39.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.930+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f272c0ad0f0 con 0x7f2734158c20 2024-04-26T22:09:39.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.930+0000 7f273259c640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f271406c9c0 0x7f271406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:39.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.931+0000 7f273259c640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f271406c9c0 0x7f271406ee80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2724000f60 tx=0x7f2724009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:39.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:39.936+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f272c049060 con 0x7f2734158c20 2024-04-26T22:09:40.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.388+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27340507a0 con 0x7f2734158c20 2024-04-26T22:09:40.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.392+0000 7f2722ffd640 1 -- 172.21.15.135:0/2223641005 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f272c07ae80 con 0x7f2734158c20 2024-04-26T22:09:40.397 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:40.397 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:40.397 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:40.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.397+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f271406c9c0 msgr2=0x7f271406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:40.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.397+0000 7f27389f0640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f271406c9c0 0x7f271406ee80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2724000f60 tx=0x7f2724009040 comp rx=0 tx=0).stop 2024-04-26T22:09:40.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.397+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2734158c20 msgr2=0x7f27341f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:40.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.397+0000 7f27389f0640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2734158c20 0x7f27341f40e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f272c031000 tx=0x7f272c004a40 comp rx=0 tx=0).stop 2024-04-26T22:09:40.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 shutdown_connections 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f271406c9c0 0x7f271406ee80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2734158c20 0x7f27341f40e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27340c90c0 0x7f27340b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 --2- 172.21.15.135:0/2223641005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27340bfbb0 0x7f27340b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 >> 172.21.15.135:0/2223641005 conn(0x7f27340ae010 msgr2=0x7f27340ac060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 shutdown_connections 2024-04-26T22:09:40.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:40.401+0000 7f27389f0640 1 -- 172.21.15.135:0/2223641005 wait complete. 2024-04-26T22:09:40.423 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:09:42.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:42.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.602+0000 7f154a237640 1 Processor -- start 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.602+0000 7f154a237640 1 -- start start 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.603+0000 7f154a237640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 0x7f15441533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.603+0000 7f154a237640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15440ab6a0 0x7f15441538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.603+0000 7f154a237640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154406ff20 con 0x7f1544150200 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.603+0000 7f154a237640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f15440c4bd0 con 0x7f15440ab6a0 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.603+0000 7f154a237640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f154406f4e0 con 0x7f15440c2cd0 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.604+0000 7f1542ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 0x7f15441533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:42.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.604+0000 7f1542ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 0x7f15441533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:41992/0 (socket says 172.21.15.135:41992) 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.604+0000 7f1542ffd640 1 -- 172.21.15.135:0/765760949 learned_addr learned my addr 172.21.15.135:0/765760949 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.604+0000 7f1543fff640 1 --2- 172.21.15.135:0/765760949 >> v2:172.21.15.135:3300/0 conn(0x7f15440ab6a0 0x7f15441538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.604+0000 7f1542ffd640 1 -- 172.21.15.135:0/765760949 >> v2:172.21.15.135:3300/0 conn(0x7f15440ab6a0 msgr2=0x7f15441538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.605+0000 7f1542ffd640 1 --2- 172.21.15.135:0/765760949 >> v2:172.21.15.135:3300/0 conn(0x7f15440ab6a0 0x7f15441538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.605+0000 7f1542ffd640 1 -- 172.21.15.135:0/765760949 >> v1:172.21.15.135:6789/0 conn(0x7f1544150200 legacy=0x7f15440ac140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.605+0000 7f1542ffd640 1 -- 172.21.15.135:0/765760949 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15440c48f0 con 0x7f15440c2cd0 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.605+0000 7f1543fff640 1 --2- 172.21.15.135:0/765760949 >> v2:172.21.15.135:3300/0 conn(0x7f15440ab6a0 0x7f15441538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:09:42.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.606+0000 7f1542ffd640 1 --2- 172.21.15.135:0/765760949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 0x7f15441533b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f153c009c30 tx=0x7f153c02cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a78471ff32a129bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:42.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.607+0000 7f15427fc640 1 -- 172.21.15.135:0/765760949 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c02d770 con 0x7f15440c2cd0 2024-04-26T22:09:42.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.607+0000 7f15427fc640 1 -- 172.21.15.135:0/765760949 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153c02d910 con 0x7f15440c2cd0 2024-04-26T22:09:42.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.607+0000 7f15427fc640 1 -- 172.21.15.135:0/765760949 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c02dce0 con 0x7f15440c2cd0 2024-04-26T22:09:42.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.609+0000 7f154a237640 1 -- 172.21.15.135:0/765760949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 msgr2=0x7f15441533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:42.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.609+0000 7f154a237640 1 --2- 172.21.15.135:0/765760949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 0x7f15441533b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f153c009c30 tx=0x7f153c02cc70 comp rx=0 tx=0).stop 2024-04-26T22:09:42.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.609+0000 7f154a237640 1 -- 172.21.15.135:0/765760949 shutdown_connections 2024-04-26T22:09:42.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.609+0000 7f154a237640 1 --2- 172.21.15.135:0/765760949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15440c2cd0 0x7f15441533b0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f153c009c30 tx=0x7f153c02cc70 comp rx=0 tx=0).stop 2024-04-26T22:09:42.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.609+0000 7f154a237640 1 --2- 172.21.15.135:0/765760949 >> v2:172.21.15.135:3300/0 conn(0x7f15440ab6a0 0x7f15441538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:42.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.609+0000 7f154a237640 1 -- 172.21.15.135:0/765760949 >> 172.21.15.135:0/765760949 conn(0x7f15440c82d0 msgr2=0x7f15440c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:42.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.611+0000 7f154a237640 1 -- 172.21.15.135:0/765760949 shutdown_connections 2024-04-26T22:09:42.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.612+0000 7f154a237640 1 -- 172.21.15.135:0/765760949 wait complete. 2024-04-26T22:09:42.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.614+0000 7f154a237640 1 Processor -- start 2024-04-26T22:09:42.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.614+0000 7f154a237640 1 -- start start 2024-04-26T22:09:42.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.615+0000 7f154a237640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15440ab6a0 0x7f1544156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:42.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.615+0000 7f154a237640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1544150200 0x7f1544143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:42.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.615+0000 7f154a237640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154414a4a0 0x7f1544144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.615+0000 7f154a237640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15440ac9d0 con 0x7f15440ab6a0 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.615+0000 7f154a237640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f15440ac850 con 0x7f154414a4a0 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.615+0000 7f154a237640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f15441505e0 con 0x7f1544150200 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.616+0000 7f1542ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1544150200 0x7f1544143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.616+0000 7f1542ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1544150200 0x7f1544143ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:42008/0 (socket says 172.21.15.135:42008) 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.616+0000 7f1542ffd640 1 -- 172.21.15.135:0/605291603 learned_addr learned my addr 172.21.15.135:0/605291603 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.616+0000 7f1543fff640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154414a4a0 0x7f1544144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.618+0000 7f15437fe640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15440ab6a0 0x7f1544156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.619+0000 7f15437fe640 1 -- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1544150200 msgr2=0x7f1544143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.619+0000 7f15437fe640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1544150200 0x7f1544143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:42.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.619+0000 7f15437fe640 1 -- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154414a4a0 msgr2=0x7f1544144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:42.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.619+0000 7f15437fe640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154414a4a0 0x7f1544144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:42.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.619+0000 7f15437fe640 1 -- 172.21.15.135:0/605291603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1544072680 con 0x7f15440ab6a0 2024-04-26T22:09:42.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.624+0000 7f15437fe640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15440ab6a0 0x7f1544156080 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f153400b750 tx=0x7f153400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:42.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.625+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534019070 con 0x7f15440ab6a0 2024-04-26T22:09:42.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.625+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153400c720 con 0x7f15440ab6a0 2024-04-26T22:09:42.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.625+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153400be60 con 0x7f15440ab6a0 2024-04-26T22:09:42.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.626+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15440720c0 con 0x7f15440ab6a0 2024-04-26T22:09:42.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.626+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1544155b60 con 0x7f15440ab6a0 2024-04-26T22:09:42.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.629+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f153401c070 con 0x7f15440ab6a0 2024-04-26T22:09:42.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.629+0000 7f152bfff640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153006c9c0 0x7f153006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:42.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.629+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f153408b5e0 con 0x7f15440ab6a0 2024-04-26T22:09:42.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.630+0000 7f1542ffd640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153006c9c0 0x7f153006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:42.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.633+0000 7f1542ffd640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153006c9c0 0x7f153006ee80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f153c000f80 tx=0x7f153c031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:42.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.633+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15440507a0 con 0x7f15440ab6a0 2024-04-26T22:09:42.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:42.645+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1534058c60 con 0x7f15440ab6a0 2024-04-26T22:09:43.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.105+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f154406fb70 con 0x7f15440ab6a0 2024-04-26T22:09:43.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.107+0000 7f152bfff640 1 -- 172.21.15.135:0/605291603 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f153405cf60 con 0x7f15440ab6a0 2024-04-26T22:09:43.108 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:43.108 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:43.108 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:43.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153006c9c0 msgr2=0x7f153006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153006c9c0 0x7f153006ee80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f153c000f80 tx=0x7f153c031000 comp rx=0 tx=0).stop 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15440ab6a0 msgr2=0x7f1544156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15440ab6a0 0x7f1544156080 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f153400b750 tx=0x7f153400bc10 comp rx=0 tx=0).stop 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 shutdown_connections 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f153006c9c0 0x7f153006ee80 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f153c000f80 tx=0x7f153c031000 comp rx=0 tx=0).stop 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154414a4a0 0x7f1544144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1544150200 0x7f1544143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 --2- 172.21.15.135:0/605291603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15440ab6a0 0x7f1544156080 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:43.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.111+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 >> 172.21.15.135:0/605291603 conn(0x7f15440c82d0 msgr2=0x7f15440c37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:43.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.116+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 shutdown_connections 2024-04-26T22:09:43.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:43.116+0000 7f154a237640 1 -- 172.21.15.135:0/605291603 wait complete. 2024-04-26T22:09:43.137 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:09:45.138 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:45.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.316+0000 7f477e837640 1 Processor -- start 2024-04-26T22:09:45.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.316+0000 7f477e837640 1 -- start start 2024-04-26T22:09:45.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.317+0000 7f477e837640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47700070b0 0x7f4770007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.317+0000 7f477e837640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.317+0000 7f477e837640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47700de840 con 0x7f477000b900 2024-04-26T22:09:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.317+0000 7f477e837640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f47700df130 con 0x7f47700070b0 2024-04-26T22:09:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.317+0000 7f477e837640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47700deae0 con 0x7f47700e2430 2024-04-26T22:09:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.318+0000 7f477fa99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.318+0000 7f477d835640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47700070b0 0x7f4770007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.318+0000 7f477fa99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700e90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:42012/0 (socket says 172.21.15.135:42012) 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.318+0000 7f477fa99640 1 -- 172.21.15.135:0/2487353296 learned_addr learned my addr 172.21.15.135:0/2487353296 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.319+0000 7f477fa99640 1 -- 172.21.15.135:0/2487353296 >> v2:172.21.15.139:3300/0 conn(0x7f47700070b0 msgr2=0x7f4770007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.319+0000 7f477fa99640 1 --2- 172.21.15.135:0/2487353296 >> v2:172.21.15.139:3300/0 conn(0x7f47700070b0 0x7f4770007490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.319+0000 7f477fa99640 1 -- 172.21.15.135:0/2487353296 >> v1:172.21.15.139:6789/0 conn(0x7f477000b900 legacy=0x7f47700e1d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.319+0000 7f477fa99640 1 -- 172.21.15.135:0/2487353296 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47700dedc0 con 0x7f47700e2430 2024-04-26T22:09:45.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.320+0000 7f477fa99640 1 --2- 172.21.15.135:0/2487353296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700e90b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4778065e50 tx=0x7f47780a7160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a401582e1889d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.321+0000 7f477c833640 1 -- 172.21.15.135:0/2487353296 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780b4070 con 0x7f47700e2430 2024-04-26T22:09:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.321+0000 7f477c833640 1 -- 172.21.15.135:0/2487353296 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47780af440 con 0x7f47700e2430 2024-04-26T22:09:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.322+0000 7f477e837640 1 -- 172.21.15.135:0/2487353296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 msgr2=0x7f47700e90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.322+0000 7f477e837640 1 --2- 172.21.15.135:0/2487353296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700e90b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4778065e50 tx=0x7f47780a7160 comp rx=0 tx=0).stop 2024-04-26T22:09:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.323+0000 7f477e837640 1 -- 172.21.15.135:0/2487353296 shutdown_connections 2024-04-26T22:09:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.323+0000 7f477e837640 1 --2- 172.21.15.135:0/2487353296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700e90b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.323+0000 7f477e837640 1 --2- 172.21.15.135:0/2487353296 >> v2:172.21.15.139:3300/0 conn(0x7f47700070b0 0x7f4770007490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.323+0000 7f477e837640 1 -- 172.21.15.135:0/2487353296 >> 172.21.15.135:0/2487353296 conn(0x7f4770048460 msgr2=0x7f4770048840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:45.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.328+0000 7f477e837640 1 -- 172.21.15.135:0/2487353296 shutdown_connections 2024-04-26T22:09:45.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.332+0000 7f477e837640 1 -- 172.21.15.135:0/2487353296 wait complete. 2024-04-26T22:09:45.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.333+0000 7f477e837640 1 Processor -- start 2024-04-26T22:09:45.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.338+0000 7f477e837640 1 -- start start 2024-04-26T22:09:45.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477e837640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47700070b0 0x7f477016a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477e837640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 0x7f47700d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477e837640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477e837640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47700d8ae0 con 0x7f477000b900 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477e837640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4770048120 con 0x7f47700070b0 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477e837640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f477002e600 con 0x7f47700e2430 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477d034640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 0x7f47700d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477d034640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 0x7f47700d7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49532/0 (socket says 172.21.15.135:49532) 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477d034640 1 -- 172.21.15.135:0/221890728 learned_addr learned my addr 172.21.15.135:0/221890728 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477d034640 1 -- 172.21.15.135:0/221890728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 msgr2=0x7f47700d80f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477d034640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700d80f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.339+0000 7f477d034640 1 -- 172.21.15.135:0/221890728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47700070b0 msgr2=0x7f477016a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.340+0000 7f477d034640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47700070b0 0x7f477016a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.340+0000 7f477d034640 1 -- 172.21.15.135:0/221890728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4770045a30 con 0x7f477000b900 2024-04-26T22:09:45.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.340+0000 7f477d034640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 0x7f47700d7bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f477400b450 tx=0x7f477400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:45.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.340+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774015020 con 0x7f477000b900 2024-04-26T22:09:45.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.340+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4770178550 con 0x7f477000b900 2024-04-26T22:09:45.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.341+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477016a710 con 0x7f477000b900 2024-04-26T22:09:45.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.342+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47740118b0 con 0x7f477000b900 2024-04-26T22:09:45.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.342+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774011c80 con 0x7f477000b900 2024-04-26T22:09:45.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.343+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4760005290 con 0x7f477000b900 2024-04-26T22:09:45.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.344+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4774018020 con 0x7f477000b900 2024-04-26T22:09:45.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.345+0000 7f476e7fc640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f475c06c8f0 0x7f475c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:45.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.346+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f477408c430 con 0x7f477000b900 2024-04-26T22:09:45.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.346+0000 7f477d835640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f475c06c8f0 0x7f475c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:45.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.347+0000 7f477d835640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f475c06c8f0 0x7f475c06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4768006b10 tx=0x7f4768008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:45.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.355+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f477401f020 con 0x7f477000b900 2024-04-26T22:09:45.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.796+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4760005580 con 0x7f477000b900 2024-04-26T22:09:45.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.799+0000 7f476e7fc640 1 -- 172.21.15.135:0/221890728 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f477400d040 con 0x7f477000b900 2024-04-26T22:09:45.802 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:45.802 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:45.803 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:45.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f475c06c8f0 msgr2=0x7f475c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:45.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f475c06c8f0 0x7f475c06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4768006b10 tx=0x7f4768008040 comp rx=0 tx=0).stop 2024-04-26T22:09:45.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 msgr2=0x7f47700d7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:45.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 0x7f47700d7bb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f477400b450 tx=0x7f477400b910 comp rx=0 tx=0).stop 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 shutdown_connections 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f475c06c8f0 0x7f475c06edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47700e2430 0x7f47700d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f477000b900 0x7f47700d7bb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 --2- 172.21.15.135:0/221890728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47700070b0 0x7f477016a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.804+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 >> 172.21.15.135:0/221890728 conn(0x7f4770048460 msgr2=0x7f4770005fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.805+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 shutdown_connections 2024-04-26T22:09:45.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:45.806+0000 7f477e837640 1 -- 172.21.15.135:0/221890728 wait complete. 2024-04-26T22:09:45.831 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:09:47.832 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:48.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.008+0000 7f519c8da640 1 Processor -- start 2024-04-26T22:09:48.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.009+0000 7f519c8da640 1 -- start start 2024-04-26T22:09:48.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.010+0000 7f519c8da640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51981540f0 0x7f519815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.010+0000 7f519c8da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f51980ac610 0x7f519806f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.010+0000 7f519c8da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5198156bc0 con 0x7f5198157680 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.010+0000 7f519c8da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f51981571f0 con 0x7f51980ac610 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.010+0000 7f519c8da640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f51980c5350 con 0x7f51981540f0 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.010+0000 7f5196d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f51980ac610 0x7f519806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.011+0000 7f5196d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f51980ac610 0x7f519806f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49536/0 (socket says 172.21.15.135:49536) 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.011+0000 7f5196d9d640 1 -- 172.21.15.135:0/2339889463 learned_addr learned my addr 172.21.15.135:0/2339889463 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.011+0000 7f5196d9d640 1 -- 172.21.15.135:0/2339889463 >> v2:172.21.15.139:3300/0 conn(0x7f51981540f0 msgr2=0x7f519815ac70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.011+0000 7f5196d9d640 1 --2- 172.21.15.135:0/2339889463 >> v2:172.21.15.139:3300/0 conn(0x7f51981540f0 0x7f519815ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.011+0000 7f5196d9d640 1 -- 172.21.15.135:0/2339889463 >> v1:172.21.15.135:6789/0 conn(0x7f5198157680 legacy=0x7f51980c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:48.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.011+0000 7f5196d9d640 1 -- 172.21.15.135:0/2339889463 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51980c5630 con 0x7f51980ac610 2024-04-26T22:09:48.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.013+0000 7f5196d9d640 1 --2- 172.21.15.135:0/2339889463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51980ac610 0x7f519806f080 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f51980c5e70 tx=0x7f519002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2eec70fc8e633e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:48.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.013+0000 7f519559a640 1 -- 172.21.15.135:0/2339889463 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190036070 con 0x7f51980ac610 2024-04-26T22:09:48.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.013+0000 7f519559a640 1 -- 172.21.15.135:0/2339889463 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5190031600 con 0x7f51980ac610 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.013+0000 7f519559a640 1 -- 172.21.15.135:0/2339889463 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190036070 con 0x7f51980ac610 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.014+0000 7f519c8da640 1 -- 172.21.15.135:0/2339889463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51980ac610 msgr2=0x7f519806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.014+0000 7f519c8da640 1 --2- 172.21.15.135:0/2339889463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51980ac610 0x7f519806f080 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f51980c5e70 tx=0x7f519002a6f0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.014+0000 7f519c8da640 1 -- 172.21.15.135:0/2339889463 shutdown_connections 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.014+0000 7f519c8da640 1 --2- 172.21.15.135:0/2339889463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51980ac610 0x7f519806f080 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.014+0000 7f519c8da640 1 --2- 172.21.15.135:0/2339889463 >> v2:172.21.15.139:3300/0 conn(0x7f51981540f0 0x7f519815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.014+0000 7f519c8da640 1 -- 172.21.15.135:0/2339889463 >> 172.21.15.135:0/2339889463 conn(0x7f519806c310 msgr2=0x7f519806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:48.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.015+0000 7f519c8da640 1 -- 172.21.15.135:0/2339889463 shutdown_connections 2024-04-26T22:09:48.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.015+0000 7f519c8da640 1 -- 172.21.15.135:0/2339889463 wait complete. 2024-04-26T22:09:48.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.016+0000 7f519c8da640 1 Processor -- start 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.017+0000 7f519c8da640 1 -- start start 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f519c8da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51980ac610 0x7f5198147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f519c8da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 0x7f519814a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f519c8da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5198157680 0x7f51981f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f519c8da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51981544d0 con 0x7f51981540f0 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f519c8da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5198157af0 con 0x7f5198157680 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f519c8da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5198154650 con 0x7f51980ac610 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f5195d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 0x7f519814a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f5195d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 0x7f519814a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49552/0 (socket says 172.21.15.135:49552) 2024-04-26T22:09:48.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.018+0000 7f5195d9b640 1 -- 172.21.15.135:0/599619816 learned_addr learned my addr 172.21.15.135:0/599619816 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:48.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5195d9b640 1 -- 172.21.15.135:0/599619816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51980ac610 msgr2=0x7f5198147e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:09:48.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5195d9b640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51980ac610 0x7f5198147e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5195d9b640 1 -- 172.21.15.135:0/599619816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5198157680 msgr2=0x7f51981f3ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:09:48.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5195d9b640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5198157680 0x7f51981f3ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5195d9b640 1 -- 172.21.15.135:0/599619816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51981454f0 con 0x7f51981540f0 2024-04-26T22:09:48.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5195d9b640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 0x7f519814a2a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f518800c750 tx=0x7f518800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:48.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.019+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51880050d0 con 0x7f51981540f0 2024-04-26T22:09:48.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.020+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51981457d0 con 0x7f51981540f0 2024-04-26T22:09:48.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.020+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5198145d90 con 0x7f51981540f0 2024-04-26T22:09:48.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.020+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5188005270 con 0x7f51981540f0 2024-04-26T22:09:48.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.021+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518800a640 con 0x7f51981540f0 2024-04-26T22:09:48.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.023+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517c005290 con 0x7f51981540f0 2024-04-26T22:09:48.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.023+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f518800a840 con 0x7f51981540f0 2024-04-26T22:09:48.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.024+0000 7f5186ffd640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f517806c940 0x7f517806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.024+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7f518808bda0 con 0x7f51981540f0 2024-04-26T22:09:48.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.025+0000 7f519659c640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f517806c940 0x7f517806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:48.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.025+0000 7f519659c640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f517806c940 0x7f517806ee00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f518c0096c0 tx=0x7f518c006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:48.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.032+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5188059a70 con 0x7f51981540f0 2024-04-26T22:09:48.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.468+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f517c005580 con 0x7f51981540f0 2024-04-26T22:09:48.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.472+0000 7f5186ffd640 1 -- 172.21.15.135:0/599619816 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7f5188059370 con 0x7f51981540f0 2024-04-26T22:09:48.475 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:48.475 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:48.475 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:48.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f517806c940 msgr2=0x7f517806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:48.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f517806c940 0x7f517806ee00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f518c0096c0 tx=0x7f518c006bd0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 msgr2=0x7f519814a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 0x7f519814a2a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f518800c750 tx=0x7f518800cc10 comp rx=0 tx=0).stop 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 shutdown_connections 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f517806c940 0x7f517806ee00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5198157680 0x7f51981f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51981540f0 0x7f519814a2a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 --2- 172.21.15.135:0/599619816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51980ac610 0x7f5198147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.476+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 >> 172.21.15.135:0/599619816 conn(0x7f519806c310 msgr2=0x7f51980c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:48.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.478+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 shutdown_connections 2024-04-26T22:09:48.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.478+0000 7f519c8da640 1 -- 172.21.15.135:0/599619816 wait complete. 2024-04-26T22:09:48.501 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:09:48.552 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:48.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.712+0000 7fb915c6d640 1 Processor -- start 2024-04-26T22:09:48.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.712+0000 7fb915c6d640 1 -- start start 2024-04-26T22:09:48.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.713+0000 7fb915c6d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb910158c20 0x7fb91015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.713+0000 7fb915c6d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9100cbde0 con 0x7fb9100c90c0 2024-04-26T22:09:48.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.713+0000 7fb915c6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9100cc0c0 con 0x7fb9100bfbb0 2024-04-26T22:09:48.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.713+0000 7fb915c6d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb91015c150 con 0x7fb910158c20 2024-04-26T22:09:48.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.714+0000 7fb90f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb9100c90c0 0x7fb9100cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38056/0 (socket says 172.21.15.135:38056) 2024-04-26T22:09:48.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.714+0000 7fb90f7fe640 1 -- 172.21.15.135:0/1269616231 learned_addr learned my addr 172.21.15.135:0/1269616231 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:48.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.714+0000 7fb90e7fc640 1 -- 172.21.15.135:0/1269616231 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935323946 0 0) 0x7fb9100cbde0 con 0x7fb9100c90c0 2024-04-26T22:09:48.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.714+0000 7fb90e7fc640 1 -- 172.21.15.135:0/1269616231 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8f8003520 con 0x7fb9100c90c0 2024-04-26T22:09:48.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.715+0000 7fb90ffff640 1 --2- 172.21.15.135:0/1269616231 >> v2:172.21.15.139:3300/0 conn(0x7fb910158c20 0x7fb91015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:48.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.715+0000 7fb90ffff640 1 -- 172.21.15.135:0/1269616231 >> v1:172.21.15.139:6789/0 conn(0x7fb9100bfbb0 legacy=0x7fb910158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:48.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.715+0000 7fb90ffff640 1 -- 172.21.15.135:0/1269616231 >> v1:172.21.15.135:6789/0 conn(0x7fb9100c90c0 legacy=0x7fb9100cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:48.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.715+0000 7fb90ffff640 1 -- 172.21.15.135:0/1269616231 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9100ca5d0 con 0x7fb910158c20 2024-04-26T22:09:48.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.715+0000 7fb90e7fc640 1 -- 172.21.15.135:0/1269616231 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1236154578 0 0) 0x7fb8f8003520 con 0x7fb9100c90c0 2024-04-26T22:09:48.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.718+0000 7fb90ffff640 1 --2- 172.21.15.135:0/1269616231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 0x7fb91015bc10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb9000097a0 tx=0x7fb90002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3af36073956c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:48.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.718+0000 7fb90e7fc640 1 -- 172.21.15.135:0/1269616231 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900037070 con 0x7fb910158c20 2024-04-26T22:09:48.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.718+0000 7fb90e7fc640 1 -- 172.21.15.135:0/1269616231 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90002aad0 con 0x7fb910158c20 2024-04-26T22:09:48.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.718+0000 7fb90e7fc640 1 -- 172.21.15.135:0/1269616231 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080019f0 con 0x7fb910158c20 2024-04-26T22:09:48.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.720+0000 7fb915c6d640 1 -- 172.21.15.135:0/1269616231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 msgr2=0x7fb91015bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:48.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.720+0000 7fb915c6d640 1 --2- 172.21.15.135:0/1269616231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 0x7fb91015bc10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb9000097a0 tx=0x7fb90002a040 comp rx=0 tx=0).stop 2024-04-26T22:09:48.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.720+0000 7fb915c6d640 1 -- 172.21.15.135:0/1269616231 shutdown_connections 2024-04-26T22:09:48.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.720+0000 7fb915c6d640 1 --2- 172.21.15.135:0/1269616231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 0x7fb91015bc10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.720+0000 7fb915c6d640 1 -- 172.21.15.135:0/1269616231 >> 172.21.15.135:0/1269616231 conn(0x7fb9100ae010 msgr2=0x7fb9100ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:48.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.724+0000 7fb915c6d640 1 -- 172.21.15.135:0/1269616231 shutdown_connections 2024-04-26T22:09:48.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.725+0000 7fb915c6d640 1 -- 172.21.15.135:0/1269616231 wait complete. 2024-04-26T22:09:48.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.726+0000 7fb915c6d640 1 Processor -- start 2024-04-26T22:09:48.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.727+0000 7fb915c6d640 1 -- start start 2024-04-26T22:09:48.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.728+0000 7fb915c6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9100bfbb0 0x7fb91015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.728+0000 7fb915c6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 0x7fb910152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.728+0000 7fb915c6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 0x7fb91014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.728+0000 7fb915c6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9100c94a0 con 0x7fb9100c90c0 2024-04-26T22:09:48.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.728+0000 7fb915c6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9100ca380 con 0x7fb910158c20 2024-04-26T22:09:48.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.728+0000 7fb915c6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb9100c9620 con 0x7fb9100bfbb0 2024-04-26T22:09:48.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.729+0000 7fb90effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 0x7fb910152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:48.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.729+0000 7fb90effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 0x7fb910152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48602/0 (socket says 172.21.15.135:48602) 2024-04-26T22:09:48.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.729+0000 7fb90effd640 1 -- 172.21.15.135:0/4294025586 learned_addr learned my addr 172.21.15.135:0/4294025586 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:48.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.730+0000 7fb90effd640 1 -- 172.21.15.135:0/4294025586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9100bfbb0 msgr2=0x7fb91015f0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:09:48.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.730+0000 7fb90effd640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9100bfbb0 0x7fb91015f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.730+0000 7fb90effd640 1 -- 172.21.15.135:0/4294025586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 msgr2=0x7fb91014d8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.730+0000 7fb90effd640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 0x7fb91014d8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.730+0000 7fb90effd640 1 -- 172.21.15.135:0/4294025586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91015abc0 con 0x7fb9100c90c0 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.730+0000 7fb90effd640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 0x7fb910152ff0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c410 tx=0x7fb90400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.731+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904017070 con 0x7fb9100c90c0 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.731+0000 7fb915c6d640 1 -- 172.21.15.135:0/4294025586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91015aea0 con 0x7fb9100c90c0 2024-04-26T22:09:48.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.731+0000 7fb915c6d640 1 -- 172.21.15.135:0/4294025586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91015c8c0 con 0x7fb9100c90c0 2024-04-26T22:09:48.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.735+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9040035f0 con 0x7fb9100c90c0 2024-04-26T22:09:48.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.735+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904003a20 con 0x7fb9100c90c0 2024-04-26T22:09:48.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.735+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb90401a070 con 0x7fb9100c90c0 2024-04-26T22:09:48.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.736+0000 7fb8f7fff640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8fc06c9f0 0x7fb8fc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:48.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.736+0000 7fb90f7fe640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8fc06c9f0 0x7fb8fc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:48.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.737+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7fb90408b890 con 0x7fb9100c90c0 2024-04-26T22:09:48.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.737+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f0005290 con 0x7fb9100c90c0 2024-04-26T22:09:48.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.741+0000 7fb90f7fe640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8fc06c9f0 0x7fb8fc06eeb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb908005c00 tx=0x7fb908001380 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:48.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:48.746+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb904059560 con 0x7fb9100c90c0 2024-04-26T22:09:49.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.193+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8f0005580 con 0x7fb9100c90c0 2024-04-26T22:09:49.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.195+0000 7fb8f7fff640 1 -- 172.21.15.135:0/4294025586 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7fb904058e60 con 0x7fb9100c90c0 2024-04-26T22:09:49.195 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:49.195 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:49.195 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.199+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8fc06c9f0 msgr2=0x7fb8fc06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.199+0000 7fb8f5ffb640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8fc06c9f0 0x7fb8fc06eeb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb908005c00 tx=0x7fb908001380 comp rx=0 tx=0).stop 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.199+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 msgr2=0x7fb910152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.199+0000 7fb8f5ffb640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 0x7fb910152ff0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c410 tx=0x7fb90400c8d0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 shutdown_connections 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb8fc06c9f0 0x7fb8fc06eeb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb910158c20 0x7fb91014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9100c90c0 0x7fb910152ff0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 --2- 172.21.15.135:0/4294025586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9100bfbb0 0x7fb91015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 >> 172.21.15.135:0/4294025586 conn(0x7fb9100ae010 msgr2=0x7fb9100ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:49.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 shutdown_connections 2024-04-26T22:09:49.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.202+0000 7fb8f5ffb640 1 -- 172.21.15.135:0/4294025586 wait complete. 2024-04-26T22:09:49.225 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-04-26T22:09:49.225 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:09:49.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:09:49.241 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:09:49.242 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:49.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.458+0000 7fa1b170a640 1 Processor -- start 2024-04-26T22:09:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.459+0000 7fa1b170a640 1 -- start start 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.460+0000 7fa1b170a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa1ac0c4bb0 0x7fa1ac15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.460+0000 7fa1b170a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0cad50 0x7fa1ac0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.460+0000 7fa1b170a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ac06f9c0 con 0x7fa1ac15eb20 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.460+0000 7fa1b170a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ac0c4720 con 0x7fa1ac0c4bb0 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.460+0000 7fa1b170a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ac06fca0 con 0x7fa1ac0cad50 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.461+0000 7fa1aaffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa1ac15eb20 0x7fa1ac06ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38068/0 (socket says 172.21.15.135:38068) 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.461+0000 7fa1aaffd640 1 -- 172.21.15.135:0/3479942167 learned_addr learned my addr 172.21.15.135:0/3479942167 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.461+0000 7fa1aa7fc640 1 --2- 172.21.15.135:0/3479942167 >> v2:172.21.15.139:3300/0 conn(0x7fa1ac0c4bb0 0x7fa1ac15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.462+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0cad50 msgr2=0x7fa1ac0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.462+0000 7fa1ab7fe640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0cad50 0x7fa1ac0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.462+0000 7fa1aa7fc640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0cad50 0x7fa1ac0cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.462+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/3479942167 >> v1:172.21.15.135:6789/0 conn(0x7fa1ac15eb20 legacy=0x7fa1ac06ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.462+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/3479942167 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac0c5a10 con 0x7fa1ac0c4bb0 2024-04-26T22:09:49.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.463+0000 7fa1ab7fe640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0cad50 0x7fa1ac0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:09:49.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.463+0000 7fa1aa7fc640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0c4bb0 0x7fa1ac15ef00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009d90 tx=0x7fa1a403b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee2240c45d28b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:49.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.463+0000 7fa1a9ffb640 1 -- 172.21.15.135:0/3479942167 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4007740 con 0x7fa1ac0c4bb0 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.463+0000 7fa1a9ffb640 1 -- 172.21.15.135:0/3479942167 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40078e0 con 0x7fa1ac0c4bb0 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.463+0000 7fa1a9ffb640 1 -- 172.21.15.135:0/3479942167 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4007cb0 con 0x7fa1ac0c4bb0 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.464+0000 7fa1b170a640 1 -- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0c4bb0 msgr2=0x7fa1ac15ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.464+0000 7fa1b170a640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0c4bb0 0x7fa1ac15ef00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009d90 tx=0x7fa1a403b580 comp rx=0 tx=0).stop 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.465+0000 7fa1b170a640 1 -- 172.21.15.135:0/3479942167 shutdown_connections 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.465+0000 7fa1b170a640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0cad50 0x7fa1ac0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.465+0000 7fa1b170a640 1 --2- 172.21.15.135:0/3479942167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0c4bb0 0x7fa1ac15ef00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.465+0000 7fa1b170a640 1 -- 172.21.15.135:0/3479942167 >> 172.21.15.135:0/3479942167 conn(0x7fa1ac0756f0 msgr2=0x7fa1ac075ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:49.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.465+0000 7fa1b170a640 1 -- 172.21.15.135:0/3479942167 shutdown_connections 2024-04-26T22:09:49.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.465+0000 7fa1b170a640 1 -- 172.21.15.135:0/3479942167 wait complete. 2024-04-26T22:09:49.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.467+0000 7fa1b170a640 1 Processor -- start 2024-04-26T22:09:49.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.467+0000 7fa1b170a640 1 -- start start 2024-04-26T22:09:49.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.468+0000 7fa1b170a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0c4bb0 0x7fa1ac074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:49.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.468+0000 7fa1b170a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 0x7fa1ac157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:49.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1b170a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ac15eb20 0x7fa1ac0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:49.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1aa7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 0x7fa1ac157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:49.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1aa7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 0x7fa1ac157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40214/0 (socket says 172.21.15.135:40214) 2024-04-26T22:09:49.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/534408305 learned_addr learned my addr 172.21.15.135:0/534408305 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:49.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1ab7fe640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ac15eb20 0x7fa1ac0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:49.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac0cbb30 con 0x7fa1ac15eb20 2024-04-26T22:09:49.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac15f650 con 0x7fa1ac0cad50 2024-04-26T22:09:49.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.469+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ac0cbcb0 con 0x7fa1ac0c4bb0 2024-04-26T22:09:49.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.470+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0c4bb0 msgr2=0x7fa1ac074260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:09:49.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.470+0000 7fa1aa7fc640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0c4bb0 0x7fa1ac074260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.470+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ac15eb20 msgr2=0x7fa1ac0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.470+0000 7fa1aa7fc640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ac15eb20 0x7fa1ac0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.470+0000 7fa1aa7fc640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac0ad010 con 0x7fa1ac0cad50 2024-04-26T22:09:49.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.471+0000 7fa1aa7fc640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 0x7fa1ac157af0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009d90 tx=0x7fa1a4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:49.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.471+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4051070 con 0x7fa1ac0cad50 2024-04-26T22:09:49.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.471+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a4056460 con 0x7fa1ac0cad50 2024-04-26T22:09:49.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.472+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4009450 con 0x7fa1ac0cad50 2024-04-26T22:09:49.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.474+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac073780 con 0x7fa1ac0cad50 2024-04-26T22:09:49.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.475+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac158030 con 0x7fa1ac0cad50 2024-04-26T22:09:49.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.476+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac0735a0 con 0x7fa1ac0cad50 2024-04-26T22:09:49.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.480+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1a404a040 con 0x7fa1ac0cad50 2024-04-26T22:09:49.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.480+0000 7fa1937fe640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa19806c9c0 0x7fa19806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:49.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.481+0000 7fa1aaffd640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa19806c9c0 0x7fa19806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:49.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.482+0000 7fa1aaffd640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa19806c9c0 0x7fa19806ee80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa19c007fd0 tx=0x7fa19c0079a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:49.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.482+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7fa1a40bf4e0 con 0x7fa1ac0cad50 2024-04-26T22:09:49.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.487+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1ac0735a0 con 0x7fa1ac0cad50 2024-04-26T22:09:49.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.928+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1ac0696c0 con 0x7fa1ac0cad50 2024-04-26T22:09:49.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.929+0000 7fa1937fe640 1 -- 172.21.15.135:0/534408305 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7fa1a4064020 con 0x7fa1ac0cad50 2024-04-26T22:09:49.929 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:49.930 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:49.930 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:49.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.932+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa19806c9c0 msgr2=0x7fa19806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.932+0000 7fa1b170a640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa19806c9c0 0x7fa19806ee80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa19c007fd0 tx=0x7fa19c0079a0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 msgr2=0x7fa1ac157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 0x7fa1ac157af0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009d90 tx=0x7fa1a4004a40 comp rx=0 tx=0).stop 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 shutdown_connections 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa19806c9c0 0x7fa19806ee80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ac15eb20 0x7fa1ac0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1ac0cad50 0x7fa1ac157af0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 --2- 172.21.15.135:0/534408305 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ac0c4bb0 0x7fa1ac074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:49.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.933+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 >> 172.21.15.135:0/534408305 conn(0x7fa1ac0756f0 msgr2=0x7fa1ac15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:49.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.935+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 shutdown_connections 2024-04-26T22:09:49.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:49.935+0000 7fa1b170a640 1 -- 172.21.15.135:0/534408305 wait complete. 2024-04-26T22:09:49.958 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:09:50.502 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:50.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.672+0000 7fa0dd083640 1 Processor -- start 2024-04-26T22:09:50.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.672+0000 7fa0dd083640 1 -- start start 2024-04-26T22:09:50.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0dd083640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa0d806fcc0 0x7fa0d80c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:50.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0dd083640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d80ac490 0x7fa0d80c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:50.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0dd083640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d80c4210 con 0x7fa0d80718e0 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0dd083640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d80700a0 con 0x7fa0d806fcc0 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0dd083640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d806f830 con 0x7fa0d80ac490 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0d6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa0d806fcc0 0x7fa0d80c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0d6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa0d806fcc0 0x7fa0d80c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48620/0 (socket says 172.21.15.135:48620) 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.673+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/363673995 learned_addr learned my addr 172.21.15.135:0/363673995 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.674+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/363673995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d80ac490 msgr2=0x7fa0d80c35d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.674+0000 7fa0d6d9d640 1 --2- 172.21.15.135:0/363673995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d80ac490 0x7fa0d80c35d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:50.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.674+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/363673995 >> v1:172.21.15.139:6789/0 conn(0x7fa0d80718e0 legacy=0x7fa0d81674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:50.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.674+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/363673995 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d806ef00 con 0x7fa0d806fcc0 2024-04-26T22:09:50.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.675+0000 7fa0d6d9d640 1 --2- 172.21.15.135:0/363673995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d806fcc0 0x7fa0d80c4630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009870 tx=0x7fa0cc02c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa56d175aa8000f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:50.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.676+0000 7fa0d5d9b640 1 -- 172.21.15.135:0/363673995 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc038070 con 0x7fa0d806fcc0 2024-04-26T22:09:50.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.676+0000 7fa0d5d9b640 1 -- 172.21.15.135:0/363673995 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0cc031710 con 0x7fa0d806fcc0 2024-04-26T22:09:50.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.676+0000 7fa0d5d9b640 1 -- 172.21.15.135:0/363673995 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc031ae0 con 0x7fa0d806fcc0 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.678+0000 7fa0dd083640 1 -- 172.21.15.135:0/363673995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d806fcc0 msgr2=0x7fa0d80c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.678+0000 7fa0dd083640 1 --2- 172.21.15.135:0/363673995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d806fcc0 0x7fa0d80c4630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009870 tx=0x7fa0cc02c440 comp rx=0 tx=0).stop 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.679+0000 7fa0dd083640 1 -- 172.21.15.135:0/363673995 shutdown_connections 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.679+0000 7fa0dd083640 1 --2- 172.21.15.135:0/363673995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d80ac490 0x7fa0d80c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.679+0000 7fa0dd083640 1 --2- 172.21.15.135:0/363673995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d806fcc0 0x7fa0d80c4630 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.679+0000 7fa0dd083640 1 -- 172.21.15.135:0/363673995 >> 172.21.15.135:0/363673995 conn(0x7fa0d80c82d0 msgr2=0x7fa0d80c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.680+0000 7fa0dd083640 1 -- 172.21.15.135:0/363673995 shutdown_connections 2024-04-26T22:09:50.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.680+0000 7fa0dd083640 1 -- 172.21.15.135:0/363673995 wait complete. 2024-04-26T22:09:50.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.682+0000 7fa0dd083640 1 Processor -- start 2024-04-26T22:09:50.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.682+0000 7fa0dd083640 1 -- start start 2024-04-26T22:09:50.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.683+0000 7fa0dd083640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d806fcc0 0x7fa0d8073100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:50.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.683+0000 7fa0dd083640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 0x7fa0d80bd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:50.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0d659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 0x7fa0d80bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:50.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0d659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 0x7fa0d80bd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48626/0 (socket says 172.21.15.135:48626) 2024-04-26T22:09:50.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0d6d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d806fcc0 0x7fa0d8073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:50.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0d659c640 1 -- 172.21.15.135:0/1261550850 learned_addr learned my addr 172.21.15.135:0/1261550850 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:50.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0d80ac490 0x7fa0d80bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:50.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d80c3b10 con 0x7fa0d80718e0 2024-04-26T22:09:50.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d806f4c0 con 0x7fa0d80ac490 2024-04-26T22:09:50.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.684+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d8167be0 con 0x7fa0d806fcc0 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.685+0000 7fa0d759e640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0d80ac490 0x7fa0d80bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0d80ac490 msgr2=0x7fa0d80bdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d6d9d640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0d80ac490 0x7fa0d80bdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 msgr2=0x7fa0d80bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d6d9d640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 0x7fa0d80bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d6d9d640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d8202b00 con 0x7fa0d806fcc0 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d659c640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 0x7fa0d80bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:09:50.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.686+0000 7fa0d6d9d640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d806fcc0 0x7fa0d8073100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc033000 tx=0x7fa0cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:50.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.688+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc042020 con 0x7fa0d806fcc0 2024-04-26T22:09:50.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.688+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0cc033a30 con 0x7fa0d806fcc0 2024-04-26T22:09:50.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.688+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc038040 con 0x7fa0d806fcc0 2024-04-26T22:09:50.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.689+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d80be6a0 con 0x7fa0d806fcc0 2024-04-26T22:09:50.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.689+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d8073640 con 0x7fa0d806fcc0 2024-04-26T22:09:50.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.691+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d80ac960 con 0x7fa0d806fcc0 2024-04-26T22:09:50.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.693+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0cc045020 con 0x7fa0d806fcc0 2024-04-26T22:09:50.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.694+0000 7fa0bf7fe640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa0c406c9c0 0x7fa0c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:50.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.695+0000 7fa0d659c640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa0c406c9c0 0x7fa0c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:50.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.695+0000 7fa0d659c640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa0c406c9c0 0x7fa0c406ee80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80aaf60 tx=0x7fa0d0008490 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:50.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.696+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7fa0cc0ad0d0 con 0x7fa0d806fcc0 2024-04-26T22:09:50.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:50.702+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0d80ac960 con 0x7fa0d806fcc0 2024-04-26T22:09:51.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.162+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0d80507a0 con 0x7fa0d806fcc0 2024-04-26T22:09:51.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.164+0000 7fa0bf7fe640 1 -- 172.21.15.135:0/1261550850 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7fa0cc04c030 con 0x7fa0d806fcc0 2024-04-26T22:09:51.165 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:51.165 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:51.165 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:51.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.167+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa0c406c9c0 msgr2=0x7fa0c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.167+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa0c406c9c0 0x7fa0c406ee80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80aaf60 tx=0x7fa0d0008490 comp rx=0 tx=0).stop 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.167+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d806fcc0 msgr2=0x7fa0d8073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.167+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d806fcc0 0x7fa0d8073100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc033000 tx=0x7fa0cc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 shutdown_connections 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa0c406c9c0 0x7fa0c406ee80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0d80ac490 0x7fa0d80bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0d80718e0 0x7fa0d80bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 --2- 172.21.15.135:0/1261550850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0d806fcc0 0x7fa0d8073100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 >> 172.21.15.135:0/1261550850 conn(0x7fa0d80c82d0 msgr2=0x7fa0d80abc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 shutdown_connections 2024-04-26T22:09:51.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:51.168+0000 7fa0dd083640 1 -- 172.21.15.135:0/1261550850 wait complete. 2024-04-26T22:09:51.186 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-26T22:09:51.958 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:52.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.130+0000 7fe6e0ef2640 1 Processor -- start 2024-04-26T22:09:52.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.130+0000 7fe6e0ef2640 1 -- start start 2024-04-26T22:09:52.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.131+0000 7fe6e0ef2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe6dc1674d0 0x7fe6dc167920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.131+0000 7fe6e0ef2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6dc0c4a10 con 0x7fe6dc0ac390 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.131+0000 7fe6e0ef2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6dc06ef00 con 0x7fe6dc0c4630 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.131+0000 7fe6e0ef2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe6dc06f830 con 0x7fe6dc1674d0 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6dad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe6dc1674d0 0x7fe6dc167920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6dad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe6dc1674d0 0x7fe6dc167920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40230/0 (socket says 172.21.15.135:40230) 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6dad9d640 1 -- 172.21.15.135:0/1322977142 learned_addr learned my addr 172.21.15.135:0/1322977142 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6d959a640 1 -- 172.21.15.135:0/1322977142 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756660617 0 0) 0x7fe6dc0c4a10 con 0x7fe6dc0ac390 2024-04-26T22:09:52.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6d959a640 1 -- 172.21.15.135:0/1322977142 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c4003520 con 0x7fe6dc0ac390 2024-04-26T22:09:52.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6dad9d640 1 -- 172.21.15.135:0/1322977142 >> v1:172.21.15.139:6789/0 conn(0x7fe6dc0c4630 legacy=0x7fe6dc0718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:52.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6dad9d640 1 -- 172.21.15.135:0/1322977142 >> v1:172.21.15.135:6789/0 conn(0x7fe6dc0ac390 legacy=0x7fe6dc0c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:52.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.132+0000 7fe6dad9d640 1 -- 172.21.15.135:0/1322977142 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc06f1e0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.134+0000 7fe6dad9d640 1 --2- 172.21.15.135:0/1322977142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc167920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc007360 tx=0x7fe6cc02c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40e5b916504b271 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:52.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.138+0000 7fe6d959a640 1 -- 172.21.15.135:0/1322977142 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc008be0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.138+0000 7fe6d959a640 1 -- 172.21.15.135:0/1322977142 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc008d80 con 0x7fe6dc1674d0 2024-04-26T22:09:52.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.138+0000 7fe6d959a640 1 -- 172.21.15.135:0/1322977142 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc00a2f0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.139+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/1322977142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 msgr2=0x7fe6dc167920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:52.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.139+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/1322977142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc167920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc007360 tx=0x7fe6cc02c3c0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.139+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/1322977142 shutdown_connections 2024-04-26T22:09:52.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.139+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/1322977142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc167920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.139+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/1322977142 >> 172.21.15.135:0/1322977142 conn(0x7fe6dc0c82d0 msgr2=0x7fe6dc0c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:52.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.140+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/1322977142 shutdown_connections 2024-04-26T22:09:52.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.140+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/1322977142 wait complete. 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.142+0000 7fe6e0ef2640 1 Processor -- start 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.142+0000 7fe6e0ef2640 1 -- start start 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.143+0000 7fe6e0ef2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc0ac390 0x7fe6dc1f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.143+0000 7fe6e0ef2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6dc0c4630 0x7fe6dc0c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6e0ef2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc0732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6e0ef2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc071ff0 con 0x7fe6dc0ac390 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6e0ef2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc167e60 con 0x7fe6dc1674d0 2024-04-26T22:09:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6e0ef2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc0c3a70 con 0x7fe6dc0c4630 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6dc0c4630 0x7fe6dc0c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6dad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc0732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6dad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc0732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40234/0 (socket says 172.21.15.135:40234) 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.148+0000 7fe6dad9d640 1 -- 172.21.15.135:0/3993348109 learned_addr learned my addr 172.21.15.135:0/3993348109 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.150+0000 7fe6dad9d640 1 -- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6dc0c4630 msgr2=0x7fe6dc0c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.150+0000 7fe6dad9d640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6dc0c4630 0x7fe6dc0c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.150+0000 7fe6dad9d640 1 -- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc0ac390 msgr2=0x7fe6dc1f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:52.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.151+0000 7fe6da59c640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc0ac390 0x7fe6dc1f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:52.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.151+0000 7fe6dad9d640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc0ac390 0x7fe6dc1f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.151+0000 7fe6dad9d640 1 -- 172.21.15.135:0/3993348109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1f0760 con 0x7fe6dc1674d0 2024-04-26T22:09:52.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.151+0000 7fe6dad9d640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc0732f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc007480 tx=0x7fe6cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:52.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.151+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc040030 con 0x7fe6dc1674d0 2024-04-26T22:09:52.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.152+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc03c3f0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.153+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc03c7c0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.153+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc1f0a00 con 0x7fe6dc1674d0 2024-04-26T22:09:52.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.153+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1f0fc0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.156+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6cc043070 con 0x7fe6dc1674d0 2024-04-26T22:09:52.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.157+0000 7fe6caffd640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6bc06c8f0 0x7fe6bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:52.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.157+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 7698+0+0 (secure 0 0 0) 0x7fe6cc0abcb0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.158+0000 7fe6da59c640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6bc06c8f0 0x7fe6bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:52.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.158+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc0507a0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.161+0000 7fe6da59c640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6bc06c8f0 0x7fe6bc06edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4007fd0 tx=0x7fe6d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:52.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.167+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6cc079880 con 0x7fe6dc1674d0 2024-04-26T22:09:52.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.595+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6dc0700d0 con 0x7fe6dc1674d0 2024-04-26T22:09:52.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.596+0000 7fe6caffd640 1 -- 172.21.15.135:0/3993348109 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6499 (secure 0 0 0) 0x7fe6cc079180 con 0x7fe6dc1674d0 2024-04-26T22:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:07:41.896787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":8,"addr":"172.21.15.135:6837/3969385858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":3969385858},{"type":"v1","addr":"172.21.15.135:6837","nonce":3969385858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5210,"qdb_cluster":[5210]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:52.597 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 71 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6bc06c8f0 msgr2=0x7fe6bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6bc06c8f0 0x7fe6bc06edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4007fd0 tx=0x7fe6d4009040 comp rx=0 tx=0).stop 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 msgr2=0x7fe6dc0732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc0732f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc007480 tx=0x7fe6cc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 shutdown_connections 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe6bc06c8f0 0x7fe6bc06edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6dc1674d0 0x7fe6dc0732f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6dc0c4630 0x7fe6dc0c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 --2- 172.21.15.135:0/3993348109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6dc0ac390 0x7fe6dc1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 >> 172.21.15.135:0/3993348109 conn(0x7fe6dc0c82d0 msgr2=0x7fe6dc1ef3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 shutdown_connections 2024-04-26T22:09:52.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:52.600+0000 7fe6e0ef2640 1 -- 172.21.15.135:0/3993348109 wait complete. 2024-04-26T22:09:52.622 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:09:53.187 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:53.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.350+0000 7f5eddb52640 1 Processor -- start 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.350+0000 7f5eddb52640 1 -- start start 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5eddb52640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5ed00d3c40 0x7f5ed004ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5eddb52640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ed004b1b0 0x7f5ed00d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5eddb52640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d6fa0 0x7f5ed00d99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5eddb52640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed0047660 con 0x7f5ed00d3c40 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5eddb52640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed00d65d0 con 0x7f5ed004b1b0 2024-04-26T22:09:53.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5eddb52640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed00468d0 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5edcb50640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5ed00d3c40 0x7f5ed004ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5edd351640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d6fa0 0x7f5ed00d99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5edcb50640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5ed00d3c40 0x7f5ed004ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48646/0 (socket says 172.21.15.135:48646) 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5edcb50640 1 -- 172.21.15.135:0/704317141 learned_addr learned my addr 172.21.15.135:0/704317141 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.351+0000 7f5ed7fff640 1 --2- 172.21.15.135:0/704317141 >> v2:172.21.15.139:3300/0 conn(0x7f5ed004b1b0 0x7f5ed00d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.352+0000 7f5edd351640 1 -- 172.21.15.135:0/704317141 >> v2:172.21.15.139:3300/0 conn(0x7f5ed004b1b0 msgr2=0x7f5ed00d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.352+0000 7f5edd351640 1 --2- 172.21.15.135:0/704317141 >> v2:172.21.15.139:3300/0 conn(0x7f5ed004b1b0 0x7f5ed00d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.352+0000 7f5edd351640 1 -- 172.21.15.135:0/704317141 >> v2:172.21.15.135:3300/0 conn(0x7f5ed00d3c40 msgr2=0x7f5ed004ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:53.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.352+0000 7f5edd351640 1 --2- 172.21.15.135:0/704317141 >> v2:172.21.15.135:3300/0 conn(0x7f5ed00d3c40 0x7f5ed004ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.352+0000 7f5edd351640 1 -- 172.21.15.135:0/704317141 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed0046bb0 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.352+0000 7f5ed7fff640 1 --2- 172.21.15.135:0/704317141 >> v2:172.21.15.139:3300/0 conn(0x7f5ed004b1b0 0x7f5ed00d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:09:53.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.353+0000 7f5edd351640 1 --2- 172.21.15.135:0/704317141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d6fa0 0x7f5ed00d99a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009960 tx=0x7f5ec402e590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719183ce4a88191 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:53.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.353+0000 7f5ed77fe640 1 -- 172.21.15.135:0/704317141 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400e4e0 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.354+0000 7f5ed77fe640 1 -- 172.21.15.135:0/704317141 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec400c730 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.354+0000 7f5ed77fe640 1 -- 172.21.15.135:0/704317141 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400cb20 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 -- 172.21.15.135:0/704317141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d6fa0 msgr2=0x7f5ed00d99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 --2- 172.21.15.135:0/704317141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d6fa0 0x7f5ed00d99a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009960 tx=0x7f5ec402e590 comp rx=0 tx=0).stop 2024-04-26T22:09:53.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 -- 172.21.15.135:0/704317141 shutdown_connections 2024-04-26T22:09:53.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 --2- 172.21.15.135:0/704317141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d6fa0 0x7f5ed00d99a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 --2- 172.21.15.135:0/704317141 >> v2:172.21.15.139:3300/0 conn(0x7f5ed004b1b0 0x7f5ed00d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 --2- 172.21.15.135:0/704317141 >> v2:172.21.15.135:3300/0 conn(0x7f5ed00d3c40 0x7f5ed004ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.355+0000 7f5eddb52640 1 -- 172.21.15.135:0/704317141 >> 172.21.15.135:0/704317141 conn(0x7f5ed0005e90 msgr2=0x7f5ed00062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:53.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.357+0000 7f5eddb52640 1 -- 172.21.15.135:0/704317141 shutdown_connections 2024-04-26T22:09:53.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.357+0000 7f5eddb52640 1 -- 172.21.15.135:0/704317141 wait complete. 2024-04-26T22:09:53.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.359+0000 7f5eddb52640 1 Processor -- start 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.359+0000 7f5eddb52640 1 -- start start 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.360+0000 7f5eddb52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed004b1b0 0x7f5ed0049d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5eddb52640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d3c40 0x7f5ed00d8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5eddb52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 0x7f5ed00d88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5eddb52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0046360 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5eddb52640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00461e0 con 0x7f5ed004b1b0 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5eddb52640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed0046e90 con 0x7f5ed00d3c40 2024-04-26T22:09:53.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edd351640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 0x7f5ed00d88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edcb50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed004b1b0 0x7f5ed0049d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edd351640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 0x7f5ed00d88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48652/0 (socket says 172.21.15.135:48652) 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edd351640 1 -- 172.21.15.135:0/3215178858 learned_addr learned my addr 172.21.15.135:0/3215178858 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5ed7fff640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d3c40 0x7f5ed00d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edd351640 1 -- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d3c40 msgr2=0x7f5ed00d8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edd351640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d3c40 0x7f5ed00d8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.361+0000 7f5edd351640 1 -- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed004b1b0 msgr2=0x7f5ed0049d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.362+0000 7f5edd351640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed004b1b0 0x7f5ed0049d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.362+0000 7f5edd351640 1 -- 172.21.15.135:0/3215178858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed0177cb0 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.362+0000 7f5edcb50640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed004b1b0 0x7f5ed0049d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.362+0000 7f5edd351640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 0x7f5ed00d88c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402e5d0 tx=0x7f5ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.362+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4042030 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.362+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec400caf0 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.364 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.363+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4003610 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.363+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed0177f90 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.363+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed004a290 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.365+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed0005390 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.367+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ec4045070 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.368+0000 7f5ed57fa640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5ebc06c9c0 0x7f5ebc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.368+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7f5ec40ad0d0 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.368+0000 7f5edcb50640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5ebc06c9c0 0x7f5ebc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.369+0000 7f5edcb50640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5ebc06c9c0 0x7f5ebc06ee80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8051250 tx=0x7f5ed8090040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:53.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.374+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ec404c070 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.789+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ed000af10 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.790+0000 7f5ed57fa640 1 -- 172.21.15.135:0/3215178858 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7f5ec407a700 con 0x7f5ed00d6fa0 2024-04-26T22:09:53.791 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:53.791 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:53.791 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5ebc06c9c0 msgr2=0x7f5ebc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5ebc06c9c0 0x7f5ebc06ee80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8051250 tx=0x7f5ed8090040 comp rx=0 tx=0).stop 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 msgr2=0x7f5ed00d88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 0x7f5ed00d88c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402e5d0 tx=0x7f5ec40046c0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 shutdown_connections 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f5ebc06c9c0 0x7f5ebc06ee80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed00d6fa0 0x7f5ed00d88c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed00d3c40 0x7f5ed00d8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 --2- 172.21.15.135:0/3215178858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed004b1b0 0x7f5ed0049d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.794+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 >> 172.21.15.135:0/3215178858 conn(0x7f5ed0005e90 msgr2=0x7f5ed002c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:53.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.795+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 shutdown_connections 2024-04-26T22:09:53.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.795+0000 7f5eddb52640 1 -- 172.21.15.135:0/3215178858 wait complete. 2024-04-26T22:09:53.813 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-04-26T22:09:53.813 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-04-26T22:09:53.813 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:53.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.969+0000 7f1ec7809640 1 Processor -- start 2024-04-26T22:09:53.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.969+0000 7f1ec7809640 1 -- start start 2024-04-26T22:09:53.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.970+0000 7f1ec7809640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.970+0000 7f1ec7809640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec00c4b50 con 0x7f1ec006d860 2024-04-26T22:09:53.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.970+0000 7f1ec7809640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec01514a0 con 0x7f1ec01510c0 2024-04-26T22:09:53.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.970+0000 7f1ec7809640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00c50d0 con 0x7f1ec00c4160 2024-04-26T22:09:53.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1ec55a5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1ec01510c0 0x7f1ec0072f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:55152/0 (socket says 172.21.15.135:55152) 2024-04-26T22:09:53.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1ec55a5640 1 -- 172.21.15.135:0/1008487819 learned_addr learned my addr 172.21.15.135:0/1008487819 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:53.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1ec4da4640 1 --2- 172.21.15.135:0/1008487819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1eb7fff640 1 -- 172.21.15.135:0/1008487819 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086327308 0 0) 0x7f1ec01514a0 con 0x7f1ec01510c0 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1eb7fff640 1 -- 172.21.15.135:0/1008487819 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1eac003520 con 0x7f1ec01510c0 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1ec4da4640 1 -- 172.21.15.135:0/1008487819 >> v1:172.21.15.139:6789/0 conn(0x7f1ec01510c0 legacy=0x7f1ec0072f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1ec4da4640 1 -- 172.21.15.135:0/1008487819 >> v1:172.21.15.135:6789/0 conn(0x7f1ec006d860 legacy=0x7f1ec015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.971+0000 7f1ec4da4640 1 -- 172.21.15.135:0/1008487819 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec00736a0 con 0x7f1ec00c4160 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.973+0000 7f1ec4da4640 1 --2- 172.21.15.135:0/1008487819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec006d320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800bfd0 tx=0x7f1eb802c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3eee95fc4b9150 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.973+0000 7f1eb7fff640 1 -- 172.21.15.135:0/1008487819 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8038070 con 0x7f1ec00c4160 2024-04-26T22:09:53.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.973+0000 7f1eb7fff640 1 -- 172.21.15.135:0/1008487819 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb8009540 con 0x7f1ec00c4160 2024-04-26T22:09:53.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.974+0000 7f1ec7809640 1 -- 172.21.15.135:0/1008487819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 msgr2=0x7f1ec006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.974+0000 7f1ec7809640 1 --2- 172.21.15.135:0/1008487819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec006d320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800bfd0 tx=0x7f1eb802c7c0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.975+0000 7f1ec7809640 1 -- 172.21.15.135:0/1008487819 shutdown_connections 2024-04-26T22:09:53.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.975+0000 7f1ec7809640 1 --2- 172.21.15.135:0/1008487819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec006d320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.975+0000 7f1ec7809640 1 -- 172.21.15.135:0/1008487819 >> 172.21.15.135:0/1008487819 conn(0x7f1ec00c5ce0 msgr2=0x7f1ec00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:53.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.975+0000 7f1ec7809640 1 -- 172.21.15.135:0/1008487819 shutdown_connections 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.976+0000 7f1ec7809640 1 -- 172.21.15.135:0/1008487819 wait complete. 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.977+0000 7f1ec7809640 1 Processor -- start 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.977+0000 7f1ec7809640 1 -- start start 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.978+0000 7f1ec7809640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec006d860 0x7f1ec014c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.978+0000 7f1ec7809640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec0135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.978+0000 7f1ec7809640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ec01510c0 0x7f1ec013ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.978+0000 7f1ec7809640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00c5400 con 0x7f1ec006d860 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.978+0000 7f1ec7809640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00c4df0 con 0x7f1ec01510c0 2024-04-26T22:09:53.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.978+0000 7f1ec7809640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00c5920 con 0x7f1ec00c4160 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec5da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ec01510c0 0x7f1ec013ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec5da6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ec01510c0 0x7f1ec013ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40264/0 (socket says 172.21.15.135:40264) 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec5da6640 1 -- 172.21.15.135:0/3542609354 learned_addr learned my addr 172.21.15.135:0/3542609354 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec4da4640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec0135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec4da4640 1 -- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ec01510c0 msgr2=0x7f1ec013ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec4da4640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ec01510c0 0x7f1ec013ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec4da4640 1 -- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec006d860 msgr2=0x7f1ec014c830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec4da4640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec006d860 0x7f1ec014c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.979+0000 7f1ec4da4640 1 -- 172.21.15.135:0/3542609354 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec01ed0b0 con 0x7f1ec00c4160 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.980+0000 7f1ec4da4640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec0135490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8032000 tx=0x7f1eb8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:53.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.980+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8038040 con 0x7f1ec00c4160 2024-04-26T22:09:53.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.980+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec01ecdd0 con 0x7f1ec00c4160 2024-04-26T22:09:53.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.980+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01529a0 con 0x7f1ec00c4160 2024-04-26T22:09:53.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.981+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb80092b0 con 0x7f1ec00c4160 2024-04-26T22:09:53.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.981+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8033c30 con 0x7f1ec00c4160 2024-04-26T22:09:53.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.983+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1eb8009610 con 0x7f1ec00c4160 2024-04-26T22:09:53.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.984+0000 7f1eb5ffb640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1ea406c8f0 0x7f1ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:53.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.984+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7f1eb80acee0 con 0x7f1ec00c4160 2024-04-26T22:09:53.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.984+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea0005290 con 0x7f1ec00c4160 2024-04-26T22:09:53.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.985+0000 7f1ec55a5640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1ea406c8f0 0x7f1ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:53.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.986+0000 7f1ec55a5640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1ea406c8f0 0x7f1ea406edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00c4a00 tx=0x7f1ebc0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:53.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:53.993+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1eb804c020 con 0x7f1ec00c4160 2024-04-26T22:09:54.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.400+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea0005580 con 0x7f1ec00c4160 2024-04-26T22:09:54.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.401+0000 7f1eb5ffb640 1 -- 172.21.15.135:0/3542609354 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7f1eb807aae0 con 0x7f1ec00c4160 2024-04-26T22:09:54.401 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:54.401 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:54.401 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1ea406c8f0 msgr2=0x7f1ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1ea406c8f0 0x7f1ea406edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00c4a00 tx=0x7f1ebc0078f0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 msgr2=0x7f1ec0135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec0135490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8032000 tx=0x7f1eb8004a40 comp rx=0 tx=0).stop 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 shutdown_connections 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f1ea406c8f0 0x7f1ea406edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ec01510c0 0x7f1ec013ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ec00c4160 0x7f1ec0135490 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8032000 tx=0x7f1eb8004a40 comp rx=0 tx=0).stop 2024-04-26T22:09:54.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 --2- 172.21.15.135:0/3542609354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec006d860 0x7f1ec014c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 >> 172.21.15.135:0/3542609354 conn(0x7f1ec00c5ce0 msgr2=0x7f1ec0072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:54.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 shutdown_connections 2024-04-26T22:09:54.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.404+0000 7f1ec7809640 1 -- 172.21.15.135:0/3542609354 wait complete. 2024-04-26T22:09:54.421 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 72, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 72, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5653": { "addr": "172.21.15.139:6833/112034263", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 112034263, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 112034263, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5653, "incarnation": 72, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:09:52.864862+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5653 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 70, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5970": { "addr": "172.21.15.135:6835/3800113814", "addrs": { "addrvec": [ { "addr": "172.21.15.135:6834", "nonce": 3800113814, "type": "v2" }, { "addr": "172.21.15.135:6835", "nonce": 3800113814, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5970, "incarnation": 67, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:09:05.625009+0000", "qdb_cluster": [ 5970 ], "qdb_leader": 5970, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5970 } } } ], "standbys": [ { "addr": "172.21.15.139:6835/1717329133", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 1717329133, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 1717329133, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 65, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5291, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-26T22:09:54.623 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:54.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.766+0000 7f2c10a23640 1 Processor -- start 2024-04-26T22:09:54.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.766+0000 7f2c10a23640 1 -- start start 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c10a23640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2c04050720 0x7f2c040062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c10a23640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2c0400fcf0 0x7f2c04006800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c10a23640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c040d7d00 con 0x7f2c040d7920 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2c040d7920 0x7f2c0400edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:55154/0 (socket says 172.21.15.135:55154) 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0b7fe640 1 -- 172.21.15.135:0/3363591613 learned_addr learned my addr 172.21.15.135:0/3363591613 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0affd640 1 --2- 172.21.15.135:0/3363591613 >> v2:172.21.15.135:3300/0 conn(0x7f2c04050720 0x7f2c040062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:54.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2c040db250 con 0x7f2c04050720 2024-04-26T22:09:54.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2c0404ba00 con 0x7f2c0400fcf0 2024-04-26T22:09:54.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0bfff640 1 --2- 172.21.15.135:0/3363591613 >> v2:172.21.15.139:3300/0 conn(0x7f2c0400fcf0 0x7f2c04006800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:54.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0affd640 1 -- 172.21.15.135:0/3363591613 >> v2:172.21.15.139:3300/0 conn(0x7f2c0400fcf0 msgr2=0x7f2c04006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:54.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0affd640 1 --2- 172.21.15.135:0/3363591613 >> v2:172.21.15.139:3300/0 conn(0x7f2c0400fcf0 0x7f2c04006800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0affd640 1 -- 172.21.15.135:0/3363591613 >> v1:172.21.15.139:6789/0 conn(0x7f2c040d7920 legacy=0x7f2c0400edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:54.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.767+0000 7f2c0affd640 1 -- 172.21.15.135:0/3363591613 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0404bce0 con 0x7f2c04050720 2024-04-26T22:09:54.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.768+0000 7f2c0bfff640 1 --2- 172.21.15.135:0/3363591613 >> v2:172.21.15.139:3300/0 conn(0x7f2c0400fcf0 0x7f2c04006800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:09:54.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.769+0000 7f2c0affd640 1 --2- 172.21.15.135:0/3363591613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c04050720 0x7f2c040062c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2c040da2d0 tx=0x7f2bf802a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7f7a578475d947 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:54.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.769+0000 7f2c0a7fc640 1 -- 172.21.15.135:0/3363591613 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8037070 con 0x7f2c04050720 2024-04-26T22:09:54.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.769+0000 7f2c0a7fc640 1 -- 172.21.15.135:0/3363591613 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf802aca0 con 0x7f2c04050720 2024-04-26T22:09:54.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.769+0000 7f2c0a7fc640 1 -- 172.21.15.135:0/3363591613 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8032660 con 0x7f2c04050720 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.770+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c04050720 msgr2=0x7f2c040062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.770+0000 7f2c10a23640 1 --2- 172.21.15.135:0/3363591613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c04050720 0x7f2c040062c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2c040da2d0 tx=0x7f2bf802a010 comp rx=0 tx=0).stop 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.770+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 shutdown_connections 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.770+0000 7f2c10a23640 1 --2- 172.21.15.135:0/3363591613 >> v2:172.21.15.139:3300/0 conn(0x7f2c0400fcf0 0x7f2c04006800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.770+0000 7f2c10a23640 1 --2- 172.21.15.135:0/3363591613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c04050720 0x7f2c040062c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.770+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 >> 172.21.15.135:0/3363591613 conn(0x7f2c0404fc00 msgr2=0x7f2c04050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:54.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.771+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 shutdown_connections 2024-04-26T22:09:54.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.771+0000 7f2c10a23640 1 -- 172.21.15.135:0/3363591613 wait complete. 2024-04-26T22:09:54.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.772+0000 7f2c10a23640 1 Processor -- start 2024-04-26T22:09:54.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.773+0000 7f2c10a23640 1 -- start start 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c10a23640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c0400fcf0 0x7f2c04171170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c10a23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 0x7f2c04173c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c10a23640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c040d7920 0x7f2c040331f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c0b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c0400fcf0 0x7f2c04171170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c0affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 0x7f2c04173c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c0b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c0400fcf0 0x7f2c04171170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:59940/0 (socket says 172.21.15.135:59940) 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c0affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 0x7f2c04173c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:40282/0 (socket says 172.21.15.135:40282) 2024-04-26T22:09:54.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c0bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c040d7920 0x7f2c040331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c0b7fe640 1 -- 172.21.15.135:0/347286704 learned_addr learned my addr 172.21.15.135:0/347286704 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0400f4c0 con 0x7f2c040d7920 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0404c2a0 con 0x7f2c04050720 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.774+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0400f640 con 0x7f2c0400fcf0 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0bfff640 1 -- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c0400fcf0 msgr2=0x7f2c04171170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0bfff640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c0400fcf0 0x7f2c04171170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0bfff640 1 -- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 msgr2=0x7f2c04173c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0bfff640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 0x7f2c04173c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0bfff640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0404f300 con 0x7f2c040d7920 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0affd640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 0x7f2c04173c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:54.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.775+0000 7f2c0bfff640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c040d7920 0x7f2c040331f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c075190 tx=0x7f2c0c06a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:54.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.776+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c06d920 con 0x7f2c040d7920 2024-04-26T22:09:54.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.776+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c091070 con 0x7f2c040d7920 2024-04-26T22:09:54.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.776+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c06dcc0 con 0x7f2c040d7920 2024-04-26T22:09:54.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.776+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c04033730 con 0x7f2c040d7920 2024-04-26T22:09:54.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.776+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0404f020 con 0x7f2c040d7920 2024-04-26T22:09:54.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.777+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c04005390 con 0x7f2c040d7920 2024-04-26T22:09:54.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.778+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c0c095410 con 0x7f2c040d7920 2024-04-26T22:09:54.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.779+0000 7f2bf7fff640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:54.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.779+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7f2c0c108b60 con 0x7f2c040d7920 2024-04-26T22:09:54.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.780+0000 7f2c0b7fe640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:54.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.780+0000 7f2c0b7fe640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000f60 tx=0x7f2c00007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:54.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:54.785+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c0c0d6080 con 0x7f2c040d7920 2024-04-26T22:09:55.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.174+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c04049ec0 con 0x7f2c040d7920 2024-04-26T22:09:55.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.174+0000 7f2bf7fff640 1 -- 172.21.15.135:0/347286704 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7f2c0c0da380 con 0x7f2c040d7920 2024-04-26T22:09:55.176 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:55.176 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:55.176 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 msgr2=0x7f2bf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000f60 tx=0x7f2c00007ec0 comp rx=0 tx=0).stop 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c040d7920 msgr2=0x7f2c040331f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c040d7920 0x7f2c040331f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c075190 tx=0x7f2c0c06a050 comp rx=0 tx=0).stop 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 shutdown_connections 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f2bf006c920 0x7f2bf006ede0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c040d7920 0x7f2c040331f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:55.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c04050720 0x7f2c04173c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:55.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 --2- 172.21.15.135:0/347286704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c0400fcf0 0x7f2c04171170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:55.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 >> 172.21.15.135:0/347286704 conn(0x7f2c0404fc00 msgr2=0x7f2c04030f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:55.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 shutdown_connections 2024-04-26T22:09:55.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:55.178+0000 7f2c10a23640 1 -- 172.21.15.135:0/347286704 wait complete. 2024-04-26T22:09:55.193 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:09:56.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:56.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 Processor -- start 2024-04-26T22:09:56.527 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 -- start start 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f978c06d590 0x7f978c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f978c0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f978c154e10 con 0x7f978c0aef70 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f978c0af400 con 0x7f978c06d590 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.527+0000 7f979186b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f978c1559e0 con 0x7f978c070120 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978b7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f978c0aef70 0x7f978c0c3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38090/0 (socket says 172.21.15.135:38090) 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978b7fe640 1 -- 172.21.15.135:0/3589309046 learned_addr learned my addr 172.21.15.135:0/3589309046 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f9783fff640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f978c0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978affd640 1 --2- 172.21.15.135:0/3589309046 >> v2:172.21.15.139:3300/0 conn(0x7f978c06d590 0x7f978c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309681015 0 0) 0x7f978c154e10 con 0x7f978c0aef70 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9778003520 con 0x7f978c0aef70 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 214314442 0 0) 0x7f9778003520 con 0x7f978c0aef70 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 msgr2=0x7f978c0aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f978c0aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 >> v2:172.21.15.139:3300/0 conn(0x7f978c06d590 msgr2=0x7f978c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 --2- 172.21.15.135:0/3589309046 >> v2:172.21.15.139:3300/0 conn(0x7f978c06d590 0x7f978c158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c159240 con 0x7f978c0aef70 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f97840019f0 con 0x7f978c0aef70 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 >> v1:172.21.15.135:6789/0 conn(0x7f978c0aef70 legacy=0x7f978c0c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97780039b0 0x7f9778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f9778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f97780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f978c1559e0 con 0x7f9778004800 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f977800b9c0 con 0x7f97780039b0 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f977800bb40 con 0x7f978c070120 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f9783fff640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f9778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978affd640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97780039b0 0x7f9778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.528+0000 7f978b7fe640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f97780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f9783fff640 1 -- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 msgr2=0x7f97780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f9783fff640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f97780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f9783fff640 1 -- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97780039b0 msgr2=0x7f9778003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:56.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f9783fff640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97780039b0 0x7f9778003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f9783fff640 1 -- 172.21.15.135:0/3589309046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9778003520 con 0x7f9778004800 2024-04-26T22:09:56.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f978b7fe640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c070120 0x7f97780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:09:56.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.529+0000 7f9783fff640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f9778004c80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9774009af0 tx=0x7f9778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdc6dfcba025b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:56.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.530+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774036070 con 0x7f9778004800 2024-04-26T22:09:56.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.530+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9774031400 con 0x7f9778004800 2024-04-26T22:09:56.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.530+0000 7f978a7fc640 1 -- 172.21.15.135:0/3589309046 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774009690 con 0x7f9778004800 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f979186b640 1 -- 172.21.15.135:0/3589309046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 msgr2=0x7f9778004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f979186b640 1 --2- 172.21.15.135:0/3589309046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f9778004c80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9774009af0 tx=0x7f9778005830 comp rx=0 tx=0).stop 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f978b7fe640 1 -- 172.21.15.135:0/3589309046 reap_dead start 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f979186b640 1 -- 172.21.15.135:0/3589309046 shutdown_connections 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f979186b640 1 -- 172.21.15.135:0/3589309046 >> 172.21.15.135:0/3589309046 conn(0x7f978c0c5ce0 msgr2=0x7f978c0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f979186b640 1 -- 172.21.15.135:0/3589309046 shutdown_connections 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.531+0000 7f979186b640 1 -- 172.21.15.135:0/3589309046 wait complete. 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 Processor -- start 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 -- start start 2024-04-26T22:09:56.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f978c0c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c1df820 0x7f978c1f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978c1de640 0x7f978c1ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f978c155670 con 0x7f9778004800 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f978c155390 con 0x7f978c1de640 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.532+0000 7f979186b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f978c06d970 con 0x7f978c1df820 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f978c0c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f978c0c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48670/0 (socket says 172.21.15.135:48670) 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 -- 172.21.15.135:0/2347443313 learned_addr learned my addr 172.21.15.135:0/2347443313 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f9783fff640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c1df820 0x7f978c1f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978b7fe640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978c1de640 0x7f978c1ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 -- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c1df820 msgr2=0x7f978c1f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c1df820 0x7f978c1f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 -- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978c1de640 msgr2=0x7f978c1ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978c1de640 0x7f978c1ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 -- 172.21.15.135:0/2347443313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c0c7e90 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978b7fe640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978c1de640 0x7f978c1ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f978affd640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f978c0c77e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f977c011aa0 tx=0x7f977c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c00fb80 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977c00fd20 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c0c8170 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c014660 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.533+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c1efb00 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.534+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c1ef920 con 0x7f9778004800 2024-04-26T22:09:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.535+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f977c012040 con 0x7f9778004800 2024-04-26T22:09:56.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.535+0000 7f9788ff9640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f976c06c920 0x7f976c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:56.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.535+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7f977c08c960 con 0x7f9778004800 2024-04-26T22:09:56.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.535+0000 7f9783fff640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f976c06c920 0x7f976c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:56.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.536+0000 7f9783fff640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f976c06c920 0x7f976c06ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9774033000 tx=0x7f9774008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:56.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.541+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f978c1ef920 con 0x7f9778004800 2024-04-26T22:09:56.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.783+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f978c0507a0 con 0x7f9778004800 2024-04-26T22:09:56.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.784+0000 7f9788ff9640 1 -- 172.21.15.135:0/2347443313 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7f977c019030 con 0x7f9778004800 2024-04-26T22:09:56.784 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:56.784 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:56.784 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:56.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f976c06c920 msgr2=0x7f976c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:56.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f976c06c920 0x7f976c06ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9774033000 tx=0x7f9774008030 comp rx=0 tx=0).stop 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 msgr2=0x7f978c0c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f978c0c77e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f977c011aa0 tx=0x7f977c011f60 comp rx=0 tx=0).stop 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 shutdown_connections 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f976c06c920 0x7f976c06ede0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978c1de640 0x7f978c1ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978c1df820 0x7f978c1f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 --2- 172.21.15.135:0/2347443313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9778004800 0x7f978c0c77e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 >> 172.21.15.135:0/2347443313 conn(0x7f978c0c5ce0 msgr2=0x7f978c1545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 shutdown_connections 2024-04-26T22:09:56.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:56.786+0000 7f979186b640 1 -- 172.21.15.135:0/2347443313 wait complete. 2024-04-26T22:09:56.799 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:09:57.195 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:57.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.300+0000 7fe60cc99640 1 Processor -- start 2024-04-26T22:09:57.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.300+0000 7fe60cc99640 1 -- start start 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.301+0000 7fe60cc99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe608071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.301+0000 7fe60cc99640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe60806cf20 0x7fe6080737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.301+0000 7fe60cc99640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6080733a0 con 0x7fe60814ec70 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.301+0000 7fe60cc99640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe608074540 con 0x7fe60806cf20 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.301+0000 7fe60cc99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6080c30f0 con 0x7fe608074ba0 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.301+0000 7fe60659c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe60814ec70 0x7fe60806e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38096/0 (socket says 172.21.15.135:38096) 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe60659c640 1 -- 172.21.15.135:0/2380173081 learned_addr learned my addr 172.21.15.135:0/2380173081 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe605d9b640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe608071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe606d9d640 1 --2- 172.21.15.135:0/2380173081 >> v2:172.21.15.139:3300/0 conn(0x7fe60806cf20 0x7fe6080737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe60559a640 1 -- 172.21.15.135:0/2380173081 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127133746 0 0) 0x7fe6080733a0 con 0x7fe60814ec70 2024-04-26T22:09:57.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe60559a640 1 -- 172.21.15.135:0/2380173081 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5f0003520 con 0x7fe60814ec70 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe606d9d640 1 -- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 msgr2=0x7fe608071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe606d9d640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe608071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe606d9d640 1 -- 172.21.15.135:0/2380173081 >> v1:172.21.15.135:6789/0 conn(0x7fe60814ec70 legacy=0x7fe60806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe606d9d640 1 -- 172.21.15.135:0/2380173081 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe608155620 con 0x7fe60806cf20 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.302+0000 7fe605d9b640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe608071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe606d9d640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe6080737c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe600009d90 tx=0x7fe60002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657175e6622276c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60559a640 1 -- 172.21.15.135:0/2380173081 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600031600 con 0x7fe60806cf20 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60559a640 1 -- 172.21.15.135:0/2380173081 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6000317a0 con 0x7fe60806cf20 2024-04-26T22:09:57.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60559a640 1 -- 172.21.15.135:0/2380173081 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600031b50 con 0x7fe60806cf20 2024-04-26T22:09:57.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60cc99640 1 -- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 msgr2=0x7fe6080737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:57.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe6080737c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe600009d90 tx=0x7fe60002a5d0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60cc99640 1 -- 172.21.15.135:0/2380173081 shutdown_connections 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe6080737c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2380173081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe608071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.303+0000 7fe60cc99640 1 -- 172.21.15.135:0/2380173081 >> 172.21.15.135:0/2380173081 conn(0x7fe60806e090 msgr2=0x7fe60806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.304+0000 7fe60cc99640 1 -- 172.21.15.135:0/2380173081 shutdown_connections 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.304+0000 7fe60cc99640 1 -- 172.21.15.135:0/2380173081 wait complete. 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 Processor -- start 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 -- start start 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe608150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe60814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 0x7fe6081ede70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:57.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe608155900 con 0x7fe60814ec70 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe608072290 con 0x7fe60806cf20 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.305+0000 7fe60cc99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe608073d00 con 0x7fe608074ba0 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 0x7fe6081ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 0x7fe6081ede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48672/0 (socket says 172.21.15.135:48672) 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 -- 172.21.15.135:0/2609875508 learned_addr learned my addr 172.21.15.135:0/2609875508 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe605d9b640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe60814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe60659c640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe608150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 -- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 msgr2=0x7fe60814f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe60814f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 -- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 msgr2=0x7fe608150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe608150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 -- 172.21.15.135:0/2609875508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe608158ca0 con 0x7fe60814ec70 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe606d9d640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 0x7fe6081ede70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe600009eb0 tx=0x7fe600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600040070 con 0x7fe60814ec70 2024-04-26T22:09:57.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60003c850 con 0x7fe60814ec70 2024-04-26T22:09:57.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe608151d10 con 0x7fe60814ec70 2024-04-26T22:09:57.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60003cc40 con 0x7fe60814ec70 2024-04-26T22:09:57.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.306+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6080c3f00 con 0x7fe60814ec70 2024-04-26T22:09:57.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.308+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6080c9260 con 0x7fe60814ec70 2024-04-26T22:09:57.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.308+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe600039040 con 0x7fe60814ec70 2024-04-26T22:09:57.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.309+0000 7fe5f6ffd640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5e806c8f0 0x7fe5e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:57.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.309+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7fe6000ad3b0 con 0x7fe60814ec70 2024-04-26T22:09:57.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.309+0000 7fe60659c640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5e806c8f0 0x7fe5e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:57.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.309+0000 7fe60659c640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5e806c8f0 0x7fe5e806edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005880 tx=0x7fe5fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:57.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.313+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe60007a980 con 0x7fe60814ec70 2024-04-26T22:09:57.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.550+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6080507a0 con 0x7fe60814ec70 2024-04-26T22:09:57.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.551+0000 7fe5f6ffd640 1 -- 172.21.15.135:0/2609875508 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7fe60007ec80 con 0x7fe60814ec70 2024-04-26T22:09:57.551 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:57.551 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:57.551 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5e806c8f0 msgr2=0x7fe5e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5e806c8f0 0x7fe5e806edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005880 tx=0x7fe5fc009040 comp rx=0 tx=0).stop 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 msgr2=0x7fe6081ede70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 0x7fe6081ede70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe600009eb0 tx=0x7fe600004a40 comp rx=0 tx=0).stop 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 shutdown_connections 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe5e806c8f0 0x7fe5e806edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe60814ec70 0x7fe6081ede70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe608074ba0 0x7fe60814f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 --2- 172.21.15.135:0/2609875508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe60806cf20 0x7fe608150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:57.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 >> 172.21.15.135:0/2609875508 conn(0x7fe60806e090 msgr2=0x7fe60806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:57.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 shutdown_connections 2024-04-26T22:09:57.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:57.553+0000 7fe60cc99640 1 -- 172.21.15.135:0/2609875508 wait complete. 2024-04-26T22:09:57.564 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:09:58.800 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:58.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 Processor -- start 2024-04-26T22:09:58.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 -- start start 2024-04-26T22:09:58.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd401541d0 0x7efd400c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:58.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 0x7efd4015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:58.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd400c3fc0 con 0x7efd40070490 2024-04-26T22:09:58.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efd400c3500 con 0x7efd401541d0 2024-04-26T22:09:58.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.909+0000 7efd47a6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efd400c37e0 con 0x7efd4015a3c0 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd4600a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 0x7efd4015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45809640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd401541d0 0x7efd400c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45008640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efd40070490 0x7efd400c7420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:43428/0 (socket says 172.21.15.135:43428) 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd4600a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 0x7efd4015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40652/0 (socket says 172.21.15.135:40652) 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45008640 1 -- 172.21.15.135:0/2369028709 learned_addr learned my addr 172.21.15.135:0/2369028709 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45809640 1 -- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 msgr2=0x7efd4015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45809640 1 --2- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 0x7efd4015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45809640 1 -- 172.21.15.135:0/2369028709 >> v1:172.21.15.139:6789/0 conn(0x7efd40070490 legacy=0x7efd400c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd44807640 1 -- 172.21.15.135:0/2369028709 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089749138 0 0) 0x7efd400c3fc0 con 0x7efd40070490 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd45809640 1 -- 172.21.15.135:0/2369028709 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd400c3ac0 con 0x7efd401541d0 2024-04-26T22:09:58.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.910+0000 7efd4600a640 1 --2- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 0x7efd4015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.911+0000 7efd45809640 1 --2- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd401541d0 0x7efd400c8400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097a0 tx=0x7efd3c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0dc8783d91fcd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.911+0000 7efd44807640 1 -- 172.21.15.135:0/2369028709 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c037070 con 0x7efd401541d0 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.911+0000 7efd44807640 1 -- 172.21.15.135:0/2369028709 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c02ae50 con 0x7efd401541d0 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.911+0000 7efd44807640 1 -- 172.21.15.135:0/2369028709 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c032660 con 0x7efd401541d0 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 -- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd401541d0 msgr2=0x7efd400c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 --2- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd401541d0 0x7efd400c8400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097a0 tx=0x7efd3c02a100 comp rx=0 tx=0).stop 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 -- 172.21.15.135:0/2369028709 shutdown_connections 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 --2- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd4015a3c0 0x7efd4015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 --2- 172.21.15.135:0/2369028709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd401541d0 0x7efd400c8400 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:58.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 -- 172.21.15.135:0/2369028709 >> 172.21.15.135:0/2369028709 conn(0x7efd400c8c40 msgr2=0x7efd400c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:58.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 -- 172.21.15.135:0/2369028709 shutdown_connections 2024-04-26T22:09:58.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.912+0000 7efd47a6d640 1 -- 172.21.15.135:0/2369028709 wait complete. 2024-04-26T22:09:58.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.913+0000 7efd47a6d640 1 Processor -- start 2024-04-26T22:09:58.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.913+0000 7efd47a6d640 1 -- start start 2024-04-26T22:09:58.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.913+0000 7efd47a6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd40070490 0x7efd40074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:58.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd47a6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 0x7efd400ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:58.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd47a6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd4015a3c0 0x7efd400a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:58.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd47a6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd400c7cb0 con 0x7efd401541d0 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd47a6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd400c7b30 con 0x7efd4015a3c0 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd47a6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efd400c42a0 con 0x7efd40070490 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 0x7efd400ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45809640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd40070490 0x7efd40074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 0x7efd400ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40014/0 (socket says 172.21.15.135:40014) 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 -- 172.21.15.135:0/1329611881 learned_addr learned my addr 172.21.15.135:0/1329611881 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd4600a640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd4015a3c0 0x7efd400a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 -- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd40070490 msgr2=0x7efd40074680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd40070490 0x7efd40074680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 -- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd4015a3c0 msgr2=0x7efd400a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd4015a3c0 0x7efd400a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 -- 172.21.15.135:0/1329611881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd400a9b70 con 0x7efd401541d0 2024-04-26T22:09:58.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.914+0000 7efd45008640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 0x7efd400ac8a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c4a0 tx=0x7efd3000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:58.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.916+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30017070 con 0x7efd401541d0 2024-04-26T22:09:58.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.916+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30005d00 con 0x7efd401541d0 2024-04-26T22:09:58.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.916+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd401ed480 con 0x7efd401541d0 2024-04-26T22:09:58.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.916+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30007310 con 0x7efd401541d0 2024-04-26T22:09:58.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.916+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd400a8600 con 0x7efd401541d0 2024-04-26T22:09:58.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.918+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd400507a0 con 0x7efd401541d0 2024-04-26T22:09:58.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.918+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd3001a070 con 0x7efd401541d0 2024-04-26T22:09:58.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.918+0000 7efd367fc640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efd2406c920 0x7efd2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:58.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.918+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7efd3008b670 con 0x7efd401541d0 2024-04-26T22:09:58.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.919+0000 7efd45809640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efd2406c920 0x7efd2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:58.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.919+0000 7efd45809640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efd2406c920 0x7efd2406ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efd3c034000 tx=0x7efd3c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:58.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:58.923+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd30058cc0 con 0x7efd401541d0 2024-04-26T22:09:59.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.140+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd400c6aa0 con 0x7efd401541d0 2024-04-26T22:09:59.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.141+0000 7efd367fc640 1 -- 172.21.15.135:0/1329611881 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7efd3005cfc0 con 0x7efd401541d0 2024-04-26T22:09:59.141 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:59.141 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:59.141 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:59.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efd2406c920 msgr2=0x7efd2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efd2406c920 0x7efd2406ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efd3c034000 tx=0x7efd3c02e000 comp rx=0 tx=0).stop 2024-04-26T22:09:59.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 msgr2=0x7efd400ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 0x7efd400ac8a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c4a0 tx=0x7efd3000c8a0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.143 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 shutdown_connections 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efd2406c920 0x7efd2406ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd4015a3c0 0x7efd400a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd401541d0 0x7efd400ac8a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 --2- 172.21.15.135:0/1329611881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd40070490 0x7efd40074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 >> 172.21.15.135:0/1329611881 conn(0x7efd400c8c40 msgr2=0x7efd400c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 shutdown_connections 2024-04-26T22:09:59.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.143+0000 7efd47a6d640 1 -- 172.21.15.135:0/1329611881 wait complete. 2024-04-26T22:09:59.154 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:09:59.565 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:09:59.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.674+0000 7f02f02c8640 1 Processor -- start 2024-04-26T22:09:59.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.674+0000 7f02f02c8640 1 -- start start 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02f02c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f02e806d590 0x7f02e81539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02f02c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e8153ff0 con 0x7f02e8157be0 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02f02c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e81542d0 con 0x7f02e80c5560 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02f02c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f02e81545b0 con 0x7f02e806d590 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02ee064640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f02e806d590 0x7f02e81539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02ee865640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f02e8157be0 0x7f02e8150cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:36122/0 (socket says 172.21.15.135:36122) 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.675+0000 7f02ee865640 1 -- 172.21.15.135:0/2168788686 learned_addr learned my addr 172.21.15.135:0/2168788686 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.676+0000 7f02ee064640 1 -- 172.21.15.135:0/2168788686 >> v1:172.21.15.139:6789/0 conn(0x7f02e80c5560 legacy=0x7f02e81574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.676+0000 7f02ee064640 1 -- 172.21.15.135:0/2168788686 >> v1:172.21.15.135:6789/0 conn(0x7f02e8157be0 legacy=0x7f02e8150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.676+0000 7f02ee064640 1 -- 172.21.15.135:0/2168788686 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e8154890 con 0x7f02e806d590 2024-04-26T22:09:59.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.676+0000 7f02ed062640 1 -- 172.21.15.135:0/2168788686 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368661643 0 0) 0x7f02e8153ff0 con 0x7f02e8157be0 2024-04-26T22:09:59.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.676+0000 7f02ee064640 1 --2- 172.21.15.135:0/2168788686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e806d590 0x7f02e81539a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097a0 tx=0x7f02e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da06eb759080a098 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02ed062640 1 -- 172.21.15.135:0/2168788686 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0037070 con 0x7f02e806d590 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02ed062640 1 -- 172.21.15.135:0/2168788686 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e0032440 con 0x7f02e806d590 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02ed062640 1 -- 172.21.15.135:0/2168788686 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0032810 con 0x7f02e806d590 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 -- 172.21.15.135:0/2168788686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e806d590 msgr2=0x7f02e81539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 --2- 172.21.15.135:0/2168788686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e806d590 0x7f02e81539a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097a0 tx=0x7f02e002a1a0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 -- 172.21.15.135:0/2168788686 shutdown_connections 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 --2- 172.21.15.135:0/2168788686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e806d590 0x7f02e81539a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 -- 172.21.15.135:0/2168788686 >> 172.21.15.135:0/2168788686 conn(0x7f02e80c5ce0 msgr2=0x7f02e80c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 -- 172.21.15.135:0/2168788686 shutdown_connections 2024-04-26T22:09:59.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.677+0000 7f02f02c8640 1 -- 172.21.15.135:0/2168788686 wait complete. 2024-04-26T22:09:59.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.678+0000 7f02f02c8640 1 Processor -- start 2024-04-26T22:09:59.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.678+0000 7f02f02c8640 1 -- start start 2024-04-26T22:09:59.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02f02c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02e806d590 0x7f02e80729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:59.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02f02c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 0x7f02e81eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02f02c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02e8157be0 0x7f02e81eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02f02c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f02e80c3c70 con 0x7f02e80c5560 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02f02c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8074b30 con 0x7f02e8157be0 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02f02c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f02e80c3df0 con 0x7f02e806d590 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 0x7f02e81eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 0x7f02e81eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40038/0 (socket says 172.21.15.135:40038) 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 -- 172.21.15.135:0/670545762 learned_addr learned my addr 172.21.15.135:0/670545762 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ee064640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02e806d590 0x7f02e80729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ee865640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02e8157be0 0x7f02e81eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 -- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02e806d590 msgr2=0x7f02e80729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02e806d590 0x7f02e80729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 -- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02e8157be0 msgr2=0x7f02e81eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02e8157be0 0x7f02e81eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.679+0000 7f02ed863640 1 -- 172.21.15.135:0/670545762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e80af2c0 con 0x7f02e80c5560 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.680+0000 7f02ed863640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 0x7f02e81eab60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c450 tx=0x7f02d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:59.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.680+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8017070 con 0x7f02e80c5560 2024-04-26T22:09:59.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.680+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d80052d0 con 0x7f02e80c5560 2024-04-26T22:09:59.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.680+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8003830 con 0x7f02e80c5560 2024-04-26T22:09:59.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.680+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e81ee0b0 con 0x7f02e80c5560 2024-04-26T22:09:59.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.680+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81ee670 con 0x7f02e80c5560 2024-04-26T22:09:59.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.681+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e80c5230 con 0x7f02e80c5560 2024-04-26T22:09:59.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.681+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02d801a070 con 0x7f02e80c5560 2024-04-26T22:09:59.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.682+0000 7f02deffd640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02cc06c8f0 0x7f02cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:09:59.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.682+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7784+0+0 (secure 0 0 0) 0x7f02d808b5e0 con 0x7f02e80c5560 2024-04-26T22:09:59.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.682+0000 7f02ee064640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02cc06c8f0 0x7f02cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:09:59.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.682+0000 7f02ee064640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02cc06c8f0 0x7f02cc06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f02e0034000 tx=0x7f02e002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:09:59.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.686+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02e80c5230 con 0x7f02e80c5560 2024-04-26T22:09:59.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.899+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02e80696c0 con 0x7f02e80c5560 2024-04-26T22:09:59.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.900+0000 7f02deffd640 1 -- 172.21.15.135:0/670545762 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5664 (secure 0 0 0) 0x7f02d8058c30 con 0x7f02e80c5560 2024-04-26T22:09:59.900 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:09:59.900 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:52.864862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:09:05.625009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":21,"addr":"172.21.15.135:6835/3800113814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3800113814},{"type":"v1","addr":"172.21.15.135:6835","nonce":3800113814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":2}]} 2024-04-26T22:09:59.900 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 72 2024-04-26T22:09:59.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02cc06c8f0 msgr2=0x7f02cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02cc06c8f0 0x7f02cc06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f02e0034000 tx=0x7f02e002e000 comp rx=0 tx=0).stop 2024-04-26T22:09:59.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 msgr2=0x7f02e81eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:09:59.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 0x7f02e81eab60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c450 tx=0x7f02d800c910 comp rx=0 tx=0).stop 2024-04-26T22:09:59.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 shutdown_connections 2024-04-26T22:09:59.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02cc06c8f0 0x7f02cc06edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02e8157be0 0x7f02e81eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02e80c5560 0x7f02e81eab60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 --2- 172.21.15.135:0/670545762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02e806d590 0x7f02e80729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:09:59.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 >> 172.21.15.135:0/670545762 conn(0x7f02e80c5ce0 msgr2=0x7f02e81501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:09:59.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 shutdown_connections 2024-04-26T22:09:59.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:09:59.902+0000 7f02f02c8640 1 -- 172.21.15.135:0/670545762 wait complete. 2024-04-26T22:09:59.914 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-26T22:10:01.155 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.265+0000 7f030c372640 1 Processor -- start 2024-04-26T22:10:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.265+0000 7f030c372640 1 -- start start 2024-04-26T22:10:01.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030c372640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03040c9830 0x7f0304153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:01.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030c372640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0304156de0 con 0x7f0304154220 2024-04-26T22:10:01.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030c372640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0304157b60 con 0x7f03040c7000 2024-04-26T22:10:01.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030c372640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f03040cad40 con 0x7f03040c9830 2024-04-26T22:10:01.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030990d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03040c9830 0x7f0304153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:01.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030a90f640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f0304154220 0x7f0304072930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:36134/0 (socket says 172.21.15.135:36134) 2024-04-26T22:10:01.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030a90f640 1 -- 172.21.15.135:0/2457259073 learned_addr learned my addr 172.21.15.135:0/2457259073 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:01.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030990d640 1 -- 172.21.15.135:0/2457259073 >> v1:172.21.15.139:6789/0 conn(0x7f03040c7000 legacy=0x7f03040ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030990d640 1 -- 172.21.15.135:0/2457259073 >> v1:172.21.15.135:6789/0 conn(0x7f0304154220 legacy=0x7f0304072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.266+0000 7f030990d640 1 -- 172.21.15.135:0/2457259073 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0304157080 con 0x7f03040c9830 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.267+0000 7f030990d640 1 --2- 172.21.15.135:0/2457259073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f0304153ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f02f40097a0 tx=0x7f02f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81666bd250c9f0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.267+0000 7f030910c640 1 -- 172.21.15.135:0/2457259073 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4037070 con 0x7f03040c9830 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.267+0000 7f030910c640 1 -- 172.21.15.135:0/2457259073 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f4032440 con 0x7f03040c9830 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.267+0000 7f030910c640 1 -- 172.21.15.135:0/2457259073 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4032810 con 0x7f03040c9830 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 -- 172.21.15.135:0/2457259073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 msgr2=0x7f0304153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 --2- 172.21.15.135:0/2457259073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f0304153ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f02f40097a0 tx=0x7f02f402a230 comp rx=0 tx=0).stop 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 -- 172.21.15.135:0/2457259073 shutdown_connections 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 --2- 172.21.15.135:0/2457259073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f0304153ce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 -- 172.21.15.135:0/2457259073 >> 172.21.15.135:0/2457259073 conn(0x7f03040adf20 msgr2=0x7f03040ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:01.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 -- 172.21.15.135:0/2457259073 shutdown_connections 2024-04-26T22:10:01.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.268+0000 7f030c372640 1 -- 172.21.15.135:0/2457259073 wait complete. 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.269+0000 7f030c372640 1 Processor -- start 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.269+0000 7f030c372640 1 -- start start 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03040c7000 0x7f0304156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f030414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0304154220 0x7f030414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03040c73e0 con 0x7f03040c9830 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0304157640 con 0x7f03040c7000 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f03040c7560 con 0x7f0304154220 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f030414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030a10e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03040c7000 0x7f0304156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f030414d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40052/0 (socket says 172.21.15.135:40052) 2024-04-26T22:10:01.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 -- 172.21.15.135:0/2574461806 learned_addr learned my addr 172.21.15.135:0/2574461806 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030a90f640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0304154220 0x7f030414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 -- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0304154220 msgr2=0x7f030414d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0304154220 0x7f030414d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 -- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03040c7000 msgr2=0x7f0304156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03040c7000 0x7f0304156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 -- 172.21.15.135:0/2574461806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03041f04c0 con 0x7f03040c9830 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030990d640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f030414d410 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f02f4035040 tx=0x7f02f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4037040 con 0x7f03040c9830 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f4030770 con 0x7f03040c9830 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030415a130 con 0x7f03040c9830 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.270+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4030b40 con 0x7f03040c9830 2024-04-26T22:10:01.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.271+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03040c56b0 con 0x7f03040c9830 2024-04-26T22:10:01.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.271+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03040766c0 con 0x7f03040c9830 2024-04-26T22:10:01.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.272+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02f4044020 con 0x7f03040c9830 2024-04-26T22:10:01.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.272+0000 7f02faffd640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02e806c8f0 0x7f02e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:01.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.272+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f02f40ac810 con 0x7f03040c9830 2024-04-26T22:10:01.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.272+0000 7f030a10e640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02e806c8f0 0x7f02e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:01.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.273+0000 7f030a10e640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02e806c8f0 0x7f02e806edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0300001780 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:01.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.277+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03041f04c0 con 0x7f03040c9830 2024-04-26T22:10:01.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.491+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03040507a0 con 0x7f03040c9830 2024-04-26T22:10:01.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.492+0000 7f02faffd640 1 -- 172.21.15.135:0/2574461806 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+4830 (secure 0 0 0) 0x7f02f4032e20 con 0x7f03040c9830 2024-04-26T22:10:01.492 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:01.492 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:09:59.926300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1},{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:00.929487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:10:01.493 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 75 2024-04-26T22:10:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02e806c8f0 msgr2=0x7f02e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02e806c8f0 0x7f02e806edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0300001780 tx=0x7f0300009040 comp rx=0 tx=0).stop 2024-04-26T22:10:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 msgr2=0x7f030414d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f030414d410 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f02f4035040 tx=0x7f02f40046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 shutdown_connections 2024-04-26T22:10:01.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f02e806c8f0 0x7f02e806edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0304154220 0x7f030414d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03040c9830 0x7f030414d410 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 --2- 172.21.15.135:0/2574461806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03040c7000 0x7f0304156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 >> 172.21.15.135:0/2574461806 conn(0x7f03040adf20 msgr2=0x7f03040ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 shutdown_connections 2024-04-26T22:10:01.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:01.494+0000 7f030c372640 1 -- 172.21.15.135:0/2574461806 wait complete. 2024-04-26T22:10:01.505 INFO:tasks.mds_thrash.fs.[a]:no change 2024-04-26T22:10:01.530 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:10:01.532 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:10:01.914 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 Processor -- start 2024-04-26T22:10:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 -- start start 2024-04-26T22:10:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03f80ca5a0 0x7f03f815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f80c8df0 con 0x7f03f80c6e00 2024-04-26T22:10:02.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f03f815a6e0 con 0x7f03f80ca5a0 2024-04-26T22:10:02.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03feebd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f03f80c9100 con 0x7f03f8154160 2024-04-26T22:10:02.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.067+0000 7f03f7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03f80ca5a0 0x7f03f815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.068+0000 7f03f7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03f80ca5a0 0x7f03f815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40060/0 (socket says 172.21.15.135:40060) 2024-04-26T22:10:02.068 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.068+0000 7f03f7fff640 1 -- 172.21.15.135:0/4098691044 learned_addr learned my addr 172.21.15.135:0/4098691044 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:02.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.068+0000 7f03f7fff640 1 -- 172.21.15.135:0/4098691044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 msgr2=0x7f03f8156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:02.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.068+0000 7f03f7fff640 1 --2- 172.21.15.135:0/4098691044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.068+0000 7f03f7fff640 1 -- 172.21.15.135:0/4098691044 >> v1:172.21.15.139:6789/0 conn(0x7f03f80c6e00 legacy=0x7f03f80c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:02.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.068+0000 7f03f7fff640 1 -- 172.21.15.135:0/4098691044 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f8154540 con 0x7f03f80ca5a0 2024-04-26T22:10:02.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.069+0000 7f03f7fff640 1 --2- 172.21.15.135:0/4098691044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80ca5a0 0x7f03f815a1a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009870 tx=0x7f03e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe8bcefc9f7e941 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:02.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.069+0000 7f03f77fe640 1 -- 172.21.15.135:0/4098691044 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4037070 con 0x7f03f80ca5a0 2024-04-26T22:10:02.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.069+0000 7f03f77fe640 1 -- 172.21.15.135:0/4098691044 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e4032440 con 0x7f03f80ca5a0 2024-04-26T22:10:02.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.070+0000 7f03f77fe640 1 -- 172.21.15.135:0/4098691044 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4032810 con 0x7f03f80ca5a0 2024-04-26T22:10:02.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.070+0000 7f03feebd640 1 -- 172.21.15.135:0/4098691044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80ca5a0 msgr2=0x7f03f815a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.070+0000 7f03feebd640 1 --2- 172.21.15.135:0/4098691044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80ca5a0 0x7f03f815a1a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009870 tx=0x7f03e402a210 comp rx=0 tx=0).stop 2024-04-26T22:10:02.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.071+0000 7f03feebd640 1 -- 172.21.15.135:0/4098691044 shutdown_connections 2024-04-26T22:10:02.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.071+0000 7f03feebd640 1 --2- 172.21.15.135:0/4098691044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80ca5a0 0x7f03f815a1a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.071+0000 7f03feebd640 1 --2- 172.21.15.135:0/4098691044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f8156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.071+0000 7f03feebd640 1 -- 172.21.15.135:0/4098691044 >> 172.21.15.135:0/4098691044 conn(0x7f03f80ae010 msgr2=0x7f03f80ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:02.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.071+0000 7f03feebd640 1 -- 172.21.15.135:0/4098691044 shutdown_connections 2024-04-26T22:10:02.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.071+0000 7f03feebd640 1 -- 172.21.15.135:0/4098691044 wait complete. 2024-04-26T22:10:02.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.072+0000 7f03feebd640 1 Processor -- start 2024-04-26T22:10:02.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.072+0000 7f03feebd640 1 -- start start 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03feebd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 0x7f03f81495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03feebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f80ca5a0 0x7f03f80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03feebd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f81f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03feebd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03f815aed0 con 0x7f03f80c6e00 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03feebd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03f815abf0 con 0x7f03f80ca5a0 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03feebd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f03f80c9ca0 con 0x7f03f8154160 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 0x7f03f81495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 0x7f03f81495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40064/0 (socket says 172.21.15.135:40064) 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 -- 172.21.15.135:0/1171317646 learned_addr learned my addr 172.21.15.135:0/1171317646 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03f7fff640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f80ca5a0 0x7f03f80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fd45a640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f81f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 -- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 msgr2=0x7f03f81f02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f81f02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 -- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f80ca5a0 msgr2=0x7f03f80ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f80ca5a0 0x7f03f80ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.073+0000 7f03fcc59640 1 -- 172.21.15.135:0/1171317646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f814c3f0 con 0x7f03f80c6e00 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.074+0000 7f03fcc59640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 0x7f03f81495f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00b410 tx=0x7f03ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.074+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec015020 con 0x7f03f80c6e00 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.074+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec011850 con 0x7f03f80c6e00 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.074+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f814b020 con 0x7f03f80c6e00 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.074+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec011c40 con 0x7f03f80c6e00 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.074+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f814b5e0 con 0x7f03f80c6e00 2024-04-26T22:10:02.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.075+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f80abd20 con 0x7f03f80c6e00 2024-04-26T22:10:02.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.076+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03ec018020 con 0x7f03f80c6e00 2024-04-26T22:10:02.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.076+0000 7f03f57fa640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f03dc06c8f0 0x7f03dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.076+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f03ec08c240 con 0x7f03f80c6e00 2024-04-26T22:10:02.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.077+0000 7f03f7fff640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f03dc06c8f0 0x7f03dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.077+0000 7f03f7fff640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f03dc06c8f0 0x7f03dc06edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f03e4034000 tx=0x7f03e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:02.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.085+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03f80abd20 con 0x7f03f80c6e00 2024-04-26T22:10:02.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.447+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03f8076650 con 0x7f03f80c6e00 2024-04-26T22:10:02.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.448+0000 7f03f57fa640 1 -- 172.21.15.135:0/1171317646 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+4837 (secure 0 0 0) 0x7f03ec01f020 con 0x7f03f80c6e00 2024-04-26T22:10:02.448 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:02.448 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:00.929487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:10:02.448 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 76 2024-04-26T22:10:02.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f03dc06c8f0 msgr2=0x7f03dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f03dc06c8f0 0x7f03dc06edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f03e4034000 tx=0x7f03e402e000 comp rx=0 tx=0).stop 2024-04-26T22:10:02.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 msgr2=0x7f03f81495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 0x7f03f81495f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00b410 tx=0x7f03ec00b8d0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 shutdown_connections 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f03dc06c8f0 0x7f03dc06edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03f8154160 0x7f03f81f02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03f80ca5a0 0x7f03f80ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 --2- 172.21.15.135:0/1171317646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f80c6e00 0x7f03f81495f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 >> 172.21.15.135:0/1171317646 conn(0x7f03f80ae010 msgr2=0x7f03f80c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 shutdown_connections 2024-04-26T22:10:02.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.450+0000 7f03feebd640 1 -- 172.21.15.135:0/1171317646 wait complete. 2024-04-26T22:10:02.463 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-04-26T22:10:02.463 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-04-26T22:10:02.463 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:02.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.608+0000 7efdc4cbd640 1 Processor -- start 2024-04-26T22:10:02.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.608+0000 7efdc4cbd640 1 -- start start 2024-04-26T22:10:02.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdc4cbd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efdc0150200 0x7efdc00ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdc4cbd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efdc00c2cd0 0x7efdc01533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdc4cbd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc006ff20 con 0x7efdc00ac680 2024-04-26T22:10:02.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdc4cbd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efdc00c4bd0 con 0x7efdc00c2cd0 2024-04-26T22:10:02.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdc4cbd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efdc006f4e0 con 0x7efdc0150200 2024-04-26T22:10:02.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdbed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efdc00ac680 0x7efdc0156210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:43474/0 (socket says 172.21.15.135:43474) 2024-04-26T22:10:02.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.610+0000 7efdbed9d640 1 -- 172.21.15.135:0/2751410110 learned_addr learned my addr 172.21.15.135:0/2751410110 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:02.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.611+0000 7efdbd59a640 1 -- 172.21.15.135:0/2751410110 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3072087299 0 0) 0x7efdb8001490 con 0x7efdc00ac680 2024-04-26T22:10:02.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.611+0000 7efdbd59a640 1 -- 172.21.15.135:0/2751410110 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda8003520 con 0x7efdc00ac680 2024-04-26T22:10:02.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.611+0000 7efdbe59c640 1 --2- 172.21.15.135:0/2751410110 >> v2:172.21.15.139:3300/0 conn(0x7efdc0150200 0x7efdc00ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.611+0000 7efdbdd9b640 1 --2- 172.21.15.135:0/2751410110 >> v2:172.21.15.135:3300/0 conn(0x7efdc00c2cd0 0x7efdc01533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.612+0000 7efdbdd9b640 1 -- 172.21.15.135:0/2751410110 >> v2:172.21.15.139:3300/0 conn(0x7efdc0150200 msgr2=0x7efdc00ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:02.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.612+0000 7efdbdd9b640 1 --2- 172.21.15.135:0/2751410110 >> v2:172.21.15.139:3300/0 conn(0x7efdc0150200 0x7efdc00ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.612+0000 7efdbdd9b640 1 -- 172.21.15.135:0/2751410110 >> v1:172.21.15.139:6789/0 conn(0x7efdc00ac680 legacy=0x7efdc0156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.612+0000 7efdbdd9b640 1 -- 172.21.15.135:0/2751410110 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc00c48f0 con 0x7efdc00c2cd0 2024-04-26T22:10:02.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.617+0000 7efdbdd9b640 1 --2- 172.21.15.135:0/2751410110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 0x7efdc01533b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efdc0071320 tx=0x7efdb002a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6e3459f0a8008a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:02.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.618+0000 7efdbd59a640 1 -- 172.21.15.135:0/2751410110 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0037070 con 0x7efdc00c2cd0 2024-04-26T22:10:02.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.618+0000 7efdbd59a640 1 -- 172.21.15.135:0/2751410110 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb0032440 con 0x7efdc00c2cd0 2024-04-26T22:10:02.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.618+0000 7efdbd59a640 1 -- 172.21.15.135:0/2751410110 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0037070 con 0x7efdc00c2cd0 2024-04-26T22:10:02.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.619+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2751410110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 msgr2=0x7efdc01533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.619+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2751410110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 0x7efdc01533b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efdc0071320 tx=0x7efdb002a240 comp rx=0 tx=0).stop 2024-04-26T22:10:02.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.621+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2751410110 shutdown_connections 2024-04-26T22:10:02.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.621+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2751410110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 0x7efdc01533b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.621+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2751410110 >> v2:172.21.15.139:3300/0 conn(0x7efdc0150200 0x7efdc00ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.621+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2751410110 >> 172.21.15.135:0/2751410110 conn(0x7efdc00c82d0 msgr2=0x7efdc00c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:02.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.621+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2751410110 shutdown_connections 2024-04-26T22:10:02.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.621+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2751410110 wait complete. 2024-04-26T22:10:02.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.623+0000 7efdc4cbd640 1 Processor -- start 2024-04-26T22:10:02.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.623+0000 7efdc4cbd640 1 -- start start 2024-04-26T22:10:02.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.624+0000 7efdc4cbd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdc00ac680 0x7efdc0149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.624+0000 7efdc4cbd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 0x7efdc01f0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.624+0000 7efdc4cbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 0x7efdc0148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdc4cbd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0150760 con 0x7efdc00c2cd0 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdc4cbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efdc01505e0 con 0x7efdc0150200 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdc4cbd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00c8dc0 con 0x7efdc00ac680 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 0x7efdc0148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 0x7efdc0148820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:37072/0 (socket says 172.21.15.135:37072) 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 -- 172.21.15.135:0/2412890899 learned_addr learned my addr 172.21.15.135:0/2412890899 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbe59c640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdc00ac680 0x7efdc0149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 -- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdc00ac680 msgr2=0x7efdc0149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdc00ac680 0x7efdc0149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 -- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 msgr2=0x7efdc01f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 0x7efdc01f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 -- 172.21.15.135:0/2412890899 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc01f0fd0 con 0x7efdc0150200 2024-04-26T22:10:02.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.625+0000 7efdbed9d640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 0x7efdc0148820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efdb80087e0 tx=0x7efdb8008ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:02.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.626+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8008310 con 0x7efdc0150200 2024-04-26T22:10:02.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.626+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc01f12b0 con 0x7efdc0150200 2024-04-26T22:10:02.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.626+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc00c6ff0 con 0x7efdc0150200 2024-04-26T22:10:02.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.626+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb800d6c0 con 0x7efdc0150200 2024-04-26T22:10:02.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.626+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb800da70 con 0x7efdc0150200 2024-04-26T22:10:02.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.629+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb8012070 con 0x7efdc0150200 2024-04-26T22:10:02.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.629+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda4005290 con 0x7efdc0150200 2024-04-26T22:10:02.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.630+0000 7efdaeffd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:02.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.630+0000 7efdbe59c640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:02.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.630+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7efdb808c6a0 con 0x7efdc0150200 2024-04-26T22:10:02.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.631+0000 7efdbe59c640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efdb400f8a0 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:02.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.641+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdc006ff20 con 0x7efdc0150200 2024-04-26T22:10:02.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.937+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda4005850 con 0x7efdc0150200 2024-04-26T22:10:02.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.947+0000 7efdaeffd640 1 -- 172.21.15.135:0/2412890899 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+4837 (secure 0 0 0) 0x7efdb805a1d0 con 0x7efdc0150200 2024-04-26T22:10:02.947 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:02.947 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:00.929487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:10:02.947 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 76 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda006c8f0 msgr2=0x7efda006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efdb400f8a0 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 msgr2=0x7efdc0148820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 0x7efdc0148820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efdb80087e0 tx=0x7efdb8008ca0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 shutdown_connections 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdc0150200 0x7efdc0148820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efdc00c2cd0 0x7efdc01f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 --2- 172.21.15.135:0/2412890899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdc00ac680 0x7efdc0149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:02.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 >> 172.21.15.135:0/2412890899 conn(0x7efdc00c82d0 msgr2=0x7efdc00c35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:02.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 shutdown_connections 2024-04-26T22:10:02.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:02.950+0000 7efdc4cbd640 1 -- 172.21.15.135:0/2412890899 wait complete. 2024-04-26T22:10:02.967 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T21:59:56.985345+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5653": { "addr": "172.21.15.139:6833/112034263", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6832", "nonce": 112034263, "type": "v2" }, { "addr": "172.21.15.139:6833", "nonce": 112034263, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5653, "incarnation": 72, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-26T22:10:01.932108+0000", "qdb_cluster": [ 5653 ], "qdb_leader": 5653, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5653 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-26T22:00:06.148422+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 75, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5291": { "addr": "172.21.15.139:6835/1717329133", "addrs": { "addrvec": [ { "addr": "172.21.15.139:6834", "nonce": 1717329133, "type": "v2" }, { "addr": "172.21.15.139:6835", "nonce": 1717329133, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5291, "incarnation": 75, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-26T22:10:00.929487+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5291 } } } ], "standbys": [] } 2024-04-26T22:10:03.506 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:03.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.640+0000 7fa5b782a640 1 Processor -- start 2024-04-26T22:10:03.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.640+0000 7fa5b782a640 1 -- start start 2024-04-26T22:10:03.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.641+0000 7fa5b782a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5b0150b40 0x7fa5b00c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.641+0000 7fa5b782a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b00c5690 0x7fa5b0157600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.641+0000 7fa5b782a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b00acf60 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.641+0000 7fa5b782a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b00ac470 con 0x7fa5b0150b40 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.641+0000 7fa5b782a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b00ac6c0 con 0x7fa5b00c5690 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b4dc5640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa5b00c3bf0 0x7fa5b0156ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:36146/0 (socket says 172.21.15.135:36146) 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b4dc5640 1 -- 172.21.15.135:0/4216174051 learned_addr learned my addr 172.21.15.135:0/4216174051 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b55c6640 1 --2- 172.21.15.135:0/4216174051 >> v2:172.21.15.139:3300/0 conn(0x7fa5b0150b40 0x7fa5b00c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5a7fff640 1 -- 172.21.15.135:0/4216174051 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699945017 0 0) 0x7fa5b00acf60 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5a7fff640 1 -- 172.21.15.135:0/4216174051 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa59c003520 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b55c6640 1 -- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b00c5690 msgr2=0x7fa5b0157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b55c6640 1 --2- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b00c5690 0x7fa5b0157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b55c6640 1 -- 172.21.15.135:0/4216174051 >> v1:172.21.15.135:6789/0 conn(0x7fa5b00c3bf0 legacy=0x7fa5b0156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:03.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.642+0000 7fa5b55c6640 1 -- 172.21.15.135:0/4216174051 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b00ac9a0 con 0x7fa5b0150b40 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.643+0000 7fa5b55c6640 1 --2- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b0150b40 0x7fa5b00c36b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097a0 tx=0x7fa5a002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db60b3e3e62a8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.644+0000 7fa5a7fff640 1 -- 172.21.15.135:0/4216174051 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0037070 con 0x7fa5b0150b40 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.644+0000 7fa5a7fff640 1 -- 172.21.15.135:0/4216174051 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a0032440 con 0x7fa5b0150b40 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.644+0000 7fa5a7fff640 1 -- 172.21.15.135:0/4216174051 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00327f0 con 0x7fa5b0150b40 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.644+0000 7fa5b782a640 1 -- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b0150b40 msgr2=0x7fa5b00c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 --2- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b0150b40 0x7fa5b00c36b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097a0 tx=0x7fa5a002a180 comp rx=0 tx=0).stop 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 -- 172.21.15.135:0/4216174051 shutdown_connections 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 --2- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b00c5690 0x7fa5b0157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 --2- 172.21.15.135:0/4216174051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b0150b40 0x7fa5b00c36b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 -- 172.21.15.135:0/4216174051 >> 172.21.15.135:0/4216174051 conn(0x7fa5b00c6d80 msgr2=0x7fa5b00c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 -- 172.21.15.135:0/4216174051 shutdown_connections 2024-04-26T22:10:03.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.645+0000 7fa5b782a640 1 -- 172.21.15.135:0/4216174051 wait complete. 2024-04-26T22:10:03.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.646+0000 7fa5b782a640 1 Processor -- start 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.646+0000 7fa5b782a640 1 -- start start 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b782a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 0x7fa5b01efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b782a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b00c5690 0x7fa5b01f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b782a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b0150b40 0x7fa5b0154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b782a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0150f20 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b55c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 0x7fa5b01efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b55c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 0x7fa5b01efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40084/0 (socket says 172.21.15.135:40084) 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b55c6640 1 -- 172.21.15.135:0/2510257188 learned_addr learned my addr 172.21.15.135:0/2510257188 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b4dc5640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b00c5690 0x7fa5b01f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b5dc7640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b0150b40 0x7fa5b0154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.647+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00c4930 con 0x7fa5b00c5690 2024-04-26T22:10:03.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b00ab930 con 0x7fa5b0150b40 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b55c6640 1 -- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b0150b40 msgr2=0x7fa5b0154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b55c6640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b0150b40 0x7fa5b0154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b55c6640 1 -- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b00c5690 msgr2=0x7fa5b01f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b55c6640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b00c5690 0x7fa5b01f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b55c6640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b01495c0 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5b55c6640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 0x7fa5b01efcb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa5a002a1c0 tx=0x7fa5a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.648+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0037040 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.649+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a0042070 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.649+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0006e10 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.649+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b01f0830 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.649+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b0149b60 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.651+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5a00327f0 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.651+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b0149f40 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.652+0000 7fa5a5ffb640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa59406c8f0 0x7fa59406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:03.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.652+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fa5a00acae0 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.656+0000 7fa5b4dc5640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa59406c8f0 0x7fa59406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:03.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.657+0000 7fa5b4dc5640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa59406c8f0 0x7fa59406edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa5b014cf50 tx=0x7fa5ac007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:03.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.660+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5a007a0e0 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.998+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5b0076650 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:03.999+0000 7fa5a5ffb640 1 -- 172.21.15.135:0/2510257188 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+4837 (secure 0 0 0) 0x7fa5a007e3e0 con 0x7fa5b00c3bf0 2024-04-26T22:10:03.999 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:03.999 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:00.929487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:10:03.999 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 76 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa59406c8f0 msgr2=0x7fa59406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa59406c8f0 0x7fa59406edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa5b014cf50 tx=0x7fa5ac007d30 comp rx=0 tx=0).stop 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 msgr2=0x7fa5b01efcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 0x7fa5b01efcb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa5a002a1c0 tx=0x7fa5a00046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 shutdown_connections 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa59406c8f0 0x7fa59406edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5b0150b40 0x7fa5b0154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5b00c5690 0x7fa5b01f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 --2- 172.21.15.135:0/2510257188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5b00c3bf0 0x7fa5b01efcb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:04.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.002+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 >> 172.21.15.135:0/2510257188 conn(0x7fa5b00c6d80 msgr2=0x7fa5b014de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:04.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.003+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 shutdown_connections 2024-04-26T22:10:04.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:04.003+0000 7fa5b782a640 1 -- 172.21.15.135:0/2510257188 wait complete. 2024-04-26T22:10:04.020 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=5210 2024-04-26T22:10:04.020 INFO:tasks.mds_thrash.fs.[a]:waiting for 7 secs before reviving mds.a 2024-04-26T22:10:04.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:05.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.092+0000 7f14b9fbf640 1 Processor -- start 2024-04-26T22:10:05.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.092+0000 7f14b9fbf640 1 -- start start 2024-04-26T22:10:05.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b9fbf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f14b40ca5a0 0x7f14b4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b9fbf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14b415a1a0 0x7f14b40c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b9fbf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40c8df0 con 0x7f14b4154160 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b9fbf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f14b40c9100 con 0x7f14b40ca5a0 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b9fbf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f14b4154540 con 0x7f14b415a1a0 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f14b40ca5a0 0x7f14b4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f14b40ca5a0 0x7f14b4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40090/0 (socket says 172.21.15.135:40090) 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 -- 172.21.15.135:0/2131925622 learned_addr learned my addr 172.21.15.135:0/2131925622 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b3fff640 1 --2- 172.21.15.135:0/2131925622 >> v2:172.21.15.139:3300/0 conn(0x7f14b415a1a0 0x7f14b40c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:05.095 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 -- 172.21.15.135:0/2131925622 >> v2:172.21.15.139:3300/0 conn(0x7f14b415a1a0 msgr2=0x7f14b40c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:05.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 --2- 172.21.15.135:0/2131925622 >> v2:172.21.15.139:3300/0 conn(0x7f14b415a1a0 0x7f14b40c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 -- 172.21.15.135:0/2131925622 >> v1:172.21.15.139:6789/0 conn(0x7f14b4154160 legacy=0x7f14b40c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:05.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.093+0000 7f14b2ffd640 1 -- 172.21.15.135:0/2131925622 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b415a910 con 0x7f14b40ca5a0 2024-04-26T22:10:05.096 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.094+0000 7f14b2ffd640 1 --2- 172.21.15.135:0/2131925622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b40ca5a0 0x7f14b4156bb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f14b40c9280 tx=0x7f14a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fc34c2f3e9d241 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.094+0000 7f14b27fc640 1 -- 172.21.15.135:0/2131925622 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8037070 con 0x7f14b40ca5a0 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.094+0000 7f14b27fc640 1 -- 172.21.15.135:0/2131925622 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a8032440 con 0x7f14b40ca5a0 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.094+0000 7f14b27fc640 1 -- 172.21.15.135:0/2131925622 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8032810 con 0x7f14b40ca5a0 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2131925622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b40ca5a0 msgr2=0x7f14b4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2131925622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b40ca5a0 0x7f14b4156bb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f14b40c9280 tx=0x7f14a802a270 comp rx=0 tx=0).stop 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2131925622 shutdown_connections 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2131925622 >> v2:172.21.15.139:3300/0 conn(0x7f14b415a1a0 0x7f14b40c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2131925622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b40ca5a0 0x7f14b4156bb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2131925622 >> 172.21.15.135:0/2131925622 conn(0x7f14b40ae010 msgr2=0x7f14b40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:05.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2131925622 shutdown_connections 2024-04-26T22:10:05.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.095+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2131925622 wait complete. 2024-04-26T22:10:05.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.096+0000 7f14b9fbf640 1 Processor -- start 2024-04-26T22:10:05.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.096+0000 7f14b9fbf640 1 -- start start 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.097+0000 7f14b9fbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b40ca5a0 0x7f14b4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.097+0000 7f14b9fbf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b4154160 0x7f14b4148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.097+0000 7f14b9fbf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 0x7f14b41492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.097+0000 7f14b9fbf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14b415a5b0 con 0x7f14b4154160 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.097+0000 7f14b9fbf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14b415aed0 con 0x7f14b40ca5a0 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.097+0000 7f14b9fbf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14b40c7510 con 0x7f14b415a1a0 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 0x7f14b41492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 0x7f14b41492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:40732/0 (socket says 172.21.15.135:40732) 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 -- 172.21.15.135:0/2042478993 learned_addr learned my addr 172.21.15.135:0/2042478993 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:05.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b37fe640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b40ca5a0 0x7f14b4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:05.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 -- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b40ca5a0 msgr2=0x7f14b4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:05.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b40ca5a0 0x7f14b4153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 -- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b4154160 msgr2=0x7f14b4148d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:10:05.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b4154160 0x7f14b4148d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.100+0000 7f14b3fff640 1 -- 172.21.15.135:0/2042478993 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b4152a00 con 0x7f14b415a1a0 2024-04-26T22:10:05.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.101+0000 7f14b3fff640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 0x7f14b41492a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d30 tx=0x7f14ac00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:05.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.101+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac018020 con 0x7f14b415a1a0 2024-04-26T22:10:05.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.102+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac014480 con 0x7f14b415a1a0 2024-04-26T22:10:05.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.102+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac014870 con 0x7f14b415a1a0 2024-04-26T22:10:05.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.102+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b4149b60 con 0x7f14b415a1a0 2024-04-26T22:10:05.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.102+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b414ba40 con 0x7f14b415a1a0 2024-04-26T22:10:05.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.104+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b4076650 con 0x7f14b415a1a0 2024-04-26T22:10:05.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.106+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14ac005d10 con 0x7f14b415a1a0 2024-04-26T22:10:05.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.106+0000 7f149bfff640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a006c8f0 0x7f14a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:05.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.107+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f14ac08d5a0 con 0x7f14b415a1a0 2024-04-26T22:10:05.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.107+0000 7f14b37fe640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a006c8f0 0x7f14a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:05.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.107+0000 7f14b37fe640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a006c8f0 0x7f14a006edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f14a4000f60 tx=0x7f14a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:05.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.118+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14ac022070 con 0x7f14b415a1a0 2024-04-26T22:10:05.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.529+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b414bc10 con 0x7f14b415a1a0 2024-04-26T22:10:05.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.531+0000 7f149bfff640 1 -- 172.21.15.135:0/2042478993 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+4841 (secure 0 0 0) 0x7f14ac010040 con 0x7f14b415a1a0 2024-04-26T22:10:05.532 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:05.532 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:04.850999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:reconnect","state_seq":36,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T22:10:05.532 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 77 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a006c8f0 msgr2=0x7f14a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a006c8f0 0x7f14a006edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f14a4000f60 tx=0x7f14a4009040 comp rx=0 tx=0).stop 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 msgr2=0x7f14b41492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 0x7f14b41492a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d30 tx=0x7f14ac00c490 comp rx=0 tx=0).stop 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 shutdown_connections 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f14a006c8f0 0x7f14a006edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b415a1a0 0x7f14b41492a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14b4154160 0x7f14b4148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 --2- 172.21.15.135:0/2042478993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b40ca5a0 0x7f14b4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:05.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.535+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 >> 172.21.15.135:0/2042478993 conn(0x7f14b40ae010 msgr2=0x7f14b4156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:05.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.536+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 shutdown_connections 2024-04-26T22:10:05.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:05.537+0000 7f14b9fbf640 1 -- 172.21.15.135:0/2042478993 wait complete. 2024-04-26T22:10:05.556 INFO:tasks.mds_thrash.fs.[b]:no change 2024-04-26T22:10:06.702 INFO:tasks.ceph.mds.d.smithi139.stderr:2024-04-26T22:10:06.702+0000 7fe9282db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:10:07.394 INFO:tasks.workunit.client.1.smithi139.stdout:Syncing()...38 sec 2024-04-26T22:10:07.394 INFO:tasks.workunit.client.1.smithi139.stdout:FFSB benchmark finished at: Fri Apr 26 22:10:07 2024 2024-04-26T22:10:07.394 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout:Results: 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout:Benchmark took 343.00 sec 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout:Total Results 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout:=============== 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout: ======= ============ ========= ======= =========== ========== 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout: write : 13536000 39463.61 100.000% 100.000% 154MB/sec 2024-04-26T22:10:07.395 INFO:tasks.workunit.client.1.smithi139.stdout:- 2024-04-26T22:10:07.397 INFO:tasks.workunit.client.1.smithi139.stdout:39463.61 Transactions per Second 2024-04-26T22:10:07.397 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.397 INFO:tasks.workunit.client.1.smithi139.stdout:Throughput Results 2024-04-26T22:10:07.397 INFO:tasks.workunit.client.1.smithi139.stdout:=================== 2024-04-26T22:10:07.397 INFO:tasks.workunit.client.1.smithi139.stdout:Write Throughput: 154MB/sec 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout:System Call Latency statistics in millisecs 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout:===== 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout: Min Avg Max Total Calls 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout: ======== ======== ======== ============ 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout:[ open] 0.004000 86.930459 25746.664062 10575 2024-04-26T22:10:07.448 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[0] 5000.000000 - 10000.000000 : 2 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[1] 2000.000000 - 5000.000000 : 66 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[2] 1000.000000 - 2000.000000 : 45 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[4] 200.000000 - 500.000000 : 6 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[5] 100.000000 - 200.000000 : 56 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[6] 50.000000 - 100.000000 : 100 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[7] 20.000000 - 50.000000 : 180 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[8] 10.000000 - 20.000000 : 83 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[9] 5.000000 - 10.000000 : 97 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[10] 2.000000 - 5.000000 : 176 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[11] 1.000000 - 2.000000 : 128 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[12] 0.500000 - 1.000000 : 20 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[13] 0.200000 - 0.500000 : 11 2024-04-26T22:10:07.449 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[14] 0.100000 - 0.200000 : 6 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[15] 0.050000 - 0.100000 : 40 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[16] 0.020000 - 0.050000 : 2209 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[17] 0.010000 - 0.020000 : 6927 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[18] 0.000000 - 0.010000 : 391 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout:[ write] 0.001000 0.633679 27640.714844 13536000 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[0] 5000.000000 - 10000.000000 : 179 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[1] 2000.000000 - 5000.000000 : 644 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[2] 1000.000000 - 2000.000000 : 416 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[3] 500.000000 - 1000.000000 : 443 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[4] 200.000000 - 500.000000 : 951 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[5] 100.000000 - 200.000000 : 7377 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[6] 50.000000 - 100.000000 : 15780 2024-04-26T22:10:07.450 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[7] 20.000000 - 50.000000 : 7267 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[8] 10.000000 - 20.000000 : 952 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[9] 5.000000 - 10.000000 : 1180 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[10] 2.000000 - 5.000000 : 1819 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[11] 1.000000 - 2.000000 : 1264 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[12] 0.500000 - 1.000000 : 1086 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[13] 0.200000 - 0.500000 : 1527 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[14] 0.100000 - 0.200000 : 1545 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[15] 0.050000 - 0.100000 : 3785 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[16] 0.020000 - 0.050000 : 21174 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[17] 0.010000 - 0.020000 : 70450 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[18] 0.000000 - 0.010000 : 13398060 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout:[ lseek] 0.000000 0.000544 113.192001 13536000 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-26T22:10:07.451 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[5] 100.000000 - 200.000000 : 2 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[6] 50.000000 - 100.000000 : 2 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[7] 20.000000 - 50.000000 : 13 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[8] 10.000000 - 20.000000 : 32 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[9] 5.000000 - 10.000000 : 64 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[10] 2.000000 - 5.000000 : 90 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[11] 1.000000 - 2.000000 : 84 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[12] 0.500000 - 1.000000 : 90 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[13] 0.200000 - 0.500000 : 118 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[14] 0.100000 - 0.200000 : 116 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[15] 0.050000 - 0.100000 : 271 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[16] 0.020000 - 0.050000 : 1398 2024-04-26T22:10:07.452 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[17] 0.010000 - 0.020000 : 3219 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[18] 0.000000 - 0.010000 : 13530501 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout:[ close] 0.001000 0.006740 3.548000 10575 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-04-26T22:10:07.453 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[11] 1.000000 - 2.000000 : 1 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[12] 0.500000 - 1.000000 : 1 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[13] 0.200000 - 0.500000 : 3 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[14] 0.100000 - 0.200000 : 1 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[15] 0.050000 - 0.100000 : 7 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[16] 0.020000 - 0.050000 : 29 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[17] 0.010000 - 0.020000 : 75 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: msec_range[18] 0.000000 - 0.010000 : 10456 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout: 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout:1.7% User Time 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout:13.8% System Time 2024-04-26T22:10:07.454 INFO:tasks.workunit.client.1.smithi139.stdout:15.5% CPU Utilization 2024-04-26T22:10:07.463 INFO:tasks.workunit.client.1.smithi139.stderr:+ cd .. 2024-04-26T22:10:07.464 INFO:tasks.workunit.client.1.smithi139.stderr:+ rm -r tmp ffsb 2024-04-26T22:10:07.557 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:07.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.659+0000 7fb23be7b640 1 Processor -- start 2024-04-26T22:10:07.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.659+0000 7fb23be7b640 1 -- start start 2024-04-26T22:10:07.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb23be7b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2340ab3f0 0x7fb2340c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:07.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb23be7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb2340c52c0 0x7fb2341574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:07.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb23be7b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2340c3810 con 0x7fb2341570d0 2024-04-26T22:10:07.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb23be7b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb2340c4330 con 0x7fb2340ab3f0 2024-04-26T22:10:07.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb23be7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb2340c3ab0 con 0x7fb2340c52c0 2024-04-26T22:10:07.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2340ab3f0 0x7fb2340c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb23a418640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb2340c52c0 0x7fb2341574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2340ab3f0 0x7fb2340c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40092/0 (socket says 172.21.15.135:40092) 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 -- 172.21.15.135:0/819912117 learned_addr learned my addr 172.21.15.135:0/819912117 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 -- 172.21.15.135:0/819912117 >> v2:172.21.15.139:3300/0 conn(0x7fb2340c52c0 msgr2=0x7fb2341574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 --2- 172.21.15.135:0/819912117 >> v2:172.21.15.139:3300/0 conn(0x7fb2340c52c0 0x7fb2341574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 -- 172.21.15.135:0/819912117 >> v1:172.21.15.139:6789/0 conn(0x7fb2341570d0 legacy=0x7fb234150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:07.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.660+0000 7fb239416640 1 -- 172.21.15.135:0/819912117 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2340c4580 con 0x7fb2340ab3f0 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb239416640 1 --2- 172.21.15.135:0/819912117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c4d80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb2340c44b0 tx=0x7fb22402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f5029751290a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb238c15640 1 -- 172.21.15.135:0/819912117 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224037070 con 0x7fb2340ab3f0 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb238c15640 1 -- 172.21.15.135:0/819912117 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb224032440 con 0x7fb2340ab3f0 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb238c15640 1 -- 172.21.15.135:0/819912117 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224032810 con 0x7fb2340ab3f0 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 -- 172.21.15.135:0/819912117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 msgr2=0x7fb2340c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 --2- 172.21.15.135:0/819912117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c4d80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb2340c44b0 tx=0x7fb22402a140 comp rx=0 tx=0).stop 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 -- 172.21.15.135:0/819912117 shutdown_connections 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 --2- 172.21.15.135:0/819912117 >> v2:172.21.15.139:3300/0 conn(0x7fb2340c52c0 0x7fb2341574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 --2- 172.21.15.135:0/819912117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c4d80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 -- 172.21.15.135:0/819912117 >> 172.21.15.135:0/819912117 conn(0x7fb23406c310 msgr2=0x7fb23406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:07.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 -- 172.21.15.135:0/819912117 shutdown_connections 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.661+0000 7fb23be7b640 1 -- 172.21.15.135:0/819912117 wait complete. 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 Processor -- start 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 -- start start 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2340c52c0 0x7fb2340c1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2341570d0 0x7fb2341efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2341512e0 con 0x7fb2340ab3f0 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb234151160 con 0x7fb2341570d0 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23be7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb2340aba60 con 0x7fb2340c52c0 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb239c17640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb239c17640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40094/0 (socket says 172.21.15.135:40094) 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb239c17640 1 -- 172.21.15.135:0/3348721953 learned_addr learned my addr 172.21.15.135:0/3348721953 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb23a418640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2341570d0 0x7fb2341efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:07.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.663+0000 7fb239416640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2340c52c0 0x7fb2340c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb239c17640 1 -- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2340c52c0 msgr2=0x7fb2340c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb239c17640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2340c52c0 0x7fb2340c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb239c17640 1 -- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2341570d0 msgr2=0x7fb2341efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb239c17640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2341570d0 0x7fb2341efac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb239c17640 1 -- 172.21.15.135:0/3348721953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2341547b0 con 0x7fb2340ab3f0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb23a418640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2341570d0 0x7fb2341efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb239c17640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c6aa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c3c0 tx=0x7fb23000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300052d0 con 0x7fb2340ab3f0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb230005490 con 0x7fb2340ab3f0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23406cb60 con 0x7fb2340ab3f0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230007260 con 0x7fb2340ab3f0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.664+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2340740e0 con 0x7fb2340ab3f0 2024-04-26T22:10:07.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.665+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2340507a0 con 0x7fb2340ab3f0 2024-04-26T22:10:07.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.666+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb230007400 con 0x7fb2340ab3f0 2024-04-26T22:10:07.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.666+0000 7fb22a7fc640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb21806c8f0 0x7fb21806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:07.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.666+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fb23008b780 con 0x7fb2340ab3f0 2024-04-26T22:10:07.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.666+0000 7fb239416640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb21806c8f0 0x7fb21806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:07.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.667+0000 7fb239416640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb21806c8f0 0x7fb21806edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb224034000 tx=0x7fb22402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:07.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.671+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb230058d80 con 0x7fb2340ab3f0 2024-04-26T22:10:07.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.886+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2340c4110 con 0x7fb2340ab3f0 2024-04-26T22:10:07.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.886+0000 7fb22a7fc640 1 -- 172.21.15.135:0/3348721953 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+4845 (secure 0 0 0) 0x7fb23005d080 con 0x7fb2340ab3f0 2024-04-26T22:10:07.888 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:07.888 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:10:07.888 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 79 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.888+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb21806c8f0 msgr2=0x7fb21806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.888+0000 7fb23be7b640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb21806c8f0 0x7fb21806edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb224034000 tx=0x7fb22402e000 comp rx=0 tx=0).stop 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.888+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 msgr2=0x7fb2340c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.888+0000 7fb23be7b640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c6aa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c3c0 tx=0x7fb23000c880 comp rx=0 tx=0).stop 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.888+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 shutdown_connections 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.888+0000 7fb23be7b640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb21806c8f0 0x7fb21806edb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.889+0000 7fb23be7b640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2341570d0 0x7fb2341efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.889+0000 7fb23be7b640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2340c52c0 0x7fb2340c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.889+0000 7fb23be7b640 1 --2- 172.21.15.135:0/3348721953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2340ab3f0 0x7fb2340c6aa0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.889+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 >> 172.21.15.135:0/3348721953 conn(0x7fb23406c310 msgr2=0x7fb2341574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:07.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.889+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 shutdown_connections 2024-04-26T22:10:07.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:07.889+0000 7fb23be7b640 1 -- 172.21.15.135:0/3348721953 wait complete. 2024-04-26T22:10:07.901 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=5970 2024-04-26T22:10:07.901 INFO:tasks.mds_thrash.fs.[b]:waiting for 1 secs before reviving mds.c 2024-04-26T22:10:08.902 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-04-26T22:10:08.902 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:10:08.902 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:10:08.905 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:10:08.905 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:09.023 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-26T22:10:09.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.047+0000 7f05cefb4640 1 Processor -- start 2024-04-26T22:10:09.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.048+0000 7f05cefb4640 1 -- start start 2024-04-26T22:10:09.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05cefb4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f05c8158640 0x7f05c80bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:09.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05cefb4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c80bffe0 con 0x7f05c815f0e0 2024-04-26T22:10:09.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05cefb4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c815e600 con 0x7f05c80c5ac0 2024-04-26T22:10:09.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05cefb4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f05c80c02c0 con 0x7f05c8158640 2024-04-26T22:10:09.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05c7fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f05c815f0e0 0x7f05c80ab470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47558/0 (socket says 172.21.15.135:47558) 2024-04-26T22:10:09.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05c7fff640 1 -- 172.21.15.135:0/3320302258 learned_addr learned my addr 172.21.15.135:0/3320302258 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:09.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05ccd50640 1 --2- 172.21.15.135:0/3320302258 >> v2:172.21.15.139:3300/0 conn(0x7f05c8158640 0x7f05c80bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:09.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05c77fe640 1 -- 172.21.15.135:0/3320302258 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594636870 0 0) 0x7f05c80bffe0 con 0x7f05c815f0e0 2024-04-26T22:10:09.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05c77fe640 1 -- 172.21.15.135:0/3320302258 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b0003520 con 0x7f05c815f0e0 2024-04-26T22:10:09.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05ccd50640 1 -- 172.21.15.135:0/3320302258 >> v1:172.21.15.139:6789/0 conn(0x7f05c80c5ac0 legacy=0x7f05c80c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05ccd50640 1 -- 172.21.15.135:0/3320302258 >> v1:172.21.15.135:6789/0 conn(0x7f05c815f0e0 legacy=0x7f05c80ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.049+0000 7f05ccd50640 1 -- 172.21.15.135:0/3320302258 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c815e8e0 con 0x7f05c8158640 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.051+0000 7f05ccd50640 1 --2- 172.21.15.135:0/3320302258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c8158640 0x7f05c80bfa00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009d90 tx=0x7f05c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8607c8f9ba1a19a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.053+0000 7f05c77fe640 1 -- 172.21.15.135:0/3320302258 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0007740 con 0x7f05c8158640 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.053+0000 7f05c77fe640 1 -- 172.21.15.135:0/3320302258 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c00078e0 con 0x7f05c8158640 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.054+0000 7f05cefb4640 1 -- 172.21.15.135:0/3320302258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c8158640 msgr2=0x7f05c80bfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.054+0000 7f05cefb4640 1 --2- 172.21.15.135:0/3320302258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c8158640 0x7f05c80bfa00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009d90 tx=0x7f05c002a660 comp rx=0 tx=0).stop 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.055+0000 7f05cefb4640 1 -- 172.21.15.135:0/3320302258 shutdown_connections 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.055+0000 7f05cefb4640 1 --2- 172.21.15.135:0/3320302258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c8158640 0x7f05c80bfa00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.056 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.055+0000 7f05cefb4640 1 -- 172.21.15.135:0/3320302258 >> 172.21.15.135:0/3320302258 conn(0x7f05c80caad0 msgr2=0x7f05c80c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:09.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.056+0000 7f05cefb4640 1 -- 172.21.15.135:0/3320302258 shutdown_connections 2024-04-26T22:10:09.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.056+0000 7f05cefb4640 1 -- 172.21.15.135:0/3320302258 wait complete. 2024-04-26T22:10:09.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.058+0000 7f05cefb4640 1 Processor -- start 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.058+0000 7f05cefb4640 1 -- start start 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05cefb4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 0x7f05c8151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05cefb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05c8158640 0x7f05c81f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05cefb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c815f0e0 0x7f05c81f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05cefb4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80abb80 con 0x7f05c80c5ac0 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05cefb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8074bd0 con 0x7f05c815f0e0 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05cefb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f05c80c67a0 con 0x7f05c8158640 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.059+0000 7f05ccd50640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 0x7f05c8151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 0x7f05c8151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51186/0 (socket says 172.21.15.135:51186) 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 -- 172.21.15.135:0/4211519731 learned_addr learned my addr 172.21.15.135:0/4211519731 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05cd551640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c815f0e0 0x7f05c81f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:09.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05c7fff640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05c8158640 0x7f05c81f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:09.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 -- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05c8158640 msgr2=0x7f05c81f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05c8158640 0x7f05c81f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 -- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c815f0e0 msgr2=0x7f05c81f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c815f0e0 0x7f05c81f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 -- 172.21.15.135:0/4211519731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c812f9c0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.060+0000 7f05ccd50640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 0x7f05c8151060 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009eb0 tx=0x7f05c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:09.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.061+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0040070 con 0x7f05c80c5ac0 2024-04-26T22:10:09.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.061+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0045660 con 0x7f05c80c5ac0 2024-04-26T22:10:09.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.061+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c812fca0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.061+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0045a10 con 0x7f05c80c5ac0 2024-04-26T22:10:09.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.061+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c80c80c0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.062+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c80506c0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.063+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05c0039040 con 0x7f05c80c5ac0 2024-04-26T22:10:09.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.063+0000 7f05c57fa640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f05b406c940 0x7f05b406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:09.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.064+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f05c00adab0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.064+0000 7f05c7fff640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f05b406c940 0x7f05b406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:09.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.064+0000 7f05c7fff640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f05b406c940 0x7f05b406ee00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05c80c63e0 tx=0x7f05bc007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:09.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.070+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05c007b7e0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.459+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c80c6130 con 0x7f05c80c5ac0 2024-04-26T22:10:09.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.460+0000 7f05c57fa640 1 -- 172.21.15.135:0/4211519731 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+4845 (secure 0 0 0) 0x7f05c007b0e0 con 0x7f05c80c5ac0 2024-04-26T22:10:09.461 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:09.461 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:10:09.461 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 79 2024-04-26T22:10:09.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f05b406c940 msgr2=0x7f05b406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f05b406c940 0x7f05b406ee00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05c80c63e0 tx=0x7f05bc007cf0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 msgr2=0x7f05c8151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 0x7f05c8151060 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009eb0 tx=0x7f05c0004a40 comp rx=0 tx=0).stop 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 shutdown_connections 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f05b406c940 0x7f05b406ee00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05c815f0e0 0x7f05c81f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05c8158640 0x7f05c81f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 --2- 172.21.15.135:0/4211519731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c5ac0 0x7f05c8151060 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.462+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 >> 172.21.15.135:0/4211519731 conn(0x7f05c80caad0 msgr2=0x7f05c80ac500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.463+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 shutdown_connections 2024-04-26T22:10:09.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:09.463+0000 7f05cefb4640 1 -- 172.21.15.135:0/4211519731 wait complete. 2024-04-26T22:10:09.481 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-26T22:10:09.538 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T22:10:09.538 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-26T22:10:09.591 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.1... 2024-04-26T22:10:09.592 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-26T22:10:09.984 DEBUG:teuthology.parallel:result is None 2024-04-26T22:10:09.985 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T22:10:10.018 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T22:10:10.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-26T22:10:10.084 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-26T22:10:10.084 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-04-26T22:10:10.093 INFO:tasks.mds_thrash:joining mds_thrasher 2024-04-26T22:10:10.967 INFO:tasks.ceph.mds.b.smithi139.stderr:2024-04-26T22:10:10.967+0000 7f4cf4f7f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:10:11.020 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-04-26T22:10:11.021 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:10:11.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:10:11.023 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:10:11.023 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:11.149 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-26T22:10:11.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.183+0000 7fef4a4d1640 1 Processor -- start 2024-04-26T22:10:11.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.183+0000 7fef4a4d1640 1 -- start start 2024-04-26T22:10:11.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.184+0000 7fef4a4d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3650 0x7fef4406e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.184+0000 7fef4a4d1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fef440aa8c0 0x7fef440c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.184+0000 7fef4a4d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef440c3f70 0x7fef440abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.184+0000 7fef4a4d1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fef4415c0d0 con 0x7fef440aa8c0 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.184+0000 7fef4a4d1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fef440ab4b0 con 0x7fef440c3f70 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.184+0000 7fef4a4d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef4415bdf0 con 0x7fef440c3650 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fef440aa8c0 0x7fef440c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef48a6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef440c3f70 0x7fef440abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef48a6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef440c3f70 0x7fef440abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55786/0 (socket says 172.21.15.135:55786) 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fef440aa8c0 0x7fef440c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51208/0 (socket says 172.21.15.135:51208) 2024-04-26T22:10:11.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef48a6e640 1 -- 172.21.15.135:0/731549512 learned_addr learned my addr 172.21.15.135:0/731549512 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 -- 172.21.15.135:0/731549512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3650 msgr2=0x7fef4406e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 --2- 172.21.15.135:0/731549512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3650 0x7fef4406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 -- 172.21.15.135:0/731549512 >> v2:172.21.15.139:3300/0 conn(0x7fef440c3f70 msgr2=0x7fef440abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 --2- 172.21.15.135:0/731549512 >> v2:172.21.15.139:3300/0 conn(0x7fef440c3f70 0x7fef440abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.185+0000 7fef437fe640 1 -- 172.21.15.135:0/731549512 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440ab750 con 0x7fef440aa8c0 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.186+0000 7fef437fe640 1 --2- 172.21.15.135:0/731549512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440aa8c0 0x7fef440c3a30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fef44071620 tx=0x7fef3c02a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652bc53424924d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.187+0000 7fef42ffd640 1 -- 172.21.15.135:0/731549512 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c007740 con 0x7fef440aa8c0 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.187+0000 7fef42ffd640 1 -- 172.21.15.135:0/731549512 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c0078e0 con 0x7fef440aa8c0 2024-04-26T22:10:11.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.187+0000 7fef42ffd640 1 -- 172.21.15.135:0/731549512 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c007cb0 con 0x7fef440aa8c0 2024-04-26T22:10:11.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.190+0000 7fef4a4d1640 1 -- 172.21.15.135:0/731549512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440aa8c0 msgr2=0x7fef440c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.190+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/731549512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440aa8c0 0x7fef440c3a30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fef44071620 tx=0x7fef3c02a4b0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 -- 172.21.15.135:0/731549512 shutdown_connections 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/731549512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440aa8c0 0x7fef440c3a30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/731549512 >> v2:172.21.15.139:3300/0 conn(0x7fef440c3f70 0x7fef440abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/731549512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3650 0x7fef4406e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 -- 172.21.15.135:0/731549512 >> 172.21.15.135:0/731549512 conn(0x7fef440c82d0 msgr2=0x7fef440c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 -- 172.21.15.135:0/731549512 shutdown_connections 2024-04-26T22:10:11.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.191+0000 7fef4a4d1640 1 -- 172.21.15.135:0/731549512 wait complete. 2024-04-26T22:10:11.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.193+0000 7fef4a4d1640 1 Processor -- start 2024-04-26T22:10:11.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.193+0000 7fef4a4d1640 1 -- start start 2024-04-26T22:10:11.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef4a4d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 0x7fef4415df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef4a4d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440c3650 0x7fef4415e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef4a4d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3f70 0x7fef441f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef4a4d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef440ac610 con 0x7fef440c3650 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef4a4d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef440ac490 con 0x7fef440aa8c0 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef4a4d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef440ac790 con 0x7fef440c3f70 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 0x7fef4415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef437fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440c3650 0x7fef4415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 0x7fef4415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55796/0 (socket says 172.21.15.135:55796) 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef43fff640 1 -- 172.21.15.135:0/1015600997 learned_addr learned my addr 172.21.15.135:0/1015600997 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.194+0000 7fef48a6e640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3f70 0x7fef441f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.195+0000 7fef43fff640 1 -- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3f70 msgr2=0x7fef441f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.195+0000 7fef43fff640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3f70 0x7fef441f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.195+0000 7fef43fff640 1 -- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440c3650 msgr2=0x7fef4415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.195+0000 7fef43fff640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440c3650 0x7fef4415e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.195+0000 7fef43fff640 1 -- 172.21.15.135:0/1015600997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef44072390 con 0x7fef440aa8c0 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.195+0000 7fef437fe640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440c3650 0x7fef4415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:11.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.196+0000 7fef43fff640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 0x7fef4415df30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fef3400b4d0 tx=0x7fef3400b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.196+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34014070 con 0x7fef440aa8c0 2024-04-26T22:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.196+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441e4780 con 0x7fef440aa8c0 2024-04-26T22:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.196+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef441f4ea0 con 0x7fef440aa8c0 2024-04-26T22:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.196+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef34010870 con 0x7fef440aa8c0 2024-04-26T22:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.197+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3401e060 con 0x7fef440aa8c0 2024-04-26T22:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.198+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef24005290 con 0x7fef440aa8c0 2024-04-26T22:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.198+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef34017070 con 0x7fef440aa8c0 2024-04-26T22:10:11.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.199+0000 7fef40ff9640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef3006c8f0 0x7fef3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.199+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fef3408c0f0 con 0x7fef440aa8c0 2024-04-26T22:10:11.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.199+0000 7fef437fe640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef3006c8f0 0x7fef3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.201 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.200+0000 7fef437fe640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef3006c8f0 0x7fef3006edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef3c034000 tx=0x7fef3c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:11.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.208+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef34059670 con 0x7fef440aa8c0 2024-04-26T22:10:11.482 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:11.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.594+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef24005850 con 0x7fef440aa8c0 2024-04-26T22:10:11.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.597+0000 7fef40ff9640 1 -- 172.21.15.135:0/1015600997 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5672 (secure 0 0 0) 0x7fef3405d970 con 0x7fef440aa8c0 2024-04-26T22:10:11.598 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:11.598 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:10:11.598 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 80 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef3006c8f0 msgr2=0x7fef3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef3006c8f0 0x7fef3006edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef3c034000 tx=0x7fef3c02f000 comp rx=0 tx=0).stop 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 msgr2=0x7fef4415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 0x7fef4415df30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fef3400b4d0 tx=0x7fef3400b990 comp rx=0 tx=0).stop 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 shutdown_connections 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fef3006c8f0 0x7fef3006edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef440c3f70 0x7fef441f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef440c3650 0x7fef4415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 --2- 172.21.15.135:0/1015600997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440aa8c0 0x7fef4415df30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 >> 172.21.15.135:0/1015600997 conn(0x7fef440c82d0 msgr2=0x7fef44159580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:11.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 shutdown_connections 2024-04-26T22:10:11.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.600+0000 7fef4a4d1640 1 -- 172.21.15.135:0/1015600997 wait complete. 2024-04-26T22:10:11.624 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-26T22:10:11.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.643+0000 7ff3dcc74640 1 Processor -- start 2024-04-26T22:10:11.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.643+0000 7ff3dcc74640 1 -- start start 2024-04-26T22:10:11.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3dcc74640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3d80abfc0 0x7ff3d815e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3dcc74640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d80c3750 0x7ff3d815da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3dcc74640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d815e3f0 con 0x7ff3d815dfa0 2024-04-26T22:10:11.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3dcc74640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d815eba0 con 0x7ff3d80abfc0 2024-04-26T22:10:11.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3dcc74640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d806e3d0 con 0x7ff3d80c3750 2024-04-26T22:10:11.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3d6d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff3d815dfa0 0x7ff3d806eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:43914/0 (socket says 172.21.15.135:43914) 2024-04-26T22:10:11.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.644+0000 7ff3d6d9d640 1 -- 172.21.15.135:0/288357920 learned_addr learned my addr 172.21.15.135:0/288357920 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:11.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d5d9b640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d80c3750 0x7ff3d815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881717910 0 0) 0x7ff3d815e3f0 con 0x7ff3d815dfa0 2024-04-26T22:10:11.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3c0003520 con 0x7ff3d815dfa0 2024-04-26T22:10:11.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2386022817 0 0) 0x7ff3c0003520 con 0x7ff3d815dfa0 2024-04-26T22:10:11.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d80c3750 msgr2=0x7ff3d815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:11.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d80c3750 0x7ff3d815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 >> v2:172.21.15.139:3300/0 conn(0x7ff3d80abfc0 msgr2=0x7ff3d815e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 --2- 172.21.15.135:0/288357920 >> v2:172.21.15.139:3300/0 conn(0x7ff3d80abfc0 0x7ff3d815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d815edd0 con 0x7ff3d815dfa0 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7ff3c8001650 con 0x7ff3d815dfa0 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 >> v1:172.21.15.139:6789/0 conn(0x7ff3d815dfa0 legacy=0x7ff3d806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3c00039d0 0x7ff3c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c0004390 0x7ff3c0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c0005060 0x7ff3c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d806e3d0 con 0x7ff3c0005060 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c000c010 con 0x7ff3c0004390 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.645+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c000c210 con 0x7ff3c00039d0 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.649+0000 7ff3d5d9b640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d80c3750 0x7ff3d815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.649+0000 7ff3d659c640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3c00039d0 0x7ff3c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d6d9d640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c0005060 0x7ff3c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d5d9b640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c0004390 0x7ff3c0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d6d9d640 1 -- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3c00039d0 msgr2=0x7ff3c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:11.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d6d9d640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3c00039d0 0x7ff3c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d6d9d640 1 -- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c0004390 msgr2=0x7ff3c0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:11.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d6d9d640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c0004390 0x7ff3c0004860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.651+0000 7ff3d6d9d640 1 -- 172.21.15.135:0/288357920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d815edd0 con 0x7ff3c0005060 2024-04-26T22:10:11.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.652+0000 7ff3d6d9d640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c0005060 0x7ff3c000b9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c370 tx=0x7ff3c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bde350f0362c40b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:11.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.653+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8007dd0 con 0x7ff3c0005060 2024-04-26T22:10:11.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.653+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c8004aa0 con 0x7ff3c0005060 2024-04-26T22:10:11.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.653+0000 7ff3d559a640 1 -- 172.21.15.135:0/288357920 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80070d0 con 0x7ff3c0005060 2024-04-26T22:10:11.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.654+0000 7ff3dcc74640 1 -- 172.21.15.135:0/288357920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c0005060 msgr2=0x7ff3c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.654+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/288357920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c0005060 0x7ff3c000b9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c370 tx=0x7ff3c0005830 comp rx=0 tx=0).stop 2024-04-26T22:10:11.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.655+0000 7ff3d6d9d640 1 -- 172.21.15.135:0/288357920 reap_dead start 2024-04-26T22:10:11.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.655+0000 7ff3dcc74640 1 -- 172.21.15.135:0/288357920 shutdown_connections 2024-04-26T22:10:11.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.655+0000 7ff3dcc74640 1 -- 172.21.15.135:0/288357920 >> 172.21.15.135:0/288357920 conn(0x7ff3d806c310 msgr2=0x7ff3d806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:11.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.655+0000 7ff3dcc74640 1 -- 172.21.15.135:0/288357920 shutdown_connections 2024-04-26T22:10:11.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.657+0000 7ff3dcc74640 1 -- 172.21.15.135:0/288357920 wait complete. 2024-04-26T22:10:11.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.659+0000 7ff3dcc74640 1 Processor -- start 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.660+0000 7ff3dcc74640 1 -- start start 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3dcc74640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 0x7ff3d81ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3dcc74640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d8148140 0x7ff3d81485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3dcc74640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d8073010 0x7ff3d8073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3dcc74640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d806e620 con 0x7ff3d8148140 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3dcc74640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d815f390 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3dcc74640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d815c560 con 0x7ff3d8073010 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3d659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 0x7ff3d81ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3d659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 0x7ff3d81ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55810/0 (socket says 172.21.15.135:55810) 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.661+0000 7ff3d659c640 1 -- 172.21.15.135:0/1106936946 learned_addr learned my addr 172.21.15.135:0/1106936946 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.662+0000 7ff3d659c640 1 -- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d8073010 msgr2=0x7ff3d8073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.662+0000 7ff3d659c640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d8073010 0x7ff3d8073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.662+0000 7ff3d659c640 1 -- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d8148140 msgr2=0x7ff3d81485c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:10:11.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.662+0000 7ff3d659c640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d8148140 0x7ff3d81485c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:11.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.662+0000 7ff3d659c640 1 -- 172.21.15.135:0/1106936946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d8148b00 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.662+0000 7ff3d659c640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 0x7ff3d81ebda0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000ddc0 tx=0x7ff3d000b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:11.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.663+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0014070 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.663+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d0006e50 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.663+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0008040 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.663+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d815bd50 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.663+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d815ba70 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.665+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d80507a0 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.666+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3d0039460 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.667+0000 7ff3c6ffd640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff3b806c9c0 0x7ff3b806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:11.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.667+0000 7ff3d5d9b640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff3b806c9c0 0x7ff3b806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:11.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.667+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7ff3d008c850 con 0x7ff3d81eb9c0 2024-04-26T22:10:11.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.668+0000 7ff3d5d9b640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff3b806c9c0 0x7ff3b806ee80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff3d814d9d0 tx=0x7ff3cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:11.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:11.673+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3d005a480 con 0x7ff3d81eb9c0 2024-04-26T22:10:12.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.036+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d80c3fb0 con 0x7ff3d81eb9c0 2024-04-26T22:10:12.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.037+0000 7ff3c6ffd640 1 -- 172.21.15.135:0/1106936946 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6500 (secure 0 0 0) 0x7ff3d0059d80 con 0x7ff3d81eb9c0 2024-04-26T22:10:12.038 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:12.038 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1333118834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1333118834},{"type":"v1","addr":"172.21.15.135:6837","nonce":1333118834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:10:12.038 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 81 2024-04-26T22:10:12.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff3b806c9c0 msgr2=0x7ff3b806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff3b806c9c0 0x7ff3b806ee80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff3d814d9d0 tx=0x7ff3cc008040 comp rx=0 tx=0).stop 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 msgr2=0x7ff3d81ebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 0x7ff3d81ebda0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000ddc0 tx=0x7ff3d000b700 comp rx=0 tx=0).stop 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 shutdown_connections 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ff3b806c9c0 0x7ff3b806ee80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d8073010 0x7ff3d8073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d8148140 0x7ff3d81485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 --2- 172.21.15.135:0/1106936946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d81eb9c0 0x7ff3d81ebda0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 >> 172.21.15.135:0/1106936946 conn(0x7ff3d806c310 msgr2=0x7ff3d80c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 shutdown_connections 2024-04-26T22:10:12.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.039+0000 7ff3dcc74640 1 -- 172.21.15.135:0/1106936946 wait complete. 2024-04-26T22:10:12.051 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-04-26T22:10:12.051 INFO:tasks.mds_thrash.fs.[b]:stat['max_mds'] = 0 2024-04-26T22:10:12.051 INFO:tasks.mds_thrash.fs.[b]:stat['deactivate'] = 0 2024-04-26T22:10:12.051 INFO:tasks.mds_thrash.fs.[b]:stat['kill'] = 8 2024-04-26T22:10:12.052 INFO:tasks.mds_thrash:done joining 2024-04-26T22:10:12.052 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T22:10:12.061 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T22:10:12.062 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T22:10:12.062 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:10:12.126 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T22:10:12.126 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:10:12.181 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:10:12.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.317+0000 7f97186bd640 1 Processor -- start 2024-04-26T22:10:12.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.317+0000 7f97186bd640 1 -- start start 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.318+0000 7f97186bd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9710154160 0x7f9710156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.318+0000 7f97186bd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97100c8df0 con 0x7f971015a1a0 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.318+0000 7f97186bd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9710154540 con 0x7f97100ca5a0 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.318+0000 7f97186bd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f97100c9100 con 0x7f9710154160 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.318+0000 7f9716c5a640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f971015a1a0 0x7f97100c9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47560/0 (socket says 172.21.15.135:47560) 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.318+0000 7f9716c5a640 1 -- 172.21.15.135:0/220016893 learned_addr learned my addr 172.21.15.135:0/220016893 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9716459640 1 --2- 172.21.15.135:0/220016893 >> v2:172.21.15.139:3300/0 conn(0x7f9710154160 0x7f9710156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809724461 0 0) 0x7f9708001110 con 0x7f971015a1a0 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96fc003520 con 0x7f971015a1a0 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2298679563 0 0) 0x7f96fc003520 con 0x7f971015a1a0 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 >> v2:172.21.15.139:3300/0 conn(0x7f9710154160 msgr2=0x7f9710156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:12.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 --2- 172.21.15.135:0/220016893 >> v2:172.21.15.139:3300/0 conn(0x7f9710154160 0x7f9710156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 >> v1:172.21.15.139:6789/0 conn(0x7f97100ca5a0 legacy=0x7f97100c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971015a910 con 0x7f971015a1a0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f9708001840 con 0x7f971015a1a0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 >> v1:172.21.15.135:6789/0 conn(0x7f971015a1a0 legacy=0x7f97100c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc0039b0 0x7f96fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96fc004800 0x7f96fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96fc0051c0 0x7f96fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97100c9100 con 0x7f96fc004800 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc00c060 con 0x7f96fc0051c0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc00c260 con 0x7f96fc0039b0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715c58640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96fc004800 0x7f96fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9716c5a640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96fc0051c0 0x7f96fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9716459640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc0039b0 0x7f96fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715c58640 1 -- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc0039b0 msgr2=0x7f96fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715c58640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc0039b0 0x7f96fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715c58640 1 -- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96fc0051c0 msgr2=0x7f96fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715c58640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96fc0051c0 0x7f96fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9715c58640 1 -- 172.21.15.135:0/220016893 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc003520 con 0x7f96fc004800 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.319+0000 7f9716c5a640 1 -- 172.21.15.135:0/220016893 reap_dead start 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.320+0000 7f9715c58640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96fc004800 0x7f96fc004c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9700001720 tx=0x7f96fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaa4f025baa81d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.320+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700001370 con 0x7f96fc004800 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.320+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97000052d0 con 0x7f96fc004800 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.320+0000 7f9715457640 1 -- 172.21.15.135:0/220016893 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970002ab90 con 0x7f96fc004800 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 -- 172.21.15.135:0/220016893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96fc004800 msgr2=0x7f96fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96fc004800 0x7f96fc004c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9700001720 tx=0x7f96fc005830 comp rx=0 tx=0).stop 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 -- 172.21.15.135:0/220016893 shutdown_connections 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 --2- 172.21.15.135:0/220016893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96fc004800 0x7f96fc004c80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 -- 172.21.15.135:0/220016893 >> 172.21.15.135:0/220016893 conn(0x7f97100ae010 msgr2=0x7f97100ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:12.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 -- 172.21.15.135:0/220016893 shutdown_connections 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.321+0000 7f97186bd640 1 -- 172.21.15.135:0/220016893 wait complete. 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 Processor -- start 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 -- start start 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc004800 0x7f97101e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 0x7f9710148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9710148c90 0x7f97101f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f971015a5b0 con 0x7f9710149980 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f971015aed0 con 0x7f9710148c90 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f97186bd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9710156bb0 con 0x7f96fc004800 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.322+0000 7f9715c58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 0x7f9710148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 0x7f9710148750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51270/0 (socket says 172.21.15.135:51270) 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 -- 172.21.15.135:0/3795810340 learned_addr learned my addr 172.21.15.135:0/3795810340 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9716459640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc004800 0x7f97101e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9716c5a640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9710148c90 0x7f97101f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 -- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc004800 msgr2=0x7f97101e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc004800 0x7f97101e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 -- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9710148c90 msgr2=0x7f97101f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9710148c90 0x7f97101f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 -- 172.21.15.135:0/3795810340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97100ac100 con 0x7f9710149980 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9715c58640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 0x7f9710148750 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9700001840 tx=0x7f9700001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700007a50 con 0x7f9710149980 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9700041070 con 0x7f9710149980 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97100adaf0 con 0x7f9710149980 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700005480 con 0x7f9710149980 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.323+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97100ac3e0 con 0x7f9710149980 2024-04-26T22:10:12.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.324+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9710076650 con 0x7f9710149980 2024-04-26T22:10:12.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.325+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97000067d0 con 0x7f9710149980 2024-04-26T22:10:12.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.325+0000 7f9706ffd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f96f406c8f0 0x7f96f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.325+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f97000ac540 con 0x7f9710149980 2024-04-26T22:10:12.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.325+0000 7f9716459640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f96f406c8f0 0x7f96f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.325+0000 7f9716459640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f96f406c8f0 0x7f96f406edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f970c014510 tx=0x7f970c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.329+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9700007680 con 0x7f9710149980 2024-04-26T22:10:12.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.505+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9710157b50 con 0x7f9710149980 2024-04-26T22:10:12.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.507+0000 7f9706ffd640 1 -- 172.21.15.135:0/3795810340 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+22533 (secure 0 0 0) 0x7f9700079a20 con 0x7f9710149980 2024-04-26T22:10:12.508 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:12.508 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":40,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:10:00.929237+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6837/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6836/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6834/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6835/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.135:6837/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.139:6832/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6835/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6834/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6835/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.139:6833/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6836/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.139:6834/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6834/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.139:6835/302589348":"2024-04-27T22:05:52.818178+0000","172.21.15.139:6832/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.139:6835/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6833/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.135:6836/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6836/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.135:6837/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6835/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.135:6837/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.135:6835/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6834/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6835/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6834/302589348":"2024-04-27T22:05:52.818178+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f96f406c8f0 msgr2=0x7f96f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f96f406c8f0 0x7f96f406edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f970c014510 tx=0x7f970c00b040 comp rx=0 tx=0).stop 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 msgr2=0x7f9710148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 0x7f9710148750 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9700001840 tx=0x7f9700001120 comp rx=0 tx=0).stop 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 shutdown_connections 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f96f406c8f0 0x7f96f406edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9710148c90 0x7f97101f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9710149980 0x7f9710148750 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 --2- 172.21.15.135:0/3795810340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96fc004800 0x7f97101e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.510 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 >> 172.21.15.135:0/3795810340 conn(0x7f97100ae010 msgr2=0x7f9710154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:12.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 shutdown_connections 2024-04-26T22:10:12.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.509+0000 7f97186bd640 1 -- 172.21.15.135:0/3795810340 wait complete. 2024-04-26T22:10:12.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:10:12.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:10:12.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.623+0000 7f56de6ea640 1 Processor -- start 2024-04-26T22:10:12.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.623+0000 7f56de6ea640 1 -- start start 2024-04-26T22:10:12.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56de6ea640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56d81541d0 0x7f56d80c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56de6ea640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d80c3fc0 con 0x7f56d8070490 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56de6ea640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d80c3500 con 0x7f56d815a3c0 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56de6ea640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f56d80c37e0 con 0x7f56d81541d0 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d7fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56d81541d0 0x7f56d80c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d77fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f56d8070490 0x7f56d80c7420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47574/0 (socket says 172.21.15.135:47574) 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d77fe640 1 -- 172.21.15.135:0/2363501883 learned_addr learned my addr 172.21.15.135:0/2363501883 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d7fff640 1 -- 172.21.15.135:0/2363501883 >> v1:172.21.15.139:6789/0 conn(0x7f56d815a3c0 legacy=0x7f56d815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d7fff640 1 -- 172.21.15.135:0/2363501883 >> v1:172.21.15.135:6789/0 conn(0x7f56d8070490 legacy=0x7f56d80c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d6ffd640 1 -- 172.21.15.135:0/2363501883 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933821491 0 0) 0x7f56d80c3fc0 con 0x7f56d8070490 2024-04-26T22:10:12.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.624+0000 7f56d7fff640 1 -- 172.21.15.135:0/2363501883 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d80c3ac0 con 0x7f56d81541d0 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.625+0000 7f56d7fff640 1 --2- 172.21.15.135:0/2363501883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81541d0 0x7f56d80c8400 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f56c80097a0 tx=0x7f56c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7282fb19205c1dee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.625+0000 7f56d6ffd640 1 -- 172.21.15.135:0/2363501883 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8037070 con 0x7f56d81541d0 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.625+0000 7f56d6ffd640 1 -- 172.21.15.135:0/2363501883 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c8032440 con 0x7f56d81541d0 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.625+0000 7f56d6ffd640 1 -- 172.21.15.135:0/2363501883 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8032810 con 0x7f56d81541d0 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.625+0000 7f56de6ea640 1 -- 172.21.15.135:0/2363501883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81541d0 msgr2=0x7f56d80c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.625+0000 7f56de6ea640 1 --2- 172.21.15.135:0/2363501883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81541d0 0x7f56d80c8400 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f56c80097a0 tx=0x7f56c802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.626+0000 7f56de6ea640 1 -- 172.21.15.135:0/2363501883 shutdown_connections 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.626+0000 7f56de6ea640 1 --2- 172.21.15.135:0/2363501883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81541d0 0x7f56d80c8400 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.626+0000 7f56de6ea640 1 -- 172.21.15.135:0/2363501883 >> 172.21.15.135:0/2363501883 conn(0x7f56d80c8c40 msgr2=0x7f56d80c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.626+0000 7f56de6ea640 1 -- 172.21.15.135:0/2363501883 shutdown_connections 2024-04-26T22:10:12.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.626+0000 7f56de6ea640 1 -- 172.21.15.135:0/2363501883 wait complete. 2024-04-26T22:10:12.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 Processor -- start 2024-04-26T22:10:12.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 -- start start 2024-04-26T22:10:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 0x7f56d814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56d81541d0 0x7f56d81ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56d815a3c0 0x7f56d80ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56d80c7cb0 con 0x7f56d8070490 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56d80c7b30 con 0x7f56d815a3c0 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.627+0000 7f56de6ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f56d815aef0 con 0x7f56d81541d0 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 0x7f56d814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 0x7f56d814c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51284/0 (socket says 172.21.15.135:51284) 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 -- 172.21.15.135:0/599471768 learned_addr learned my addr 172.21.15.135:0/599471768 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 -- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56d81541d0 msgr2=0x7f56d81ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56d81541d0 0x7f56d81ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 -- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56d815a3c0 msgr2=0x7f56d80ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56dcc87640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56d815a3c0 0x7f56d80ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56d815a3c0 0x7f56d80ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 -- 172.21.15.135:0/599471768 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d806e0c0 con 0x7f56d8070490 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56dcc87640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56d815a3c0 0x7f56d80ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d7fff640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 0x7f56d814c630 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f56c80097a0 tx=0x7f56c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8037050 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c8038040 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d81ec6f0 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8004490 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.628+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d81ec970 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.629+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d8076690 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.630+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56c8030900 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.630+0000 7f56d4ff9640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f56c406c8f0 0x7f56c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.630+0000 7f56d77fe640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f56c406c8f0 0x7f56c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.630+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f56c80acfe0 con 0x7f56d8070490 2024-04-26T22:10:12.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.631+0000 7f56d77fe640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f56c406c8f0 0x7f56c406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f56cc007fd0 tx=0x7f56cc007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.634+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56d81ec6f0 con 0x7f56d8070490 2024-04-26T22:10:12.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.807+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56d80aa580 con 0x7f56d8070490 2024-04-26T22:10:12.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.807+0000 7f56d4ff9640 1 -- 172.21.15.135:0/599471768 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f56d806e0c0 con 0x7f56d8070490 2024-04-26T22:10:12.809 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f56c406c8f0 msgr2=0x7f56c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f56c406c8f0 0x7f56c406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f56cc007fd0 tx=0x7f56cc007900 comp rx=0 tx=0).stop 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 msgr2=0x7f56d814c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 0x7f56d814c630 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f56c80097a0 tx=0x7f56c80046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 shutdown_connections 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f56c406c8f0 0x7f56c406edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56d815a3c0 0x7f56d80ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56d81541d0 0x7f56d81ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 --2- 172.21.15.135:0/599471768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d8070490 0x7f56d814c630 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.809+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 >> 172.21.15.135:0/599471768 conn(0x7f56d80c8c40 msgr2=0x7f56d8152560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.810+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 shutdown_connections 2024-04-26T22:10:12.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.810+0000 7f56de6ea640 1 -- 172.21.15.135:0/599471768 wait complete. 2024-04-26T22:10:12.822 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:10:12.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 Processor -- start 2024-04-26T22:10:12.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 -- start start 2024-04-26T22:10:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f841c0ca5a0 0x7f841c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c15a1a0 0x7f841c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841c0c8df0 con 0x7f841c154160 2024-04-26T22:10:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f841c0c9100 con 0x7f841c0ca5a0 2024-04-26T22:10:12.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f8422215640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f841c154540 con 0x7f841c15a1a0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f841affd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f841c0ca5a0 0x7f841c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f841b7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f841c154160 0x7f841c0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47580/0 (socket says 172.21.15.135:47580) 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.922+0000 7f841b7fe640 1 -- 172.21.15.135:0/3426255876 learned_addr learned my addr 172.21.15.135:0/3426255876 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.923+0000 7f841bfff640 1 --2- 172.21.15.135:0/3426255876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c15a1a0 0x7f841c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.923+0000 7f841affd640 1 -- 172.21.15.135:0/3426255876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c15a1a0 msgr2=0x7f841c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.923+0000 7f841affd640 1 --2- 172.21.15.135:0/3426255876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c15a1a0 0x7f841c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.923+0000 7f841affd640 1 -- 172.21.15.135:0/3426255876 >> v1:172.21.15.135:6789/0 conn(0x7f841c154160 legacy=0x7f841c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.923+0000 7f841a7fc640 1 -- 172.21.15.135:0/3426255876 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452758639 0 0) 0x7f840c001110 con 0x7f841c154160 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.923+0000 7f841affd640 1 -- 172.21.15.135:0/3426255876 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c15a910 con 0x7f841c0ca5a0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f841affd640 1 --2- 172.21.15.135:0/3426255876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c156bb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f84100097a0 tx=0x7f841002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878d726790705f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f841a7fc640 1 -- 172.21.15.135:0/3426255876 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410037070 con 0x7f841c0ca5a0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f841a7fc640 1 -- 172.21.15.135:0/3426255876 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8410032440 con 0x7f841c0ca5a0 2024-04-26T22:10:12.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f841a7fc640 1 -- 172.21.15.135:0/3426255876 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410032810 con 0x7f841c0ca5a0 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 -- 172.21.15.135:0/3426255876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 msgr2=0x7f841c156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 --2- 172.21.15.135:0/3426255876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c156bb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f84100097a0 tx=0x7f841002a180 comp rx=0 tx=0).stop 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 -- 172.21.15.135:0/3426255876 shutdown_connections 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 --2- 172.21.15.135:0/3426255876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c15a1a0 0x7f841c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 --2- 172.21.15.135:0/3426255876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c156bb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 -- 172.21.15.135:0/3426255876 >> 172.21.15.135:0/3426255876 conn(0x7f841c0ae010 msgr2=0x7f841c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 -- 172.21.15.135:0/3426255876 shutdown_connections 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.924+0000 7f8422215640 1 -- 172.21.15.135:0/3426255876 wait complete. 2024-04-26T22:10:12.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.925+0000 7f8422215640 1 Processor -- start 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 -- start start 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c154160 0x7f841c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841c15a1a0 0x7f841c1495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f841c15a5f0 con 0x7f841c0ca5a0 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f841c15aed0 con 0x7f841c15a1a0 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f8422215640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f841c0c7510 con 0x7f841c154160 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f841b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f841b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c1565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51308/0 (socket says 172.21.15.135:51308) 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f841b7fe640 1 -- 172.21.15.135:0/1805582447 learned_addr learned my addr 172.21.15.135:0/1805582447 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:12.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f841affd640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c154160 0x7f841c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.926+0000 7f841bfff640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841c15a1a0 0x7f841c1495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841b7fe640 1 -- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c154160 msgr2=0x7f841c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841b7fe640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c154160 0x7f841c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841b7fe640 1 -- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841c15a1a0 msgr2=0x7f841c1495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841b7fe640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841c15a1a0 0x7f841c1495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841b7fe640 1 -- 172.21.15.135:0/1805582447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c156c50 con 0x7f841c0ca5a0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841affd640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c154160 0x7f841c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841bfff640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841c15a1a0 0x7f841c1495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f841b7fe640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c1565b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f840c00c7b0 tx=0x7f840c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c018070 con 0x7f841c0ca5a0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c0072d0 con 0x7f841c0ca5a0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c0047e0 con 0x7f841c0ca5a0 2024-04-26T22:10:12.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c1ecaf0 con 0x7f841c0ca5a0 2024-04-26T22:10:12.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.927+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c1ed0b0 con 0x7f841c0ca5a0 2024-04-26T22:10:12.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.928+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c076650 con 0x7f841c0ca5a0 2024-04-26T22:10:12.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.928+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f840c01b070 con 0x7f841c0ca5a0 2024-04-26T22:10:12.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.929+0000 7f8403fff640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f840806c920 0x7f840806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:12.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.929+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f840c08b690 con 0x7f841c0ca5a0 2024-04-26T22:10:12.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.929+0000 7f841affd640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f840806c920 0x7f840806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:12.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.929+0000 7f841affd640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f840806c920 0x7f840806ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8410034000 tx=0x7f841002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:12.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:12.933+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f841c0c8df0 con 0x7f841c0ca5a0 2024-04-26T22:10:13.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.112+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f841c0507a0 con 0x7f841c0ca5a0 2024-04-26T22:10:13.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.112+0000 7f8403fff640 1 -- 172.21.15.135:0/1805582447 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v40) ==== 99+0+11 (secure 0 0 0) 0x7f840c0592e0 con 0x7f841c0ca5a0 2024-04-26T22:10:13.113 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:10:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f840806c920 msgr2=0x7f840806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f840806c920 0x7f840806ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8410034000 tx=0x7f841002e000 comp rx=0 tx=0).stop 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 msgr2=0x7f841c1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c1565b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f840c00c7b0 tx=0x7f840c00cc70 comp rx=0 tx=0).stop 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 shutdown_connections 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f840806c920 0x7f840806ede0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841c15a1a0 0x7f841c1495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f841c154160 0x7f841c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 --2- 172.21.15.135:0/1805582447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f841c0ca5a0 0x7f841c1565b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 >> 172.21.15.135:0/1805582447 conn(0x7f841c0ae010 msgr2=0x7f841c156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 shutdown_connections 2024-04-26T22:10:13.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.114+0000 7f8422215640 1 -- 172.21.15.135:0/1805582447 wait complete. 2024-04-26T22:10:13.126 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:10:13.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.226+0000 7f4f81719640 1 Processor -- start 2024-04-26T22:10:13.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.226+0000 7f4f81719640 1 -- start start 2024-04-26T22:10:13.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f81719640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c154160 0x7f4f7c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f81719640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f7c15a1a0 0x7f4f7c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f81719640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c0c8df0 con 0x7f4f7c0ca5a0 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f81719640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7c0c9100 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f81719640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c154540 con 0x7f4f7c154160 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f7c15a1a0 0x7f4f7c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f7c15a1a0 0x7f4f7c0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51324/0 (socket says 172.21.15.135:51324) 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/3663777133 learned_addr learned my addr 172.21.15.135:0/3663777133 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7affd640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c154160 0x7f4f7c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/3663777133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c154160 msgr2=0x7f4f7c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c154160 0x7f4f7c156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/3663777133 >> v1:172.21.15.139:6789/0 conn(0x7f4f7c0ca5a0 legacy=0x7f4f7c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/3663777133 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c15a910 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.227+0000 7f4f7affd640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c154160 0x7f4f7c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f7b7fe640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c0c9590 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009870 tx=0x7f4f7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fadf81bae427b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f79ffb640 1 -- 172.21.15.135:0/3663777133 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74037070 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f79ffb640 1 -- 172.21.15.135:0/3663777133 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74032440 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f79ffb640 1 -- 172.21.15.135:0/3663777133 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74032810 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f81719640 1 -- 172.21.15.135:0/3663777133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 msgr2=0x7f4f7c0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f81719640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c0c9590 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009870 tx=0x7f4f7402a210 comp rx=0 tx=0).stop 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f81719640 1 -- 172.21.15.135:0/3663777133 shutdown_connections 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f81719640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c0c9590 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f81719640 1 --2- 172.21.15.135:0/3663777133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c154160 0x7f4f7c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.228+0000 7f4f81719640 1 -- 172.21.15.135:0/3663777133 >> 172.21.15.135:0/3663777133 conn(0x7f4f7c0ae010 msgr2=0x7f4f7c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.229+0000 7f4f81719640 1 -- 172.21.15.135:0/3663777133 shutdown_connections 2024-04-26T22:10:13.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.229+0000 7f4f81719640 1 -- 172.21.15.135:0/3663777133 wait complete. 2024-04-26T22:10:13.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 Processor -- start 2024-04-26T22:10:13.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 -- start start 2024-04-26T22:10:13.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c0ca5a0 0x7f4f7c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f7c154160 0x7f4f7c1f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c1f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c15a630 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c15aed0 con 0x7f4f7c154160 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f81719640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c0c7510 con 0x7f4f7c0ca5a0 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c1f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.230+0000 7f4f7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c1f06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51336/0 (socket says 172.21.15.135:51336) 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/2118315522 learned_addr learned my addr 172.21.15.135:0/2118315522 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7affd640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c0ca5a0 0x7f4f7c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7a7fc640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f7c154160 0x7f4f7c1f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c0ca5a0 msgr2=0x7f4f7c151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c0ca5a0 0x7f4f7c151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f7c154160 msgr2=0x7f4f7c1f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f7c154160 0x7f4f7c1f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 -- 172.21.15.135:0/2118315522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c143e80 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7a7fc640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f7c154160 0x7f4f7c1f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:13.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f7b7fe640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c1f06f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4f7402a250 tx=0x7f4f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74041020 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74030990 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c1439a0 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74037040 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.231+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c159b30 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.232+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c143790 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.232+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f74044020 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.233+0000 7f4f637fe640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f6806c8f0 0x7f4f6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.233+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f4f740acc80 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.233+0000 7f4f7affd640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f6806c8f0 0x7f4f6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.233+0000 7f4f7affd640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f6806c8f0 0x7f4f6806edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00d620 tx=0x7f4f6c007810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.238+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f7404b030 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.409+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f4f7c076650 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.410+0000 7f4f637fe640 1 -- 172.21.15.135:0/2118315522 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v40) ==== 95+0+11 (secure 0 0 0) 0x7f4f7c0c8df0 con 0x7f4f7c15a1a0 2024-04-26T22:10:13.411 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.411+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f6806c8f0 msgr2=0x7f4f6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.411+0000 7f4f81719640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f6806c8f0 0x7f4f6806edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00d620 tx=0x7f4f6c007810 comp rx=0 tx=0).stop 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.411+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 msgr2=0x7f4f7c1f06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.411+0000 7f4f81719640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c1f06f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4f7402a250 tx=0x7f4f740046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 shutdown_connections 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f6806c8f0 0x7f4f6806edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f7c15a1a0 0x7f4f7c1f06f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f7c154160 0x7f4f7c1f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 --2- 172.21.15.135:0/2118315522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7c0ca5a0 0x7f4f7c151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 >> 172.21.15.135:0/2118315522 conn(0x7f4f7c0ae010 msgr2=0x7f4f7c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 shutdown_connections 2024-04-26T22:10:13.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.412+0000 7f4f81719640 1 -- 172.21.15.135:0/2118315522 wait complete. 2024-04-26T22:10:13.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:10:13.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.523+0000 7fee426b0640 1 Processor -- start 2024-04-26T22:10:13.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.523+0000 7fee426b0640 1 -- start start 2024-04-26T22:10:13.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.523+0000 7fee426b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fee3c06e640 0x7fee3c0ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee426b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 0x7fee3c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee426b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee3c153d90 con 0x7fee3c06f760 2024-04-26T22:10:13.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee426b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fee3c0c2e50 con 0x7fee3c06e640 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee426b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c0c3130 con 0x7fee3c0c4720 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fee3c06e640 0x7fee3c0ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee40c4d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 0x7fee3c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fee3c06f760 0x7fee3c0c3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:43946/0 (socket says 172.21.15.135:43946) 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee40c4d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 0x7fee3c1500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:60256/0 (socket says 172.21.15.135:60256) 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fee3c06e640 0x7fee3c0ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55876/0 (socket says 172.21.15.135:55876) 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3b7fe640 1 -- 172.21.15.135:0/3474594914 learned_addr learned my addr 172.21.15.135:0/3474594914 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3bfff640 1 -- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 msgr2=0x7fee3c1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3bfff640 1 --2- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 0x7fee3c1500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3bfff640 1 -- 172.21.15.135:0/3474594914 >> v1:172.21.15.139:6789/0 conn(0x7fee3c06f760 legacy=0x7fee3c0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3bfff640 1 -- 172.21.15.135:0/3474594914 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c0c3410 con 0x7fee3c06e640 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee3affd640 1 -- 172.21.15.135:0/3474594914 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544307404 0 0) 0x7fee3c153d90 con 0x7fee3c06f760 2024-04-26T22:10:13.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.524+0000 7fee40c4d640 1 --2- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 0x7fee3c1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.525+0000 7fee3bfff640 1 --2- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c06e640 0x7fee3c0ab380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fee300097d0 tx=0x7fee3002a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0280e914b0c20fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.525+0000 7fee3affd640 1 -- 172.21.15.135:0/3474594914 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30037070 con 0x7fee3c06e640 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.525+0000 7fee3affd640 1 -- 172.21.15.135:0/3474594914 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3002adb0 con 0x7fee3c06e640 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.525+0000 7fee3affd640 1 -- 172.21.15.135:0/3474594914 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30032660 con 0x7fee3c06e640 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 -- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c06e640 msgr2=0x7fee3c0ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 --2- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c06e640 0x7fee3c0ab380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fee300097d0 tx=0x7fee3002a0a0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 -- 172.21.15.135:0/3474594914 shutdown_connections 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 --2- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c0c4720 0x7fee3c1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 --2- 172.21.15.135:0/3474594914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c06e640 0x7fee3c0ab380 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 -- 172.21.15.135:0/3474594914 >> 172.21.15.135:0/3474594914 conn(0x7fee3c06c310 msgr2=0x7fee3c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 -- 172.21.15.135:0/3474594914 shutdown_connections 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.526+0000 7fee426b0640 1 -- 172.21.15.135:0/3474594914 wait complete. 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.527+0000 7fee426b0640 1 Processor -- start 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.527+0000 7fee426b0640 1 -- start start 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 0x7fee3c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c06f760 0x7fee3c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c0c4720 0x7fee3c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c06eba0 con 0x7fee3c06e640 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c06ea20 con 0x7fee3c0c4720 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c1505f0 con 0x7fee3c06f760 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 0x7fee3c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 0x7fee3c14c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51350/0 (socket says 172.21.15.135:51350) 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 -- 172.21.15.135:0/3152017405 learned_addr learned my addr 172.21.15.135:0/3152017405 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:13.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee40c4d640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c0c4720 0x7fee3c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3b7fe640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c06f760 0x7fee3c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 -- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c06f760 msgr2=0x7fee3c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c06f760 0x7fee3c156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 -- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c0c4720 msgr2=0x7fee3c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c0c4720 0x7fee3c14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 -- 172.21.15.135:0/3152017405 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c154740 con 0x7fee3c06e640 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee3bfff640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 0x7fee3c14c830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fee30035040 tx=0x7fee300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30037040 con 0x7fee3c06e640 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee30004030 con 0x7fee3c06e640 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c154a20 con 0x7fee3c06e640 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30004400 con 0x7fee3c06e640 2024-04-26T22:10:13.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.528+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c156710 con 0x7fee3c06e640 2024-04-26T22:10:13.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.529+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c0507a0 con 0x7fee3c06e640 2024-04-26T22:10:13.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.530+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee30003610 con 0x7fee3c06e640 2024-04-26T22:10:13.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.530+0000 7fee38ff9640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fee2806c8f0 0x7fee2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.530+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fee300acf90 con 0x7fee3c06e640 2024-04-26T22:10:13.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.530+0000 7fee3b7fe640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fee2806c8f0 0x7fee2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.531+0000 7fee3b7fe640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fee2806c8f0 0x7fee2806edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee3c14e980 tx=0x7fee2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.534+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee3007ad00 con 0x7fee3c06e640 2024-04-26T22:10:13.625 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:10:13.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.714+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7fee3c14a800 con 0x7fee3c06e640 2024-04-26T22:10:13.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.715+0000 7fee38ff9640 1 -- 172.21.15.135:0/3152017405 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v40) ==== 99+0+11 (secure 0 0 0) 0x7fee3007a600 con 0x7fee3c06e640 2024-04-26T22:10:13.716 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fee2806c8f0 msgr2=0x7fee2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fee2806c8f0 0x7fee2806edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee3c14e980 tx=0x7fee2c009040 comp rx=0 tx=0).stop 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 msgr2=0x7fee3c14c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 0x7fee3c14c830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fee30035040 tx=0x7fee300046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 shutdown_connections 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fee2806c8f0 0x7fee2806edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3c0c4720 0x7fee3c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee3c06f760 0x7fee3c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 --2- 172.21.15.135:0/3152017405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee3c06e640 0x7fee3c14c830 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 >> 172.21.15.135:0/3152017405 conn(0x7fee3c06c310 msgr2=0x7fee3c0c7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 shutdown_connections 2024-04-26T22:10:13.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.717+0000 7fee426b0640 1 -- 172.21.15.135:0/3152017405 wait complete. 2024-04-26T22:10:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.722+0000 7f56ae062640 1 Processor -- start 2024-04-26T22:10:13.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.722+0000 7f56ae062640 1 -- start start 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56ae062640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f56a80ab3f0 0x7f56a80c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56ae062640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a80c3810 con 0x7f56a81570d0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56ae062640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a80c3ab0 con 0x7f56a80c52c0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56ae062640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f56a80c4330 con 0x7f56a80ab3f0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56a77fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f56a81570d0 0x7f56a8150a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47596/0 (socket says 172.21.15.135:47596) 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56a77fe640 1 -- 172.21.15.135:0/4027830471 learned_addr learned my addr 172.21.15.135:0/4027830471 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f569ffff640 1 --2- 172.21.15.135:0/4027830471 >> v2:172.21.15.139:3300/0 conn(0x7f56a80ab3f0 0x7f56a80c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097397481 0 0) 0x7f56a80c3810 con 0x7f56a81570d0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.723+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5694003520 con 0x7f56a81570d0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1773942238 0 0) 0x7f5694003520 con 0x7f56a81570d0 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 >> v2:172.21.15.139:3300/0 conn(0x7f56a80ab3f0 msgr2=0x7f56a80c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 --2- 172.21.15.135:0/4027830471 >> v2:172.21.15.139:3300/0 conn(0x7f56a80ab3f0 0x7f56a80c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 >> v1:172.21.15.139:6789/0 conn(0x7f56a80c52c0 legacy=0x7f56a81574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a80c4580 con 0x7f56a81570d0 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f5698001670 con 0x7f56a81570d0 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 >> v1:172.21.15.135:6789/0 conn(0x7f56a81570d0 legacy=0x7f56a8150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940039b0 0x7f5694003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56940052e0 0x7f569400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80c4330 con 0x7f56940052e0 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f569400c1b0 con 0x7f56940039b0 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f569400c3b0 con 0x7f5694004800 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a7fff640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56940052e0 0x7f569400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f569ffff640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a77fe640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940039b0 0x7f5694003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a7fff640 1 -- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5694004800 msgr2=0x7f5694004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a7fff640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a7fff640 1 -- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940039b0 msgr2=0x7f5694003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a7fff640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940039b0 0x7f5694003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.724+0000 7f56a7fff640 1 -- 172.21.15.135:0/4027830471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5694003520 con 0x7f56940052e0 2024-04-26T22:10:13.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.725+0000 7f56a7fff640 1 -- 172.21.15.135:0/4027830471 reap_dead start 2024-04-26T22:10:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.725+0000 7f56a7fff640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56940052e0 0x7f569400b9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001e20 tx=0x7f5694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947a0bf35f6772bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.725+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0001a70 con 0x7f56940052e0 2024-04-26T22:10:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.725+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a002ae40 con 0x7f56940052e0 2024-04-26T22:10:13.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.725+0000 7f56a6ffd640 1 -- 172.21.15.135:0/4027830471 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0003bb0 con 0x7f56940052e0 2024-04-26T22:10:13.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 -- 172.21.15.135:0/4027830471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56940052e0 msgr2=0x7f569400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56940052e0 0x7f569400b9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001e20 tx=0x7f5694005830 comp rx=0 tx=0).stop 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 -- 172.21.15.135:0/4027830471 shutdown_connections 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 --2- 172.21.15.135:0/4027830471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56940052e0 0x7f569400b9c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 -- 172.21.15.135:0/4027830471 >> 172.21.15.135:0/4027830471 conn(0x7f56a806c310 msgr2=0x7f56a806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 -- 172.21.15.135:0/4027830471 shutdown_connections 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.726+0000 7f56ae062640 1 -- 172.21.15.135:0/4027830471 wait complete. 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.727+0000 7f56ae062640 1 Processor -- start 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.727+0000 7f56ae062640 1 -- start start 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56ae062640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940052e0 0x7f56a80c1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56ae062640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 0x7f56a81e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56ae062640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56a81e6760 0x7f56a81ec870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56ae062640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80abbe0 con 0x7f56a8156840 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56ae062640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80aba60 con 0x7f56940052e0 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56ae062640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f56a80c4d80 con 0x7f56a81e6760 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 0x7f56a81e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 0x7f56a81e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51370/0 (socket says 172.21.15.135:51370) 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 -- 172.21.15.135:0/3168115835 learned_addr learned my addr 172.21.15.135:0/3168115835 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56a77fe640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940052e0 0x7f56a80c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56a7fff640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56a81e6760 0x7f56a81ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 -- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56a81e6760 msgr2=0x7f56a81ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56a81e6760 0x7f56a81ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 -- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940052e0 msgr2=0x7f56a80c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940052e0 0x7f56a80c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 -- 172.21.15.135:0/3168115835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a81e6be0 con 0x7f56a8156840 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f569ffff640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 0x7f56a81e6220 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f56900118e0 tx=0x7f5690011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569000fa30 con 0x7f56a8156840 2024-04-26T22:10:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.728+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569000fbd0 con 0x7f56a8156840 2024-04-26T22:10:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.729+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a81f0660 con 0x7f56a8156840 2024-04-26T22:10:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.729+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690014440 con 0x7f56a8156840 2024-04-26T22:10:13.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.729+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a8156360 con 0x7f56a8156840 2024-04-26T22:10:13.737 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.730+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a81ec490 con 0x7f56a8156840 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.730+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5690012040 con 0x7f56a8156840 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.730+0000 7f56a4ff9640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f568806c8f0 0x7f568806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.730+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f569008c830 con 0x7f56a8156840 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.730+0000 7f56a77fe640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f568806c8f0 0x7f568806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.731+0000 7f56a77fe640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f568806c8f0 0x7f568806edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f56980010e0 tx=0x7f56980078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.735+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5690019030 con 0x7f56a8156840 2024-04-26T22:10:13.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.838+0000 7fc89656a640 1 Processor -- start 2024-04-26T22:10:13.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.838+0000 7fc89656a640 1 -- start start 2024-04-26T22:10:13.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc89656a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc890074c10 0x7fc8900c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc89656a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc890073700 0x7fc890073b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc89656a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8901579d0 con 0x7fc89013ffb0 2024-04-26T22:10:13.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc89656a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc890070880 con 0x7fc890074c10 2024-04-26T22:10:13.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc89656a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8900ab680 con 0x7fc890073700 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc890074c10 0x7fc8900c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc894b07640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc890073700 0x7fc890073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc890074c10 0x7fc8900c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:55912/0 (socket says 172.21.15.135:55912) 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 -- 172.21.15.135:0/1339764804 learned_addr learned my addr 172.21.15.135:0/1339764804 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88ffff640 1 --1- 172.21.15.135:0/1339764804 >> v1:172.21.15.139:6789/0 conn(0x7fc89013ffb0 0x7fc8900c3b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:43970/0 (socket says 172.21.15.135:43970) 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 -- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc890073700 msgr2=0x7fc890073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 --2- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc890073700 0x7fc890073b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 -- 172.21.15.135:0/1339764804 >> v1:172.21.15.139:6789/0 conn(0x7fc89013ffb0 legacy=0x7fc8900c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88f7fe640 1 -- 172.21.15.135:0/1339764804 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8900ab960 con 0x7fc890074c10 2024-04-26T22:10:13.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.839+0000 7fc88effd640 1 -- 172.21.15.135:0/1339764804 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262298061 0 0) 0x7fc8901579d0 con 0x7fc89013ffb0 2024-04-26T22:10:13.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.840+0000 7fc88f7fe640 1 --2- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890074c10 0x7fc8900c4280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc880009870 tx=0x7fc88002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c49aa7d1f8a41eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.840+0000 7fc88effd640 1 -- 172.21.15.135:0/1339764804 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880037070 con 0x7fc890074c10 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.840+0000 7fc88effd640 1 -- 172.21.15.135:0/1339764804 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc880032440 con 0x7fc890074c10 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.840+0000 7fc88effd640 1 -- 172.21.15.135:0/1339764804 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880032810 con 0x7fc890074c10 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 -- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890074c10 msgr2=0x7fc8900c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 --2- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890074c10 0x7fc8900c4280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc880009870 tx=0x7fc88002a270 comp rx=0 tx=0).stop 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 -- 172.21.15.135:0/1339764804 shutdown_connections 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 --2- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc890073700 0x7fc890073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 --2- 172.21.15.135:0/1339764804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890074c10 0x7fc8900c4280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 -- 172.21.15.135:0/1339764804 >> 172.21.15.135:0/1339764804 conn(0x7fc8900ca2f0 msgr2=0x7fc8900ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 -- 172.21.15.135:0/1339764804 shutdown_connections 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.841+0000 7fc89656a640 1 -- 172.21.15.135:0/1339764804 wait complete. 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.842+0000 7fc89656a640 1 Processor -- start 2024-04-26T22:10:13.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.842+0000 7fc89656a640 1 -- start start 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890073700 0x7fc8901ef990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 0x7fc890153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc89013ffb0 0x7fc8900aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc890157660 con 0x7fc890074c10 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc890140390 con 0x7fc890073700 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc890074090 con 0x7fc89013ffb0 2024-04-26T22:10:13.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 0x7fc890153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 0x7fc890153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51378/0 (socket says 172.21.15.135:51378) 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 -- 172.21.15.135:0/3465950413 learned_addr learned my addr 172.21.15.135:0/3465950413 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88ffff640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890073700 0x7fc8901ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc894b07640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc89013ffb0 0x7fc8900aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 -- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc89013ffb0 msgr2=0x7fc8900aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc89013ffb0 0x7fc8900aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 -- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890073700 msgr2=0x7fc8901ef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890073700 0x7fc8901ef990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 -- 172.21.15.135:0/3465950413 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8900af620 con 0x7fc890074c10 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88f7fe640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 0x7fc890153c40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc880035040 tx=0x7fc8800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880037040 con 0x7fc890074c10 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc880004030 con 0x7fc890074c10 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.843+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc890154180 con 0x7fc890074c10 2024-04-26T22:10:13.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.844+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880004420 con 0x7fc890074c10 2024-04-26T22:10:13.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.844+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89015aca0 con 0x7fc890074c10 2024-04-26T22:10:13.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.845+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8900507a0 con 0x7fc890074c10 2024-04-26T22:10:13.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.845+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc88003a070 con 0x7fc890074c10 2024-04-26T22:10:13.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.846+0000 7fc88cff9640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc87c06c920 0x7fc87c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:13.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.846+0000 7fc88ffff640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc87c06c920 0x7fc87c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:13.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.846+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fc8800ac780 con 0x7fc890074c10 2024-04-26T22:10:13.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.846+0000 7fc88ffff640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc87c06c920 0x7fc87c06ede0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc884007fd0 tx=0x7fc884007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:13.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.850+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc880041020 con 0x7fc890074c10 2024-04-26T22:10:13.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.958+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56a80507a0 con 0x7f56a8156840 2024-04-26T22:10:13.959 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.958+0000 7f56a4ff9640 1 -- 172.21.15.135:0/3168115835 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6500 (secure 0 0 0) 0x7f569005a480 con 0x7f56a8156840 2024-04-26T22:10:13.959 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:13.961 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1333118834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1333118834},{"type":"v1","addr":"172.21.15.135:6837","nonce":1333118834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:10:13.962 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 81 2024-04-26T22:10:13.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f568806c8f0 msgr2=0x7f568806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f568806c8f0 0x7f568806edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f56980010e0 tx=0x7f56980078f0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 msgr2=0x7f56a81e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 0x7f56a81e6220 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f56900118e0 tx=0x7f5690011ce0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 shutdown_connections 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f568806c8f0 0x7f568806edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56a81e6760 0x7f56a81ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a8156840 0x7f56a81e6220 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 --2- 172.21.15.135:0/3168115835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56940052e0 0x7f56a80c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 >> 172.21.15.135:0/3168115835 conn(0x7f56a806c310 msgr2=0x7f56a8157540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 shutdown_connections 2024-04-26T22:10:13.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:13.961+0000 7f56ae062640 1 -- 172.21.15.135:0/3168115835 wait complete. 2024-04-26T22:10:13.973 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-04-26T22:10:13.974 INFO:tasks.mds_thrash.fs.[a]:waiting for 73 secs before thrashing 2024-04-26T22:10:14.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.032+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7fc890076650 con 0x7fc890074c10 2024-04-26T22:10:14.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.032+0000 7fc88cff9640 1 -- 172.21.15.135:0/3465950413 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v40) ==== 95+0+11 (secure 0 0 0) 0x7fc880079d80 con 0x7fc890074c10 2024-04-26T22:10:14.033 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:10:14.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc87c06c920 msgr2=0x7fc87c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc87c06c920 0x7fc87c06ede0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc884007fd0 tx=0x7fc884007e00 comp rx=0 tx=0).stop 2024-04-26T22:10:14.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 msgr2=0x7fc890153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 0x7fc890153c40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc880035040 tx=0x7fc8800046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 shutdown_connections 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc87c06c920 0x7fc87c06ede0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc89013ffb0 0x7fc8900aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890074c10 0x7fc890153c40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 --2- 172.21.15.135:0/3465950413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc890073700 0x7fc8901ef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 >> 172.21.15.135:0/3465950413 conn(0x7fc8900ca2f0 msgr2=0x7fc89006cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 shutdown_connections 2024-04-26T22:10:14.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.034+0000 7fc89656a640 1 -- 172.21.15.135:0/3465950413 wait complete. 2024-04-26T22:10:14.046 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:10:14.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.143+0000 7f412b220640 1 Processor -- start 2024-04-26T22:10:14.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.143+0000 7f412b220640 1 -- start start 2024-04-26T22:10:14.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f412b220640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4124074c10 0x7f41240c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f412b220640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4124073700 0x7f4124073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f412b220640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41241579d0 con 0x7f412413ffb0 2024-04-26T22:10:14.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f412b220640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4124070880 con 0x7f4124074c10 2024-04-26T22:10:14.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f412b220640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f41240ab680 con 0x7f4124073700 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f4123fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4124074c10 0x7f41240c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f4123fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4124074c10 0x7f41240c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51380/0 (socket says 172.21.15.135:51380) 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.144+0000 7f4123fff640 1 -- 172.21.15.135:0/3288076988 learned_addr learned my addr 172.21.15.135:0/3288076988 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f4123fff640 1 -- 172.21.15.135:0/3288076988 >> v2:172.21.15.139:3300/0 conn(0x7f4124073700 msgr2=0x7f4124073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f41297bd640 1 --2- 172.21.15.135:0/3288076988 >> v2:172.21.15.139:3300/0 conn(0x7f4124073700 0x7f4124073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f4123fff640 1 --2- 172.21.15.135:0/3288076988 >> v2:172.21.15.139:3300/0 conn(0x7f4124073700 0x7f4124073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f4123fff640 1 -- 172.21.15.135:0/3288076988 >> v1:172.21.15.139:6789/0 conn(0x7f412413ffb0 legacy=0x7f41240c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f4123fff640 1 -- 172.21.15.135:0/3288076988 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41240ab960 con 0x7f4124074c10 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f41297bd640 1 --2- 172.21.15.135:0/3288076988 >> v2:172.21.15.139:3300/0 conn(0x7f4124073700 0x7f4124073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f4123fff640 1 --2- 172.21.15.135:0/3288076988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124074c10 0x7f41240c4280 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f41240705d0 tx=0x7f411002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5cd7d835703f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f41237fe640 1 -- 172.21.15.135:0/3288076988 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110037070 con 0x7f4124074c10 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f41237fe640 1 -- 172.21.15.135:0/3288076988 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4110032440 con 0x7f4124074c10 2024-04-26T22:10:14.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.145+0000 7f41237fe640 1 -- 172.21.15.135:0/3288076988 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110032810 con 0x7f4124074c10 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 -- 172.21.15.135:0/3288076988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124074c10 msgr2=0x7f41240c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 --2- 172.21.15.135:0/3288076988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124074c10 0x7f41240c4280 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f41240705d0 tx=0x7f411002a230 comp rx=0 tx=0).stop 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 -- 172.21.15.135:0/3288076988 shutdown_connections 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 --2- 172.21.15.135:0/3288076988 >> v2:172.21.15.139:3300/0 conn(0x7f4124073700 0x7f4124073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 --2- 172.21.15.135:0/3288076988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124074c10 0x7f41240c4280 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 -- 172.21.15.135:0/3288076988 >> 172.21.15.135:0/3288076988 conn(0x7f41240ca2f0 msgr2=0x7f41240ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 -- 172.21.15.135:0/3288076988 shutdown_connections 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.146+0000 7f412b220640 1 -- 172.21.15.135:0/3288076988 wait complete. 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.147+0000 7f412b220640 1 Processor -- start 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.147+0000 7f412b220640 1 -- start start 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f412b220640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 0x7f412413b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f412b220640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4124074c10 0x7f41241ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f412b220640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f412413ffb0 0x7f41241ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f412b220640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4124157660 con 0x7f4124073700 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f412b220640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4124140390 con 0x7f412413ffb0 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f412b220640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4124074040 con 0x7f4124074c10 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 0x7f412413b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 0x7f412413b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51382/0 (socket says 172.21.15.135:51382) 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 -- 172.21.15.135:0/1306004770 learned_addr learned my addr 172.21.15.135:0/1306004770 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4123fff640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4124074c10 0x7f41241ecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f41297bd640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f412413ffb0 0x7f41241ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 -- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4124074c10 msgr2=0x7f41241ecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4124074c10 0x7f41241ecdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 -- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f412413ffb0 msgr2=0x7f41241ed2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f412413ffb0 0x7f41241ed2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 -- 172.21.15.135:0/1306004770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41241ec760 con 0x7f4124073700 2024-04-26T22:10:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4123fff640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4124074c10 0x7f41241ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:14.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f41297bd640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f412413ffb0 0x7f41241ed2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.148+0000 7f4128fbc640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 0x7f412413b120 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f411800b420 tx=0x7f411800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.149+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118011830 con 0x7f4124073700 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.149+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4118016070 con 0x7f4124073700 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.149+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412406e0c0 con 0x7f4124073700 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.149+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118011c10 con 0x7f4124073700 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.149+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412413b660 con 0x7f4124073700 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.150+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41241587a0 con 0x7f4124073700 2024-04-26T22:10:14.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.150+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f411801b460 con 0x7f4124073700 2024-04-26T22:10:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.150+0000 7f41217fa640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f410806c920 0x7f410806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.150+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f411808bde0 con 0x7f4124073700 2024-04-26T22:10:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.150+0000 7f4123fff640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f410806c920 0x7f410806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.151+0000 7f4123fff640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f410806c920 0x7f410806ede0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4110034000 tx=0x7f411002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.155+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4118020020 con 0x7f4124073700 2024-04-26T22:10:14.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.343+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4124156e60 con 0x7f4124073700 2024-04-26T22:10:14.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.343+0000 7f41217fa640 1 -- 172.21.15.135:0/1306004770 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 28 entries v40) ==== 81+0+1734 (secure 0 0 0) 0x7f4118059a30 con 0x7f4124073700 2024-04-26T22:10:14.344 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/3969385858 2024-04-27T22:09:52.864568+0000 2024-04-26T22:10:14.344 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/3969385858 2024-04-27T22:09:52.864568+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/1037298395 2024-04-27T22:07:37.836479+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6835/3733666635 2024-04-27T22:07:32.835007+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/2165063360 2024-04-27T22:07:07.830780+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6832/3664037452 2024-04-27T22:08:57.853641+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/3214490783 2024-04-27T22:06:12.821933+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/3214490783 2024-04-27T22:06:12.821933+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/1410523687 2024-04-27T22:01:27.768292+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6833/3664037452 2024-04-27T22:08:57.853641+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/3981978490 2024-04-27T22:01:37.770149+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/3981978490 2024-04-27T22:01:37.770149+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/3800113814 2024-04-27T22:10:00.929175+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6834/2608264407 2024-04-27T22:03:07.784944+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6834/3733666635 2024-04-27T22:07:32.835007+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6835/302589348 2024-04-27T22:05:52.818178+0000 2024-04-26T22:10:14.345 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6832/3183513686 2024-04-27T22:03:27.791314+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6835/2608264407 2024-04-27T22:03:07.784944+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6833/3183513686 2024-04-27T22:03:27.791314+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/3652200818 2024-04-27T22:04:12.801037+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/2165063360 2024-04-27T22:07:07.830780+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/3652200818 2024-04-27T22:04:12.801037+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/3800113814 2024-04-27T22:10:00.929175+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/1410523687 2024-04-27T22:01:27.768292+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/2910412510 2024-04-27T22:04:37.805449+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/2910412510 2024-04-27T22:04:37.805449+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/1037298395 2024-04-27T22:07:37.836479+0000 2024-04-26T22:10:14.346 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6834/302589348 2024-04-27T22:05:52.818178+0000 2024-04-26T22:10:14.347 INFO:teuthology.orchestra.run.smithi135.stderr:listed 28 entries 2024-04-26T22:10:14.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f410806c920 msgr2=0x7f410806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f410806c920 0x7f410806ede0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4110034000 tx=0x7f411002e000 comp rx=0 tx=0).stop 2024-04-26T22:10:14.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 msgr2=0x7f412413b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 0x7f412413b120 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f411800b420 tx=0x7f411800b8e0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 shutdown_connections 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f410806c920 0x7f410806ede0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f412413ffb0 0x7f41241ed2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4124074c10 0x7f41241ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 --2- 172.21.15.135:0/1306004770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4124073700 0x7f412413b120 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 >> 172.21.15.135:0/1306004770 conn(0x7f41240ca2f0 msgr2=0x7f4124155bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 shutdown_connections 2024-04-26T22:10:14.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.345+0000 7f412b220640 1 -- 172.21.15.135:0/1306004770 wait complete. 2024-04-26T22:10:14.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T22:10:14.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:14.357 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T22:10:14.416 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:10:14.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:14.416 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T22:10:14.432 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T22:10:14.432 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:10:14.462 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T22:10:14.462 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:10:14.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:10:14.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 Processor -- start 2024-04-26T22:10:14.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 -- start start 2024-04-26T22:10:14.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee00ca5a0 0x7fdee0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdee015a1a0 0x7fdee00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00c8df0 con 0x7fdee0154160 2024-04-26T22:10:14.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdee00c9100 con 0x7fdee015a1a0 2024-04-26T22:10:14.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdee5647640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdee0154540 con 0x7fdee00ca5a0 2024-04-26T22:10:14.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdedf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdee015a1a0 0x7fdee00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdedf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdee015a1a0 0x7fdee00c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51392/0 (socket says 172.21.15.135:51392) 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.621+0000 7fdedf7fe640 1 -- 172.21.15.135:0/160795717 learned_addr learned my addr 172.21.15.135:0/160795717 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.622+0000 7fdede7fc640 1 --2- 172.21.15.135:0/160795717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee00ca5a0 0x7fdee0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.622+0000 7fdedf7fe640 1 -- 172.21.15.135:0/160795717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee00ca5a0 msgr2=0x7fdee0156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.622+0000 7fdedf7fe640 1 --2- 172.21.15.135:0/160795717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee00ca5a0 0x7fdee0156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.622+0000 7fdedf7fe640 1 -- 172.21.15.135:0/160795717 >> v1:172.21.15.139:6789/0 conn(0x7fdee0154160 legacy=0x7fdee00c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.622+0000 7fdedf7fe640 1 -- 172.21.15.135:0/160795717 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee015a910 con 0x7fdee015a1a0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.622+0000 7fdedf7fe640 1 --2- 172.21.15.135:0/160795717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee015a1a0 0x7fdee00c9590 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fded8009e50 tx=0x7fded802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795a8e54eb04684b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdeddffb640 1 -- 172.21.15.135:0/160795717 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8036070 con 0x7fdee015a1a0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdeddffb640 1 -- 172.21.15.135:0/160795717 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded8031600 con 0x7fdee015a1a0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdeddffb640 1 -- 172.21.15.135:0/160795717 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80319d0 con 0x7fdee015a1a0 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 -- 172.21.15.135:0/160795717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee015a1a0 msgr2=0x7fdee00c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 --2- 172.21.15.135:0/160795717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee015a1a0 0x7fdee00c9590 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fded8009e50 tx=0x7fded802a6c0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 -- 172.21.15.135:0/160795717 shutdown_connections 2024-04-26T22:10:14.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 --2- 172.21.15.135:0/160795717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee015a1a0 0x7fdee00c9590 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 --2- 172.21.15.135:0/160795717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee00ca5a0 0x7fdee0156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 -- 172.21.15.135:0/160795717 >> 172.21.15.135:0/160795717 conn(0x7fdee00ae010 msgr2=0x7fdee00ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:14.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 -- 172.21.15.135:0/160795717 shutdown_connections 2024-04-26T22:10:14.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.623+0000 7fdee5647640 1 -- 172.21.15.135:0/160795717 wait complete. 2024-04-26T22:10:14.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.624+0000 7fdee5647640 1 Processor -- start 2024-04-26T22:10:14.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.624+0000 7fdee5647640 1 -- start start 2024-04-26T22:10:14.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdee5647640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 0x7fdee00c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdee5647640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee0154160 0x7fdee00c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdee5647640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee015a1a0 0x7fdee0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdee5647640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdee015a630 con 0x7fdee00ca5a0 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdee5647640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdee015aed0 con 0x7fdee0154160 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdee5647640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00c7510 con 0x7fdee015a1a0 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdedeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 0x7fdee00c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdedeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 0x7fdee00c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51398/0 (socket says 172.21.15.135:51398) 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdedeffd640 1 -- 172.21.15.135:0/2614102967 learned_addr learned my addr 172.21.15.135:0/2614102967 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdede7fc640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee0154160 0x7fdee00c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdedf7fe640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee015a1a0 0x7fdee0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.625+0000 7fdedeffd640 1 -- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee015a1a0 msgr2=0x7fdee0155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdedeffd640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee015a1a0 0x7fdee0155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdedeffd640 1 -- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee0154160 msgr2=0x7fdee00c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdedeffd640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee0154160 0x7fdee00c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdedeffd640 1 -- 172.21.15.135:0/2614102967 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00c3bf0 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdedf7fe640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee015a1a0 0x7fdee0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdede7fc640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee0154160 0x7fdee00c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdedeffd640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 0x7fdee00c3550 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fded400c460 tx=0x7fded400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4016020 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4012870 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee00c2ed0 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4012c60 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.626+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee00cab70 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.627+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee0076650 con 0x7fdee00ca5a0 2024-04-26T22:10:14.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.627+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fded4019020 con 0x7fdee00ca5a0 2024-04-26T22:10:14.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.628+0000 7fdec77fe640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdecc06c8f0 0x7fdecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.628+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fded408be40 con 0x7fdee00ca5a0 2024-04-26T22:10:14.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.628+0000 7fdede7fc640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdecc06c8f0 0x7fdecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.628+0000 7fdede7fc640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdecc06c8f0 0x7fdecc06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdee014c3d0 tx=0x7fded0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.632+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fded4020070 con 0x7fdee00ca5a0 2024-04-26T22:10:14.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.809+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdee00507a0 con 0x7fdee00ca5a0 2024-04-26T22:10:14.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.811+0000 7fdec77fe640 1 -- 172.21.15.135:0/2614102967 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+22533 (secure 0 0 0) 0x7fded4059a00 con 0x7fdee00ca5a0 2024-04-26T22:10:14.811 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:14.811 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":40,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:10:00.929237+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6837/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6836/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6834/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6835/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.135:6837/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.139:6832/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6835/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6834/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6835/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.139:6833/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6836/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.139:6834/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6834/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.139:6835/302589348":"2024-04-27T22:05:52.818178+0000","172.21.15.139:6832/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.139:6835/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6833/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.135:6836/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6836/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.135:6837/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6835/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.135:6837/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.135:6835/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6834/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6835/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6834/302589348":"2024-04-27T22:05:52.818178+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:10:14.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdecc06c8f0 msgr2=0x7fdecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdecc06c8f0 0x7fdecc06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdee014c3d0 tx=0x7fded0008040 comp rx=0 tx=0).stop 2024-04-26T22:10:14.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 msgr2=0x7fdee00c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 0x7fdee00c3550 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fded400c460 tx=0x7fded400c920 comp rx=0 tx=0).stop 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 shutdown_connections 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdecc06c8f0 0x7fdecc06edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee015a1a0 0x7fdee0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee0154160 0x7fdee00c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 --2- 172.21.15.135:0/2614102967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdee00ca5a0 0x7fdee00c3550 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 >> 172.21.15.135:0/2614102967 conn(0x7fdee00ae010 msgr2=0x7fdee00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 shutdown_connections 2024-04-26T22:10:14.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.813+0000 7fdee5647640 1 -- 172.21.15.135:0/2614102967 wait complete. 2024-04-26T22:10:14.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:59:45.083457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-04-26T21:59:55.002813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-04-26T21:59:55.939338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-04-26T22:00:03.970577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-04-26T22:00:05.113722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:10:14.825 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:10:14.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.926+0000 7ffbfc596640 1 Processor -- start 2024-04-26T22:10:14.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.926+0000 7ffbfc596640 1 -- start start 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfc596640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf406d590 0x7ffbf4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfc596640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffbf4070120 0x7ffbf40aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfc596640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbf40aef70 0x7ffbf40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfc596640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ffbf40af400 con 0x7ffbf40aef70 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfc596640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffbf41559e0 con 0x7ffbf4070120 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfc596640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf4154e10 con 0x7ffbf406d590 2024-04-26T22:10:14.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbf40aef70 0x7ffbf40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbf40aef70 0x7ffbf40c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51402/0 (socket says 172.21.15.135:51402) 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 -- 172.21.15.135:0/3319376818 learned_addr learned my addr 172.21.15.135:0/3319376818 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbf9b31640 1 --2- 172.21.15.135:0/3319376818 >> v2:172.21.15.139:3300/0 conn(0x7ffbf4070120 0x7ffbf40aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfa332640 1 --2- 172.21.15.135:0/3319376818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf406d590 0x7ffbf4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 -- 172.21.15.135:0/3319376818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf406d590 msgr2=0x7ffbf4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 --2- 172.21.15.135:0/3319376818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf406d590 0x7ffbf4158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 -- 172.21.15.135:0/3319376818 >> v2:172.21.15.139:3300/0 conn(0x7ffbf4070120 msgr2=0x7ffbf40aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 --2- 172.21.15.135:0/3319376818 >> v2:172.21.15.139:3300/0 conn(0x7ffbf4070120 0x7ffbf40aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.927+0000 7ffbfab33640 1 -- 172.21.15.135:0/3319376818 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf4159240 con 0x7ffbf40aef70 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.928+0000 7ffbfab33640 1 --2- 172.21.15.135:0/3319376818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf40aef70 0x7ffbf40c3be0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ffbf406dd80 tx=0x7ffbec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf652b9a18917bad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.928+0000 7ffbf9330640 1 -- 172.21.15.135:0/3319376818 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec037070 con 0x7ffbf40aef70 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.928+0000 7ffbf9330640 1 -- 172.21.15.135:0/3319376818 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbec032440 con 0x7ffbf40aef70 2024-04-26T22:10:14.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.928+0000 7ffbf9330640 1 -- 172.21.15.135:0/3319376818 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec032810 con 0x7ffbf40aef70 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 -- 172.21.15.135:0/3319376818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf40aef70 msgr2=0x7ffbf40c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 --2- 172.21.15.135:0/3319376818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf40aef70 0x7ffbf40c3be0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ffbf406dd80 tx=0x7ffbec02a140 comp rx=0 tx=0).stop 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 -- 172.21.15.135:0/3319376818 shutdown_connections 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 --2- 172.21.15.135:0/3319376818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf40aef70 0x7ffbf40c3be0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 --2- 172.21.15.135:0/3319376818 >> v2:172.21.15.139:3300/0 conn(0x7ffbf4070120 0x7ffbf40aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 --2- 172.21.15.135:0/3319376818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf406d590 0x7ffbf4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 -- 172.21.15.135:0/3319376818 >> 172.21.15.135:0/3319376818 conn(0x7ffbf40c5ce0 msgr2=0x7ffbf40c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 -- 172.21.15.135:0/3319376818 shutdown_connections 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.929+0000 7ffbfc596640 1 -- 172.21.15.135:0/3319376818 wait complete. 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.930+0000 7ffbfc596640 1 Processor -- start 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.930+0000 7ffbfc596640 1 -- start start 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 0x7ffbf40c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4070120 0x7ffbf4148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf40aef70 0x7ffbf4149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4155670 con 0x7ffbf406d590 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4155390 con 0x7ffbf4070120 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf406d970 con 0x7ffbf40aef70 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 0x7ffbf40c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 0x7ffbf40c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51404/0 (socket says 172.21.15.135:51404) 2024-04-26T22:10:14.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 -- 172.21.15.135:0/2274729360 learned_addr learned my addr 172.21.15.135:0/2274729360 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbf9b31640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4070120 0x7ffbf4148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfab33640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf40aef70 0x7ffbf4149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 -- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf40aef70 msgr2=0x7ffbf4149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf40aef70 0x7ffbf4149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 -- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4070120 msgr2=0x7ffbf4148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4070120 0x7ffbf4148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 -- 172.21.15.135:0/2274729360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf414a200 con 0x7ffbf406d590 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbf9b31640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4070120 0x7ffbf4148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfab33640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf40aef70 0x7ffbf4149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfa332640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 0x7ffbf40c81c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ffbf000ef00 tx=0x7ffbf000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbf0018020 con 0x7ffbf406d590 2024-04-26T22:10:14.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbf000bd90 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbf0014620 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.931+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf414a4e0 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.932+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf414aaa0 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.932+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf414afc0 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.933+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbf0005d10 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.933+0000 7ffbeaffd640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbd806c920 0x7ffbd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.933+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7ffbf008cba0 con 0x7ffbf406d590 2024-04-26T22:10:14.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.933+0000 7ffbf9b31640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbd806c920 0x7ffbd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:14.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.934+0000 7ffbf9b31640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbd806c920 0x7ffbd806ede0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffbf40c8a20 tx=0x7ffbe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:14.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:14.938+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffbf0022070 con 0x7ffbf406d590 2024-04-26T22:10:15.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.114+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbf40c4740 con 0x7ffbf406d590 2024-04-26T22:10:15.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.115+0000 7ffbeaffd640 1 -- 172.21.15.135:0/2274729360 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7ffbf005a760 con 0x7ffbf406d590 2024-04-26T22:10:15.115 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbd806c920 msgr2=0x7ffbd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbd806c920 0x7ffbd806ede0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffbf40c8a20 tx=0x7ffbe4008040 comp rx=0 tx=0).stop 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 msgr2=0x7ffbf40c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 0x7ffbf40c81c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ffbf000ef00 tx=0x7ffbf000b480 comp rx=0 tx=0).stop 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 shutdown_connections 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffbd806c920 0x7ffbd806ede0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbf40aef70 0x7ffbf4149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbf4070120 0x7ffbf4148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 --2- 172.21.15.135:0/2274729360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf406d590 0x7ffbf40c81c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 >> 172.21.15.135:0/2274729360 conn(0x7ffbf40c5ce0 msgr2=0x7ffbf40c31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:15.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 shutdown_connections 2024-04-26T22:10:15.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.117+0000 7ffbfc596640 1 -- 172.21.15.135:0/2274729360 wait complete. 2024-04-26T22:10:15.128 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-04-26T22:10:15.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.228+0000 7f2a072da640 1 Processor -- start 2024-04-26T22:10:15.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.228+0000 7f2a072da640 1 -- start start 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a072da640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a000c4820 0x7f2a00073a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a072da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a00156f60 con 0x7f2a000ab630 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a072da640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a00157240 con 0x7f2a000aecf0 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a072da640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2a00157520 con 0x7f2a000c4820 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a05877640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f2a000ab630 0x7f2a00073fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47608/0 (socket says 172.21.15.135:47608) 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a05877640 1 -- 172.21.15.135:0/547984603 learned_addr learned my addr 172.21.15.135:0/547984603 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f2a04875640 1 --2- 172.21.15.135:0/547984603 >> v2:172.21.15.139:3300/0 conn(0x7f2a000c4820 0x7f2a00073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2715427453 0 0) 0x7f2a00156f60 con 0x7f2a000ab630 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29ec003520 con 0x7f2a000ab630 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2623088013 0 0) 0x7f29ec003520 con 0x7f2a000ab630 2024-04-26T22:10:15.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 >> v2:172.21.15.139:3300/0 conn(0x7f2a000c4820 msgr2=0x7f2a00073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 --2- 172.21.15.135:0/547984603 >> v2:172.21.15.139:3300/0 conn(0x7f2a000c4820 0x7f2a00073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 >> v1:172.21.15.139:6789/0 conn(0x7f2a000aecf0 legacy=0x7f2a000c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a00157800 con 0x7f2a000ab630 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f29fc001670 con 0x7f2a000ab630 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.229+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 >> v1:172.21.15.135:6789/0 conn(0x7f2a000ab630 legacy=0x7f2a00073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f29f7fff640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f29f7fff640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec004800 0x7f29ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f29f7fff640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0051c0 0x7f29ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00157520 con 0x7f29ec0039b0 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec00c160 con 0x7f29ec0051c0 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29ec00c360 con 0x7f29ec004800 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05076640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05877640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0051c0 0x7f29ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a04875640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec004800 0x7f29ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05076640 1 -- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec004800 msgr2=0x7f29ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05076640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec004800 0x7f29ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05076640 1 -- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0051c0 msgr2=0x7f29ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05076640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0051c0 0x7f29ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05877640 1 -- 172.21.15.135:0/547984603 reap_dead start 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.230+0000 7f2a05076640 1 -- 172.21.15.135:0/547984603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec003520 con 0x7f29ec0039b0 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a05076640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f29f0001750 tx=0x7f29ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460441566ae7b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0001370 con 0x7f29ec0039b0 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f00052d0 con 0x7f29ec0039b0 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f29f7fff640 1 -- 172.21.15.135:0/547984603 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f002abc0 con 0x7f29ec0039b0 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 -- 172.21.15.135:0/547984603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ec0039b0 msgr2=0x7f29ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f29f0001750 tx=0x7f29ec005830 comp rx=0 tx=0).stop 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 -- 172.21.15.135:0/547984603 shutdown_connections 2024-04-26T22:10:15.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 --2- 172.21.15.135:0/547984603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 -- 172.21.15.135:0/547984603 >> 172.21.15.135:0/547984603 conn(0x7f2a000c6ed0 msgr2=0x7f2a000c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:15.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 -- 172.21.15.135:0/547984603 shutdown_connections 2024-04-26T22:10:15.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.231+0000 7f2a072da640 1 -- 172.21.15.135:0/547984603 wait complete. 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.232+0000 7f2a072da640 1 Processor -- start 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.232+0000 7f2a072da640 1 -- start start 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a072da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0039b0 0x7f2a00148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a072da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0015ad40 0x7f2a001efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a072da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 0x7f2a000c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a072da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a000af0d0 con 0x7f2a00149540 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a072da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00156c10 con 0x7f29ec0039b0 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a072da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a000af250 con 0x7f2a0015ad40 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 0x7f2a000c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 0x7f2a000c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51420/0 (socket says 172.21.15.135:51420) 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 -- 172.21.15.135:0/1518881850 learned_addr learned my addr 172.21.15.135:0/1518881850 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:15.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05076640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0039b0 0x7f2a00148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a04875640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0015ad40 0x7f2a001efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 -- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0015ad40 msgr2=0x7f2a001efd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0015ad40 0x7f2a001efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 -- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0039b0 msgr2=0x7f2a00148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0039b0 0x7f2a00148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05877640 1 -- 172.21.15.135:0/1518881850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a001f02b0 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.233+0000 7f2a05076640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0039b0 0x7f2a00148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.234+0000 7f2a05877640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 0x7f2a000c80c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f29fc000c00 tx=0x7f29fc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.234+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00f6a0 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.234+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc018070 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.234+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a001492f0 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.234+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00fa60 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.234+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a00149010 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.235+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a000c4180 con 0x7f2a00149540 2024-04-26T22:10:15.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.235+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29fc00fc00 con 0x7f2a00149540 2024-04-26T22:10:15.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.236+0000 7f29f5ffb640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29e406c8f0 0x7f29e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.236+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f29fc08c030 con 0x7f2a00149540 2024-04-26T22:10:15.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.236+0000 7f2a05076640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29e406c8f0 0x7f29e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.236+0000 7f2a05076640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29e406c8f0 0x7f29e406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f29f0032000 tx=0x7f29f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.240+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29fc059c80 con 0x7f2a00149540 2024-04-26T22:10:15.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.417+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) -- 0x7f2a000507a0 con 0x7f2a00149540 2024-04-26T22:10:15.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.417+0000 7f29f5ffb640 1 -- 172.21.15.135:0/1518881850 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v40) ==== 99+0+11 (secure 0 0 0) 0x7f29fc059580 con 0x7f2a00149540 2024-04-26T22:10:15.417 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:10:15.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29e406c8f0 msgr2=0x7f29e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29e406c8f0 0x7f29e406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f29f0032000 tx=0x7f29f002e000 comp rx=0 tx=0).stop 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 msgr2=0x7f2a000c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 0x7f2a000c80c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f29fc000c00 tx=0x7f29fc007430 comp rx=0 tx=0).stop 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 shutdown_connections 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f29e406c8f0 0x7f29e406edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a00149540 0x7f2a000c80c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0015ad40 0x7f2a001efd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 --2- 172.21.15.135:0/1518881850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0039b0 0x7f2a00148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 >> 172.21.15.135:0/1518881850 conn(0x7f2a000c6ed0 msgr2=0x7f2a000ad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 shutdown_connections 2024-04-26T22:10:15.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.419+0000 7f2a072da640 1 -- 172.21.15.135:0/1518881850 wait complete. 2024-04-26T22:10:15.430 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-04-26T22:10:15.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.530+0000 7f49615e4640 1 Processor -- start 2024-04-26T22:10:15.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f49615e4640 1 -- start start 2024-04-26T22:10:15.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f49615e4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f495c154160 0x7f495c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f49615e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f495c15a1a0 0x7f495c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f49615e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c0c8df0 con 0x7f495c0ca5a0 2024-04-26T22:10:15.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f49615e4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f495c0c9100 con 0x7f495c154160 2024-04-26T22:10:15.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f49615e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f495c154540 con 0x7f495c15a1a0 2024-04-26T22:10:15.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f495affd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f495c154160 0x7f495c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f495affd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f495c154160 0x7f495c156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51436/0 (socket says 172.21.15.135:51436) 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.531+0000 7f495affd640 1 -- 172.21.15.135:0/1344481831 learned_addr learned my addr 172.21.15.135:0/1344481831 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495b7fe640 1 --2- 172.21.15.135:0/1344481831 >> v2:172.21.15.139:3300/0 conn(0x7f495c15a1a0 0x7f495c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495affd640 1 -- 172.21.15.135:0/1344481831 >> v2:172.21.15.139:3300/0 conn(0x7f495c15a1a0 msgr2=0x7f495c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495affd640 1 --2- 172.21.15.135:0/1344481831 >> v2:172.21.15.139:3300/0 conn(0x7f495c15a1a0 0x7f495c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495affd640 1 -- 172.21.15.135:0/1344481831 >> v1:172.21.15.139:6789/0 conn(0x7f495c0ca5a0 legacy=0x7f495c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495affd640 1 -- 172.21.15.135:0/1344481831 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c15a910 con 0x7f495c154160 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495b7fe640 1 --2- 172.21.15.135:0/1344481831 >> v2:172.21.15.139:3300/0 conn(0x7f495c15a1a0 0x7f495c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f495affd640 1 --2- 172.21.15.135:0/1344481831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c154160 0x7f495c156bb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f495c0c9280 tx=0x7f494c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3051e41de1b653 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f4959ffb640 1 -- 172.21.15.135:0/1344481831 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c036070 con 0x7f495c154160 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.532+0000 7f4959ffb640 1 -- 172.21.15.135:0/1344481831 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c031600 con 0x7f495c154160 2024-04-26T22:10:15.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f4959ffb640 1 -- 172.21.15.135:0/1344481831 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0319d0 con 0x7f495c154160 2024-04-26T22:10:15.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 -- 172.21.15.135:0/1344481831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c154160 msgr2=0x7f495c156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 --2- 172.21.15.135:0/1344481831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c154160 0x7f495c156bb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f495c0c9280 tx=0x7f494c02a730 comp rx=0 tx=0).stop 2024-04-26T22:10:15.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 -- 172.21.15.135:0/1344481831 shutdown_connections 2024-04-26T22:10:15.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 --2- 172.21.15.135:0/1344481831 >> v2:172.21.15.139:3300/0 conn(0x7f495c15a1a0 0x7f495c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 --2- 172.21.15.135:0/1344481831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c154160 0x7f495c156bb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 -- 172.21.15.135:0/1344481831 >> 172.21.15.135:0/1344481831 conn(0x7f495c0ae010 msgr2=0x7f495c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 -- 172.21.15.135:0/1344481831 shutdown_connections 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.533+0000 7f49615e4640 1 -- 172.21.15.135:0/1344481831 wait complete. 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.534+0000 7f49615e4640 1 Processor -- start 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.534+0000 7f49615e4640 1 -- start start 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f49615e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0ca5a0 0x7f495c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f49615e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c154160 0x7f495c14a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f49615e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 0x7f495c14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f49615e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f495c15a5b0 con 0x7f495c15a1a0 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f49615e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f495c15aed0 con 0x7f495c154160 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f49615e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f495c0c7510 con 0x7f495c0ca5a0 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 0x7f495c14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 0x7f495c14a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51452/0 (socket says 172.21.15.135:51452) 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 -- 172.21.15.135:0/3996931821 learned_addr learned my addr 172.21.15.135:0/3996931821 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495affd640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0ca5a0 0x7f495c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495a7fc640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c154160 0x7f495c14a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 -- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0ca5a0 msgr2=0x7f495c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0ca5a0 0x7f495c156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 -- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c154160 msgr2=0x7f495c14a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c154160 0x7f495c14a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.535+0000 7f495b7fe640 1 -- 172.21.15.135:0/3996931821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c1efbc0 con 0x7f495c15a1a0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.536+0000 7f495b7fe640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 0x7f495c14a760 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f495400d5f0 tx=0x7f495400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.536+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954004030 con 0x7f495c15a1a0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.536+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49540041d0 con 0x7f495c15a1a0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.536+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c1efea0 con 0x7f495c15a1a0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.536+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954003610 con 0x7f495c15a1a0 2024-04-26T22:10:15.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.536+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c1f0460 con 0x7f495c15a1a0 2024-04-26T22:10:15.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.537+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c14f870 con 0x7f495c15a1a0 2024-04-26T22:10:15.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.537+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49540037b0 con 0x7f495c15a1a0 2024-04-26T22:10:15.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.537+0000 7f49437fe640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f494806c8f0 0x7f494806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.538+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f495408c420 con 0x7f495c15a1a0 2024-04-26T22:10:15.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.538+0000 7f495affd640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f494806c8f0 0x7f494806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.538+0000 7f495affd640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f494806c8f0 0x7f494806edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f494c033000 tx=0x7f494c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.542+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f495c14f870 con 0x7f495c15a1a0 2024-04-26T22:10:15.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.719+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) -- 0x7f495c076650 con 0x7f495c15a1a0 2024-04-26T22:10:15.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.719+0000 7f49437fe640 1 -- 172.21.15.135:0/3996931821 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v40) ==== 95+0+11 (secure 0 0 0) 0x7f4954059a20 con 0x7f495c15a1a0 2024-04-26T22:10:15.720 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:10:15.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f494806c8f0 msgr2=0x7f494806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f494806c8f0 0x7f494806edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f494c033000 tx=0x7f494c009000 comp rx=0 tx=0).stop 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 msgr2=0x7f495c14a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 0x7f495c14a760 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f495400d5f0 tx=0x7f495400dab0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 shutdown_connections 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f494806c8f0 0x7f494806edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f495c15a1a0 0x7f495c14a760 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c154160 0x7f495c14a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 --2- 172.21.15.135:0/3996931821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0ca5a0 0x7f495c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 >> 172.21.15.135:0/3996931821 conn(0x7f495c0ae010 msgr2=0x7f495c0c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 shutdown_connections 2024-04-26T22:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.721+0000 7f49615e4640 1 -- 172.21.15.135:0/3996931821 wait complete. 2024-04-26T22:10:15.732 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-04-26T22:10:15.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.832+0000 7f98c13ac640 1 Processor -- start 2024-04-26T22:10:15.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.832+0000 7f98c13ac640 1 -- start start 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.832+0000 7f98c13ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f98bc0ca5a0 0x7f98bc156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.832+0000 7f98c13ac640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98bc15a1a0 0x7f98bc0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98c13ac640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc0c8df0 con 0x7f98bc154160 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98c13ac640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f98bc0c9100 con 0x7f98bc15a1a0 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98c13ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f98bc154540 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98bc15a1a0 0x7f98bc0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98bc15a1a0 0x7f98bc0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51456/0 (socket says 172.21.15.135:51456) 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 -- 172.21.15.135:0/3073612563 learned_addr learned my addr 172.21.15.135:0/3073612563 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98baffd640 1 --1- 172.21.15.135:0/3073612563 >> v1:172.21.15.135:6789/0 conn(0x7f98bc154160 0x7f98bc0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47624/0 (socket says 172.21.15.135:47624) 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98b3fff640 1 --2- 172.21.15.135:0/3073612563 >> v2:172.21.15.139:3300/0 conn(0x7f98bc0ca5a0 0x7f98bc156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 -- 172.21.15.135:0/3073612563 >> v2:172.21.15.139:3300/0 conn(0x7f98bc0ca5a0 msgr2=0x7f98bc156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 --2- 172.21.15.135:0/3073612563 >> v2:172.21.15.139:3300/0 conn(0x7f98bc0ca5a0 0x7f98bc156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 -- 172.21.15.135:0/3073612563 >> v1:172.21.15.135:6789/0 conn(0x7f98bc154160 legacy=0x7f98bc0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98bb7fe640 1 -- 172.21.15.135:0/3073612563 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc15a910 con 0x7f98bc15a1a0 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.833+0000 7f98ba7fc640 1 -- 172.21.15.135:0/3073612563 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3956000758 0 0) 0x7f98ac001110 con 0x7f98bc154160 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98bb7fe640 1 --2- 172.21.15.135:0/3073612563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc15a1a0 0x7f98bc0c9590 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0c9280 tx=0x7f98b402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d52a6824523494 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98ba7fc640 1 -- 172.21.15.135:0/3073612563 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4036070 con 0x7f98bc15a1a0 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98ba7fc640 1 -- 172.21.15.135:0/3073612563 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b4031600 con 0x7f98bc15a1a0 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98ba7fc640 1 -- 172.21.15.135:0/3073612563 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40319d0 con 0x7f98bc15a1a0 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 -- 172.21.15.135:0/3073612563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc15a1a0 msgr2=0x7f98bc0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 --2- 172.21.15.135:0/3073612563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc15a1a0 0x7f98bc0c9590 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0c9280 tx=0x7f98b402a730 comp rx=0 tx=0).stop 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 -- 172.21.15.135:0/3073612563 shutdown_connections 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 --2- 172.21.15.135:0/3073612563 >> v2:172.21.15.139:3300/0 conn(0x7f98bc0ca5a0 0x7f98bc156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 --2- 172.21.15.135:0/3073612563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc15a1a0 0x7f98bc0c9590 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 -- 172.21.15.135:0/3073612563 >> 172.21.15.135:0/3073612563 conn(0x7f98bc0ae010 msgr2=0x7f98bc0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:15.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.834+0000 7f98c13ac640 1 -- 172.21.15.135:0/3073612563 shutdown_connections 2024-04-26T22:10:15.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.835+0000 7f98c13ac640 1 -- 172.21.15.135:0/3073612563 wait complete. 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 Processor -- start 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 -- start start 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 0x7f98bc1f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98bc154160 0x7f98bc155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98bc15a1a0 0x7f98bc0bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc15a5b0 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc15aed0 con 0x7f98bc154160 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98c13ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc0c7510 con 0x7f98bc15a1a0 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98baffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 0x7f98bc1f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98baffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 0x7f98bc1f09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51472/0 (socket says 172.21.15.135:51472) 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98baffd640 1 -- 172.21.15.135:0/1530487593 learned_addr learned my addr 172.21.15.135:0/1530487593 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.836+0000 7f98b3fff640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98bc154160 0x7f98bc155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98bb7fe640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98bc15a1a0 0x7f98bc0bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98baffd640 1 -- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98bc15a1a0 msgr2=0x7f98bc0bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98baffd640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98bc15a1a0 0x7f98bc0bffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98baffd640 1 -- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98bc154160 msgr2=0x7f98bc155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98baffd640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98bc154160 0x7f98bc155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98baffd640 1 -- 172.21.15.135:0/1530487593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc0c19a0 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98bb7fe640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98bc15a1a0 0x7f98bc0bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98baffd640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 0x7f98bc1f09a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c780 tx=0x7f98ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac018070 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac0072d0 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1efec0 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0047e0 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.837+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1f0480 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.838+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc1f0680 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.838+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98ac01b070 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.839+0000 7f98b37fe640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f989c06c8f0 0x7f989c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.839+0000 7f98b3fff640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f989c06c8f0 0x7f989c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.839+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f98ac08b680 con 0x7f98bc0ca5a0 2024-04-26T22:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.839+0000 7f98b3fff640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f989c06c8f0 0x7f989c06edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f98bc153b70 tx=0x7f98a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:15.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:15.843+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98bc1f0680 con 0x7f98bc0ca5a0 2024-04-26T22:10:16.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.021+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) -- 0x7f98bc076650 con 0x7f98bc0ca5a0 2024-04-26T22:10:16.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.022+0000 7f98b37fe640 1 -- 172.21.15.135:0/1530487593 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v40) ==== 99+0+11 (secure 0 0 0) 0x7f98bc0c8df0 con 0x7f98bc0ca5a0 2024-04-26T22:10:16.023 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-26T22:10:16.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f989c06c8f0 msgr2=0x7f989c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f989c06c8f0 0x7f989c06edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f98bc153b70 tx=0x7f98a4008040 comp rx=0 tx=0).stop 2024-04-26T22:10:16.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 msgr2=0x7f98bc1f09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 0x7f98bc1f09a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c780 tx=0x7f98ac00cc40 comp rx=0 tx=0).stop 2024-04-26T22:10:16.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 shutdown_connections 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f989c06c8f0 0x7f989c06edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98bc15a1a0 0x7f98bc0bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98bc154160 0x7f98bc155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 --2- 172.21.15.135:0/1530487593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98bc0ca5a0 0x7f98bc1f09a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 >> 172.21.15.135:0/1530487593 conn(0x7f98bc0ae010 msgr2=0x7f98bc14f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 shutdown_connections 2024-04-26T22:10:16.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.024+0000 7f98c13ac640 1 -- 172.21.15.135:0/1530487593 wait complete. 2024-04-26T22:10:16.035 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-04-26T22:10:16.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.135+0000 7f3b88e3e640 1 Processor -- start 2024-04-26T22:10:16.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.135+0000 7f3b88e3e640 1 -- start start 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b88e3e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 0x7f3b84156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b88e3e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3b8415a1a0 0x7f3b840c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b88e3e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b840c8df0 con 0x7f3b84154160 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b88e3e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3b840c9100 con 0x7f3b8415a1a0 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b88e3e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b84154540 con 0x7f3b840ca5a0 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b8259c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3b84154160 0x7f3b840c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47636/0 (socket says 172.21.15.135:47636) 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.136+0000 7f3b8259c640 1 -- 172.21.15.135:0/104443538 learned_addr learned my addr 172.21.15.135:0/104443538 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b82d9d640 1 --2- 172.21.15.135:0/104443538 >> v2:172.21.15.139:3300/0 conn(0x7f3b8415a1a0 0x7f3b840c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922557523 0 0) 0x7f3b78001110 con 0x7f3b84154160 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 0x7f3b84156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b6c003520 con 0x7f3b84154160 2024-04-26T22:10:16.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 245485227 0 0) 0x7f3b6c003520 con 0x7f3b84154160 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 msgr2=0x7f3b84156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 0x7f3b84156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 >> v2:172.21.15.139:3300/0 conn(0x7f3b8415a1a0 msgr2=0x7f3b840c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 --2- 172.21.15.135:0/104443538 >> v2:172.21.15.139:3300/0 conn(0x7f3b8415a1a0 0x7f3b840c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8415a910 con 0x7f3b84154160 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f3b78001840 con 0x7f3b84154160 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 >> v1:172.21.15.135:6789/0 conn(0x7f3b84154160 legacy=0x7f3b840c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6c0039b0 0x7f3b6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b6c004800 0x7f3b6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 0x7f3b6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b840c9100 con 0x7f3b6c004800 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84154540 con 0x7f3b6c0039b0 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c00b9c0 con 0x7f3b840ca5a0 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b6c004800 0x7f3b6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b8259c640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6c0039b0 0x7f3b6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b82d9d640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 0x7f3b6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 -- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 msgr2=0x7f3b6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b840ca5a0 0x7f3b6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 -- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6c0039b0 msgr2=0x7f3b6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6c0039b0 0x7f3b6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.137+0000 7f3b81d9b640 1 -- 172.21.15.135:0/104443538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c003520 con 0x7f3b6c004800 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.138+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b6c004800 0x7f3b6c004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009a90 tx=0x7f3b6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cc5337e9a70faa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.138+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74036070 con 0x7f3b6c004800 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.138+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b74031400 con 0x7f3b6c004800 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.138+0000 7f3b8159a640 1 -- 172.21.15.135:0/104443538 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74009690 con 0x7f3b6c004800 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b88e3e640 1 -- 172.21.15.135:0/104443538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b6c004800 msgr2=0x7f3b6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/104443538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b6c004800 0x7f3b6c004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009a90 tx=0x7f3b6c005830 comp rx=0 tx=0).stop 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b82d9d640 1 -- 172.21.15.135:0/104443538 reap_dead start 2024-04-26T22:10:16.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b88e3e640 1 -- 172.21.15.135:0/104443538 shutdown_connections 2024-04-26T22:10:16.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b88e3e640 1 -- 172.21.15.135:0/104443538 >> 172.21.15.135:0/104443538 conn(0x7f3b840ae010 msgr2=0x7f3b840ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:16.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b88e3e640 1 -- 172.21.15.135:0/104443538 shutdown_connections 2024-04-26T22:10:16.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.139+0000 7f3b88e3e640 1 -- 172.21.15.135:0/104443538 wait complete. 2024-04-26T22:10:16.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.140+0000 7f3b88e3e640 1 Processor -- start 2024-04-26T22:10:16.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.140+0000 7f3b88e3e640 1 -- start start 2024-04-26T22:10:16.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6c004800 0x7f3b84156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.144 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 0x7f3b841eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b840ca9b0 0x7f3b841ed230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8415a630 con 0x7f3b8414c2e0 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8415aed0 con 0x7f3b840ca9b0 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b840c7510 con 0x7f3b6c004800 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 0x7f3b841eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 0x7f3b841eccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51498/0 (socket says 172.21.15.135:51498) 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 -- 172.21.15.135:0/3442855847 learned_addr learned my addr 172.21.15.135:0/3442855847 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b8259c640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6c004800 0x7f3b84156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b82d9d640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b840ca9b0 0x7f3b841ed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 -- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6c004800 msgr2=0x7f3b84156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6c004800 0x7f3b84156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 -- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b840ca9b0 msgr2=0x7f3b841ed230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b840ca9b0 0x7f3b841ed230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 -- 172.21.15.135:0/3442855847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b840cbc90 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b81d9b640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 0x7f3b841eccf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009a90 tx=0x7f3b74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74040020 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b74003bc0 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.141+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b840cbf70 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.142+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74036040 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.142+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b841ec7d0 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.143+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b84076650 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.143+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b74043020 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.143+0000 7f3b72ffd640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b6406c8f0 0x7f3b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.143+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f3b740ac260 con 0x7f3b8414c2e0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.143+0000 7f3b8259c640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b6406c8f0 0x7f3b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.144+0000 7f3b8259c640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b6406c8f0 0x7f3b6406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3b780012f0 tx=0x7f3b78007b00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:16.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.148+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b7404a070 con 0x7f3b8414c2e0 2024-04-26T22:10:16.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.325+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) -- 0x7f3b840507a0 con 0x7f3b8414c2e0 2024-04-26T22:10:16.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.325+0000 7f3b72ffd640 1 -- 172.21.15.135:0/3442855847 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v40) ==== 95+0+11 (secure 0 0 0) 0x7f3b740044e0 con 0x7f3b8414c2e0 2024-04-26T22:10:16.326 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.327+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b6406c8f0 msgr2=0x7f3b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.327+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b6406c8f0 0x7f3b6406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3b780012f0 tx=0x7f3b78007b00 comp rx=0 tx=0).stop 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.327+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 msgr2=0x7f3b841eccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.327+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 0x7f3b841eccf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009a90 tx=0x7f3b74004730 comp rx=0 tx=0).stop 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.327+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 shutdown_connections 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.327+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f3b6406c8f0 0x7f3b6406edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.328+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b840ca9b0 0x7f3b841ed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.328+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8414c2e0 0x7f3b841eccf0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.328+0000 7f3b88e3e640 1 --2- 172.21.15.135:0/3442855847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b6c004800 0x7f3b84156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.328+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 >> 172.21.15.135:0/3442855847 conn(0x7f3b840ae010 msgr2=0x7f3b840c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:16.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.328+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 shutdown_connections 2024-04-26T22:10:16.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.328+0000 7f3b88e3e640 1 -- 172.21.15.135:0/3442855847 wait complete. 2024-04-26T22:10:16.339 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:10:16.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.437+0000 7f7409f46640 1 Processor -- start 2024-04-26T22:10:16.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.437+0000 7f7409f46640 1 -- start start 2024-04-26T22:10:16.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7409f46640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74041541d0 0x7f74040c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7409f46640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f740415a3c0 0x7f740415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7409f46640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74040c3fc0 con 0x7f7404070490 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7409f46640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f74040c3500 con 0x7f740415a3c0 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7409f46640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f74040c37e0 con 0x7f74041541d0 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f740415a3c0 0x7f740415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f740415a3c0 0x7f740415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51500/0 (socket says 172.21.15.135:51500) 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 -- 172.21.15.135:0/3670102631 learned_addr learned my addr 172.21.15.135:0/3670102631 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f74037fe640 1 --2- 172.21.15.135:0/3670102631 >> v2:172.21.15.139:3300/0 conn(0x7f74041541d0 0x7f74040c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 -- 172.21.15.135:0/3670102631 >> v2:172.21.15.139:3300/0 conn(0x7f74041541d0 msgr2=0x7f74040c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 --2- 172.21.15.135:0/3670102631 >> v2:172.21.15.139:3300/0 conn(0x7f74041541d0 0x7f74040c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 -- 172.21.15.135:0/3670102631 >> v1:172.21.15.139:6789/0 conn(0x7f7404070490 legacy=0x7f74040c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:16.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.438+0000 7f7403fff640 1 -- 172.21.15.135:0/3670102631 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74040c3ac0 con 0x7f740415a3c0 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7403fff640 1 --2- 172.21.15.135:0/3670102631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f740415a3c0 0x7f740415a7e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f74040c2f80 tx=0x7f73fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6cf6882fa88156 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f74027fc640 1 -- 172.21.15.135:0/3670102631 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc036070 con 0x7f740415a3c0 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f74027fc640 1 -- 172.21.15.135:0/3670102631 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc031600 con 0x7f740415a3c0 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f74027fc640 1 -- 172.21.15.135:0/3670102631 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0319d0 con 0x7f740415a3c0 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 -- 172.21.15.135:0/3670102631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f740415a3c0 msgr2=0x7f740415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 --2- 172.21.15.135:0/3670102631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f740415a3c0 0x7f740415a7e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f74040c2f80 tx=0x7f73fc02a660 comp rx=0 tx=0).stop 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 -- 172.21.15.135:0/3670102631 shutdown_connections 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 --2- 172.21.15.135:0/3670102631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f740415a3c0 0x7f740415a7e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 --2- 172.21.15.135:0/3670102631 >> v2:172.21.15.139:3300/0 conn(0x7f74041541d0 0x7f74040c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 -- 172.21.15.135:0/3670102631 >> 172.21.15.135:0/3670102631 conn(0x7f74040c8c40 msgr2=0x7f74040c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 -- 172.21.15.135:0/3670102631 shutdown_connections 2024-04-26T22:10:16.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.439+0000 7f7409f46640 1 -- 172.21.15.135:0/3670102631 wait complete. 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.440+0000 7f7409f46640 1 Processor -- start 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 -- start start 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 0x7f74040a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74041541d0 0x7f74041596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f740415a3c0 0x7f74041f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f74040c7cb0 con 0x7f7404070490 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f74040c7b30 con 0x7f74041541d0 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f7409f46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f74040c42a0 con 0x7f740415a3c0 2024-04-26T22:10:16.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f74037fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 0x7f74040a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f74037fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 0x7f74040a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51508/0 (socket says 172.21.15.135:51508) 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.441+0000 7f74037fe640 1 -- 172.21.15.135:0/3021766404 learned_addr learned my addr 172.21.15.135:0/3021766404 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f7403fff640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f740415a3c0 0x7f74041f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f7402ffd640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74041541d0 0x7f74041596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f74037fe640 1 -- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f740415a3c0 msgr2=0x7f74041f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f74037fe640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f740415a3c0 0x7f74041f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f74037fe640 1 -- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74041541d0 msgr2=0x7f74041596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f74037fe640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74041541d0 0x7f74041596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f74037fe640 1 -- 172.21.15.135:0/3021766404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74040733a0 con 0x7f7404070490 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f74037fe640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 0x7f74040a8b10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f73f800d830 tx=0x7f73f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800b660 con 0x7f7404070490 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f8019070 con 0x7f7404070490 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7404073680 con 0x7f7404070490 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800ba20 con 0x7f7404070490 2024-04-26T22:10:16.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.442+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74040a9050 con 0x7f7404070490 2024-04-26T22:10:16.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.443+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74040689f0 con 0x7f7404070490 2024-04-26T22:10:16.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.444+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73f801e460 con 0x7f7404070490 2024-04-26T22:10:16.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.444+0000 7f73ebfff640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73f006c8f0 0x7f73f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:16.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.444+0000 7f7402ffd640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73f006c8f0 0x7f73f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:16.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.444+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f73f808ca20 con 0x7f7404070490 2024-04-26T22:10:16.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.444+0000 7f7402ffd640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73f006c8f0 0x7f73f006edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7404153a00 tx=0x7f73f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:16.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.448+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73f805a5e0 con 0x7f7404070490 2024-04-26T22:10:16.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.634+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f74040507a0 con 0x7f7404070490 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.635+0000 7f73ebfff640 1 -- 172.21.15.135:0/3021766404 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 28 entries v40) ==== 81+0+1734 (secure 0 0 0) 0x7f73f8059ee0 con 0x7f7404070490 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/3969385858 2024-04-27T22:09:52.864568+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/3969385858 2024-04-27T22:09:52.864568+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/1037298395 2024-04-27T22:07:37.836479+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6835/3733666635 2024-04-27T22:07:32.835007+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/2165063360 2024-04-27T22:07:07.830780+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6832/3664037452 2024-04-27T22:08:57.853641+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/3214490783 2024-04-27T22:06:12.821933+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/3214490783 2024-04-27T22:06:12.821933+0000 2024-04-26T22:10:16.635 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/1410523687 2024-04-27T22:01:27.768292+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6833/3664037452 2024-04-27T22:08:57.853641+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/3981978490 2024-04-27T22:01:37.770149+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/3981978490 2024-04-27T22:01:37.770149+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/3800113814 2024-04-27T22:10:00.929175+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6834/2608264407 2024-04-27T22:03:07.784944+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6834/3733666635 2024-04-27T22:07:32.835007+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6835/302589348 2024-04-27T22:05:52.818178+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6832/3183513686 2024-04-27T22:03:27.791314+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6835/2608264407 2024-04-27T22:03:07.784944+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6833/3183513686 2024-04-27T22:03:27.791314+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/3652200818 2024-04-27T22:04:12.801037+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6836/2165063360 2024-04-27T22:07:07.830780+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/3652200818 2024-04-27T22:04:12.801037+0000 2024-04-26T22:10:16.636 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/3800113814 2024-04-27T22:10:00.929175+0000 2024-04-26T22:10:16.637 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6837/1410523687 2024-04-27T22:01:27.768292+0000 2024-04-26T22:10:16.637 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/2910412510 2024-04-27T22:04:37.805449+0000 2024-04-26T22:10:16.637 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6834/2910412510 2024-04-27T22:04:37.805449+0000 2024-04-26T22:10:16.637 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135:6835/1037298395 2024-04-27T22:07:37.836479+0000 2024-04-26T22:10:16.637 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.139:6834/302589348 2024-04-27T22:05:52.818178+0000 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:listed 28 entries 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73f006c8f0 msgr2=0x7f73f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73f006c8f0 0x7f73f006edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7404153a00 tx=0x7f73f4009040 comp rx=0 tx=0).stop 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 msgr2=0x7f74040a8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 0x7f74040a8b10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f73f800d830 tx=0x7f73f800dcf0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 shutdown_connections 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f73f006c8f0 0x7f73f006edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f740415a3c0 0x7f74041f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74041541d0 0x7f74041596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 --2- 172.21.15.135:0/3021766404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7404070490 0x7f74040a8b10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:16.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 >> 172.21.15.135:0/3021766404 conn(0x7f74040c8c40 msgr2=0x7f740415a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:16.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 shutdown_connections 2024-04-26T22:10:16.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:16.637+0000 7f7409f46640 1 -- 172.21.15.135:0/3021766404 wait complete. 2024-04-26T22:10:16.648 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T22:10:16.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T22:10:16.688 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:10:16.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.688 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T22:10:16.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.746 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-26T22:10:16.761 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T22:10:16.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.762 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:10:16.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.826 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-04-26T22:10:16.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.923 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-26T22:10:16.947 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T22:10:16.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:16.948 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T22:10:17.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:10:17.015 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-04-26T22:10:17.113 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T22:10:17.122 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T22:10:17.122 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:10:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.222+0000 7fddcea50640 1 Processor -- start 2024-04-26T22:10:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.222+0000 7fddcea50640 1 -- start start 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddcea50640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 0x7fddc81576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddcea50640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc8156c10 con 0x7fddc813feb0 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddcea50640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc80c5570 con 0x7fddc80c5050 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddcea50640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fddc815a8b0 con 0x7fddc80c2b80 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc7fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fddc813feb0 0x7fddc815a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47646/0 (socket says 172.21.15.135:47646) 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc7fff640 1 -- 172.21.15.135:0/3967787473 learned_addr learned my addr 172.21.15.135:0/3967787473 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddccfed640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 0x7fddc81576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745693386 0 0) 0x7fddc8156c10 con 0x7fddc813feb0 2024-04-26T22:10:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddb0003520 con 0x7fddc813feb0 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531041415 0 0) 0x7fddc80c5570 con 0x7fddc80c5050 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc8156c10 con 0x7fddc80c5050 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3113844644 0 0) 0x7fddb0003520 con 0x7fddc813feb0 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 msgr2=0x7fddc81576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 0x7fddc81576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 >> v1:172.21.15.139:6789/0 conn(0x7fddc80c5050 legacy=0x7fddc806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc80c5850 con 0x7fddc813feb0 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7fddb8001670 con 0x7fddc813feb0 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 >> v1:172.21.15.135:6789/0 conn(0x7fddc813feb0 legacy=0x7fddc815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb0003b70 0x7fddb0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0004800 0x7fddb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 0x7fddb00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddc815a8b0 con 0x7fddb0003b70 2024-04-26T22:10:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddb000bad0 con 0x7fddb0004800 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fddb000bcd0 con 0x7fddc80c2b80 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb0003b70 0x7fddb0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddc77fe640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0004800 0x7fddb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.223+0000 7fddccfed640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 0x7fddb00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc7fff640 1 -- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 msgr2=0x7fddb00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80c2b80 0x7fddb00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc7fff640 1 -- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0004800 msgr2=0x7fddb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0004800 0x7fddb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc7fff640 1 -- 172.21.15.135:0/3967787473 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0003520 con 0x7fddb0003b70 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb0003b70 0x7fddb0003f70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c160 tx=0x7fddb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1558d876a2738d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007ca0 con 0x7fddb0003b70 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8005af0 con 0x7fddb0003b70 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.224+0000 7fddc6ffd640 1 -- 172.21.15.135:0/3967787473 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007070 con 0x7fddb0003b70 2024-04-26T22:10:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddcea50640 1 -- 172.21.15.135:0/3967787473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb0003b70 msgr2=0x7fddb0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddcea50640 1 --2- 172.21.15.135:0/3967787473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb0003b70 0x7fddb0003f70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c160 tx=0x7fddb0005830 comp rx=0 tx=0).stop 2024-04-26T22:10:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddccfed640 1 -- 172.21.15.135:0/3967787473 reap_dead start 2024-04-26T22:10:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddcea50640 1 -- 172.21.15.135:0/3967787473 shutdown_connections 2024-04-26T22:10:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddcea50640 1 -- 172.21.15.135:0/3967787473 >> 172.21.15.135:0/3967787473 conn(0x7fddc80c5ce0 msgr2=0x7fddc80c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddcea50640 1 -- 172.21.15.135:0/3967787473 shutdown_connections 2024-04-26T22:10:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.225+0000 7fddcea50640 1 -- 172.21.15.135:0/3967787473 wait complete. 2024-04-26T22:10:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.226+0000 7fddcea50640 1 Processor -- start 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.226+0000 7fddcea50640 1 -- start start 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddcea50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0003b70 0x7fddc80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddcea50640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80ac390 0x7fddc806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddcea50640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 0x7fddc8137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddcea50640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddc806da30 con 0x7fddc8154c90 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddcea50640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8157240 con 0x7fddb0003b70 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddcea50640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fddc806dbb0 con 0x7fddc80ac390 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 0x7fddc8137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 0x7fddc8137e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51526/0 (socket says 172.21.15.135:51526) 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 -- 172.21.15.135:0/3539792753 learned_addr learned my addr 172.21.15.135:0/3539792753 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0003b70 0x7fddc80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddc77fe640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80ac390 0x7fddc806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 -- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80ac390 msgr2=0x7fddc806cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80ac390 0x7fddc806cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 -- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0003b70 msgr2=0x7fddc80adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0003b70 0x7fddc80adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 -- 172.21.15.135:0/3539792753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc81f0420 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.227+0000 7fddccfed640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 0x7fddc8137e00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fddc0011c80 tx=0x7fddc000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.228+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0011e90 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.228+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc0015430 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.228+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc81f06c0 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.228+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0015820 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.228+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc806d280 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.229+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc80ac770 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.229+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddc00159c0 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.229+0000 7fddc4ff9640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fddb406ca90 0x7fddb406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.229+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fddb406ca90 0x7fddb406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.229+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fddc008d0e0 con 0x7fddc8154c90 2024-04-26T22:10:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.230+0000 7fddc7fff640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fddb406ca90 0x7fddb406ef50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fddb8007fd0 tx=0x7fddb8034040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.234+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddc0019030 con 0x7fddc8154c90 2024-04-26T22:10:17.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.404+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddc813b880 con 0x7fddb406ca90 2024-04-26T22:10:17.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.415+0000 7fddc4ff9640 1 -- 172.21.15.135:0/3539792753 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494622 (secure 0 0 0) 0x7fddc813b880 con 0x7fddb406ca90 2024-04-26T22:10:17.421 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:17.426 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:10:17.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fddb406ca90 msgr2=0x7fddb406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fddb406ca90 0x7fddb406ef50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fddb8007fd0 tx=0x7fddb8034040 comp rx=0 tx=0).stop 2024-04-26T22:10:17.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 msgr2=0x7fddc8137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 0x7fddc8137e00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fddc0011c80 tx=0x7fddc000f580 comp rx=0 tx=0).stop 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 shutdown_connections 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fddb406ca90 0x7fddb406ef50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddc8154c90 0x7fddc8137e00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc80ac390 0x7fddc806cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.419+0000 7fddcea50640 1 --2- 172.21.15.135:0/3539792753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb0003b70 0x7fddc80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.420+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 >> 172.21.15.135:0/3539792753 conn(0x7fddc80c5ce0 msgr2=0x7fddc806f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.420+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 shutdown_connections 2024-04-26T22:10:17.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.420+0000 7fddcea50640 1 -- 172.21.15.135:0/3539792753 wait complete. 2024-04-26T22:10:17.430 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-04-26T22:10:17.308801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252980251,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":3350,"num_read_kb":829546,"num_write":9253298,"num_write_kb":113906494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610755,"ondisk_log_size":610755,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6029464,"kb_used_data":497752,"kb_used_omap":12,"kb_used_meta":5531635,"kb_avail":748945256,"statfs":{"total":773094113280,"available":766919942144,"internally_reserved":0,"allocated":509698048,"data_stored":507714988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":5664394801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20190116710,"num_objects":-5212,"num_object_clones":0,"num_object_copies":-10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5212,"num_whiteouts":0,"num_read":581,"num_read_kb":84526,"num_write":5421,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003200"},"pg_stats":[{"pgid":"5.18","version":"40'4026","reported_seq":4100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.983679+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:10:11.983679+0000","last_peered":"2024-04-26T22:10:11.983679+0000","last_clean":"2024-04-26T22:10:11.983679+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:10:11.983679+0000","last_undegraded":"2024-04-26T22:10:11.983679+0000","last_fullsized":"2024-04-26T22:10:11.983679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":60598,"num_write_kb":740487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"39'27","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.941276+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:10:00.941276+0000","last_peered":"2024-04-26T22:10:00.941276+0000","last_clean":"2024-04-26T22:10:00.941276+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:10:00.941276+0000","last_undegraded":"2024-04-26T22:10:00.941276+0000","last_fullsized":"2024-04-26T22:10:00.941276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":16408,"num_write":20,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'5701","reported_seq":5758,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.764348+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:10:05.764348+0000","last_peered":"2024-04-26T22:10:05.764348+0000","last_clean":"2024-04-26T22:10:05.764348+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:10:05.764348+0000","last_undegraded":"2024-04-26T22:10:05.764348+0000","last_fullsized":"2024-04-26T22:10:05.764348+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86528,"num_write_kb":1106988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"40'19","reported_seq":69,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.727806+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:10:10.727806+0000","last_peered":"2024-04-26T22:10:10.727806+0000","last_clean":"2024-04-26T22:10:10.727806+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:10:10.727806+0000","last_undegraded":"2024-04-26T22:10:10.727806+0000","last_fullsized":"2024-04-26T22:10:10.727806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189327,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"40'4869","reported_seq":4957,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988095+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:10:11.988095+0000","last_peered":"2024-04-26T22:10:11.988095+0000","last_clean":"2024-04-26T22:10:11.988095+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:10:11.988095+0000","last_undegraded":"2024-04-26T22:10:11.988095+0000","last_fullsized":"2024-04-26T22:10:11.988095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":73297,"num_write_kb":899566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"40'20","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857789+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:10:05.857789+0000","last_peered":"2024-04-26T22:10:05.857789+0000","last_clean":"2024-04-26T22:10:05.857789+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:10:05.857789+0000","last_undegraded":"2024-04-26T22:10:05.857789+0000","last_fullsized":"2024-04-26T22:10:05.857789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"40'6511","reported_seq":6573,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848823+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:10:05.848823+0000","last_peered":"2024-04-26T22:10:05.848823+0000","last_clean":"2024-04-26T22:10:05.848823+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:10:05.848823+0000","last_undegraded":"2024-04-26T22:10:05.848823+0000","last_fullsized":"2024-04-26T22:10:05.848823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6511,"log_dups_size":0,"ondisk_log_size":6511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":98412,"num_write_kb":1227645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"40'132","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.904919+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:10:11.904919+0000","last_peered":"2024-04-26T22:10:11.904919+0000","last_clean":"2024-04-26T22:10:11.904919+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:10:11.904919+0000","last_undegraded":"2024-04-26T22:10:11.904919+0000","last_fullsized":"2024-04-26T22:10:11.904919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":127,"num_write_kb":8276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"40'3508","reported_seq":3574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.949165+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:10:11.949165+0000","last_peered":"2024-04-26T22:10:11.949165+0000","last_clean":"2024-04-26T22:10:11.949165+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:10:11.949165+0000","last_undegraded":"2024-04-26T22:10:11.949165+0000","last_fullsized":"2024-04-26T22:10:11.949165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3508,"log_dups_size":0,"ondisk_log_size":3508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":52619,"num_write_kb":632470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"40'116","reported_seq":197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.110609+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:10:10.110609+0000","last_peered":"2024-04-26T22:10:10.110609+0000","last_clean":"2024-04-26T22:10:10.110609+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:10:10.110609+0000","last_undegraded":"2024-04-26T22:10:10.110609+0000","last_fullsized":"2024-04-26T22:10:10.110609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":45092,"num_write":102,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"40'3918","reported_seq":3975,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.754455+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:10:05.754455+0000","last_peered":"2024-04-26T22:10:05.754455+0000","last_clean":"2024-04-26T22:10:05.754455+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:10:05.754455+0000","last_undegraded":"2024-04-26T22:10:05.754455+0000","last_fullsized":"2024-04-26T22:10:05.754455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":59308,"num_write_kb":714963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"40'1268","reported_seq":1325,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.959833+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:10:00.959833+0000","last_peered":"2024-04-26T22:10:00.959833+0000","last_clean":"2024-04-26T22:10:00.959833+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:10:00.959833+0000","last_undegraded":"2024-04-26T22:10:00.959833+0000","last_fullsized":"2024-04-26T22:10:00.959833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1268,"log_dups_size":0,"ondisk_log_size":1268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":1265,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'3315","reported_seq":3384,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.932922+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:10:11.932922+0000","last_peered":"2024-04-26T22:10:11.932922+0000","last_clean":"2024-04-26T22:10:11.932922+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:10:11.932922+0000","last_undegraded":"2024-04-26T22:10:11.932922+0000","last_fullsized":"2024-04-26T22:10:11.932922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3315,"log_dups_size":0,"ondisk_log_size":3315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":49749,"num_write_kb":591259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"40'20","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.954736+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:10:01.954736+0000","last_peered":"2024-04-26T22:10:01.954736+0000","last_clean":"2024-04-26T22:10:01.954736+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:10:01.954736+0000","last_undegraded":"2024-04-26T22:10:01.954736+0000","last_fullsized":"2024-04-26T22:10:01.954736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":18,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"40'6620","reported_seq":6664,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.806952+0000","last_change":"2024-04-26T22:00:39.151257+0000","last_active":"2024-04-26T22:10:05.806952+0000","last_peered":"2024-04-26T22:10:05.806952+0000","last_clean":"2024-04-26T22:10:05.806952+0000","last_became_active":"2024-04-26T22:00:39.151025+0000","last_became_peered":"2024-04-26T22:00:39.151025+0000","last_unstale":"2024-04-26T22:10:05.806952+0000","last_undegraded":"2024-04-26T22:10:05.806952+0000","last_fullsized":"2024-04-26T22:10:05.806952+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:06.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":100009,"num_write_kb":1224131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"40'41","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955354+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:10:11.955354+0000","last_peered":"2024-04-26T22:10:11.955354+0000","last_clean":"2024-04-26T22:10:11.955354+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:10:11.955354+0000","last_undegraded":"2024-04-26T22:10:11.955354+0000","last_fullsized":"2024-04-26T22:10:11.955354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":30,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"40'4367","reported_seq":4447,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.965935+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:10:11.965935+0000","last_peered":"2024-04-26T22:10:11.965935+0000","last_clean":"2024-04-26T22:10:11.965935+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:10:11.965935+0000","last_undegraded":"2024-04-26T22:10:11.965935+0000","last_fullsized":"2024-04-26T22:10:11.965935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65356,"num_write_kb":790948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"40'4740","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.661582+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:10:05.661582+0000","last_peered":"2024-04-26T22:10:05.661582+0000","last_clean":"2024-04-26T22:10:05.661582+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:10:05.661582+0000","last_undegraded":"2024-04-26T22:10:05.661582+0000","last_fullsized":"2024-04-26T22:10:05.661582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":71515,"num_write_kb":926848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"40'206","reported_seq":280,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.865453+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:10:05.865453+0000","last_peered":"2024-04-26T22:10:05.865453+0000","last_clean":"2024-04-26T22:10:05.865453+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:10:05.865453+0000","last_undegraded":"2024-04-26T22:10:05.865453+0000","last_fullsized":"2024-04-26T22:10:05.865453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":29169,"num_write":192,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"40'39","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.495965+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:10:01.495965+0000","last_peered":"2024-04-26T22:10:01.495965+0000","last_clean":"2024-04-26T22:10:01.495965+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:10:01.495965+0000","last_undegraded":"2024-04-26T22:10:01.495965+0000","last_fullsized":"2024-04-26T22:10:01.495965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":13991,"num_write":36,"num_write_kb":4124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'3984","reported_seq":4061,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986082+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:10:11.986082+0000","last_peered":"2024-04-26T22:10:11.986082+0000","last_clean":"2024-04-26T22:10:11.986082+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:10:11.986082+0000","last_undegraded":"2024-04-26T22:10:11.986082+0000","last_fullsized":"2024-04-26T22:10:11.986082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59797,"num_write_kb":746756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"40'4602","reported_seq":4661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852754+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:10:05.852754+0000","last_peered":"2024-04-26T22:10:05.852754+0000","last_clean":"2024-04-26T22:10:05.852754+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:10:05.852754+0000","last_undegraded":"2024-04-26T22:10:05.852754+0000","last_fullsized":"2024-04-26T22:10:05.852754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69468,"num_write_kb":843448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"40'24","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.828651+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:10:05.828651+0000","last_peered":"2024-04-26T22:10:05.828651+0000","last_clean":"2024-04-26T22:10:05.828651+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:10:05.828651+0000","last_undegraded":"2024-04-26T22:10:05.828651+0000","last_fullsized":"2024-04-26T22:10:05.828651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":20,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937503+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:10:11.937503+0000","last_peered":"2024-04-26T22:10:11.937503+0000","last_clean":"2024-04-26T22:10:11.937503+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:10:11.937503+0000","last_undegraded":"2024-04-26T22:10:11.937503+0000","last_fullsized":"2024-04-26T22:10:11.937503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"40'3976","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.916627+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:10:11.916627+0000","last_peered":"2024-04-26T22:10:11.916627+0000","last_clean":"2024-04-26T22:10:11.916627+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:10:11.916627+0000","last_undegraded":"2024-04-26T22:10:11.916627+0000","last_fullsized":"2024-04-26T22:10:11.916627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3976,"log_dups_size":0,"ondisk_log_size":3976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59917,"num_write_kb":706723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"40'5423","reported_seq":5480,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.761465+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:10:05.761465+0000","last_peered":"2024-04-26T22:10:05.761465+0000","last_clean":"2024-04-26T22:10:05.761465+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:10:05.761465+0000","last_undegraded":"2024-04-26T22:10:05.761465+0000","last_fullsized":"2024-04-26T22:10:05.761465+0000","mapping_epoch":18,"log_start":"28'200","ondisk_log_start":"28'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":81931,"num_write_kb":1035269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'44","reported_seq":104,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.415234+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:10:14.415234+0000","last_peered":"2024-04-26T22:10:14.415234+0000","last_clean":"2024-04-26T22:10:14.415234+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:10:14.415234+0000","last_undegraded":"2024-04-26T22:10:14.415234+0000","last_fullsized":"2024-04-26T22:10:14.415234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363115,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":324,"num_write":38,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"40'85","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.973183+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:10:11.973183+0000","last_peered":"2024-04-26T22:10:11.973183+0000","last_clean":"2024-04-26T22:10:11.973183+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:10:11.973183+0000","last_undegraded":"2024-04-26T22:10:11.973183+0000","last_fullsized":"2024-04-26T22:10:11.973183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":6841,"num_write":81,"num_write_kb":4168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"40'4751","reported_seq":4829,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937603+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:10:11.937603+0000","last_peered":"2024-04-26T22:10:11.937603+0000","last_clean":"2024-04-26T22:10:11.937603+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:10:11.937603+0000","last_undegraded":"2024-04-26T22:10:11.937603+0000","last_fullsized":"2024-04-26T22:10:11.937603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":71494,"num_write_kb":858502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"40'5174","reported_seq":5233,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.821492+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:10:05.821492+0000","last_peered":"2024-04-26T22:10:05.821492+0000","last_clean":"2024-04-26T22:10:05.821492+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:10:05.821492+0000","last_undegraded":"2024-04-26T22:10:05.821492+0000","last_fullsized":"2024-04-26T22:10:05.821492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":78246,"num_write_kb":941647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"40'8","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823374+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:10:05.823374+0000","last_peered":"2024-04-26T22:10:05.823374+0000","last_clean":"2024-04-26T22:10:05.823374+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:10:05.823374+0000","last_undegraded":"2024-04-26T22:10:05.823374+0000","last_fullsized":"2024-04-26T22:10:05.823374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":4,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"40'37","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962744+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:10:11.962744+0000","last_peered":"2024-04-26T22:10:11.962744+0000","last_clean":"2024-04-26T22:10:11.962744+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:10:11.962744+0000","last_undegraded":"2024-04-26T22:10:11.962744+0000","last_fullsized":"2024-04-26T22:10:11.962744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":31,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"40'3913","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962293+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:10:11.962293+0000","last_peered":"2024-04-26T22:10:11.962293+0000","last_clean":"2024-04-26T22:10:11.962293+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:10:11.962293+0000","last_undegraded":"2024-04-26T22:10:11.962293+0000","last_fullsized":"2024-04-26T22:10:11.962293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":58791,"num_write_kb":740377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'4599","reported_seq":4657,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.774768+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:10:05.774768+0000","last_peered":"2024-04-26T22:10:05.774768+0000","last_clean":"2024-04-26T22:10:05.774768+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:10:05.774768+0000","last_undegraded":"2024-04-26T22:10:05.774768+0000","last_fullsized":"2024-04-26T22:10:05.774768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69140,"num_write_kb":847801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"40'8","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836838+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:10:05.836838+0000","last_peered":"2024-04-26T22:10:05.836838+0000","last_clean":"2024-04-26T22:10:05.836838+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:10:05.836838+0000","last_undegraded":"2024-04-26T22:10:05.836838+0000","last_fullsized":"2024-04-26T22:10:05.836838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"40'23","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.979062+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:10:14.979062+0000","last_peered":"2024-04-26T22:10:14.979062+0000","last_clean":"2024-04-26T22:10:14.979062+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:10:14.979062+0000","last_undegraded":"2024-04-26T22:10:14.979062+0000","last_fullsized":"2024-04-26T22:10:14.979062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":98,"num_write":20,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"40'3929","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.938066+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:10:11.938066+0000","last_peered":"2024-04-26T22:10:11.938066+0000","last_clean":"2024-04-26T22:10:11.938066+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:10:11.938066+0000","last_undegraded":"2024-04-26T22:10:11.938066+0000","last_fullsized":"2024-04-26T22:10:11.938066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":59023,"num_write_kb":718389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'5311","reported_seq":5368,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.831429+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:10:05.831429+0000","last_peered":"2024-04-26T22:10:05.831429+0000","last_clean":"2024-04-26T22:10:05.831429+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:10:05.831429+0000","last_undegraded":"2024-04-26T22:10:05.831429+0000","last_fullsized":"2024-04-26T22:10:05.831429+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80379,"num_write_kb":1004221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'218","reported_seq":282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.885693+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:10:14.885693+0000","last_peered":"2024-04-26T22:10:14.885693+0000","last_clean":"2024-04-26T22:10:14.885693+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:10:14.885693+0000","last_undegraded":"2024-04-26T22:10:14.885693+0000","last_fullsized":"2024-04-26T22:10:14.885693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":61,"num_write":416,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"40'22","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.961590+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:10:11.961590+0000","last_peered":"2024-04-26T22:10:11.961590+0000","last_clean":"2024-04-26T22:10:11.961590+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:10:11.961590+0000","last_undegraded":"2024-04-26T22:10:11.961590+0000","last_fullsized":"2024-04-26T22:10:11.961590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":15,"num_write_kb":8199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"40'4334","reported_seq":4409,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.892208+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:10:11.892208+0000","last_peered":"2024-04-26T22:10:11.892208+0000","last_clean":"2024-04-26T22:10:11.892208+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:10:11.892208+0000","last_undegraded":"2024-04-26T22:10:11.892208+0000","last_fullsized":"2024-04-26T22:10:11.892208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":65365,"num_write_kb":782662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"40'5496","reported_seq":5556,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.837207+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:10:05.837207+0000","last_peered":"2024-04-26T22:10:05.837207+0000","last_clean":"2024-04-26T22:10:05.837207+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:10:05.837207+0000","last_undegraded":"2024-04-26T22:10:05.837207+0000","last_fullsized":"2024-04-26T22:10:05.837207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":82892,"num_write_kb":1050745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'35","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.834069+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:10:05.834069+0000","last_peered":"2024-04-26T22:10:05.834069+0000","last_clean":"2024-04-26T22:10:05.834069+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:10:05.834069+0000","last_undegraded":"2024-04-26T22:10:05.834069+0000","last_fullsized":"2024-04-26T22:10:05.834069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":29,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"28'1","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940808+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:10:00.940808+0000","last_peered":"2024-04-26T22:10:00.940808+0000","last_clean":"2024-04-26T22:10:00.940808+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:10:00.940808+0000","last_undegraded":"2024-04-26T22:10:00.940808+0000","last_fullsized":"2024-04-26T22:10:00.940808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"40'3789","reported_seq":3862,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937218+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:10:11.937218+0000","last_peered":"2024-04-26T22:10:11.937218+0000","last_clean":"2024-04-26T22:10:11.937218+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:10:11.937218+0000","last_undegraded":"2024-04-26T22:10:11.937218+0000","last_fullsized":"2024-04-26T22:10:11.937218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":56913,"num_write_kb":707433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"40'4759","reported_seq":4817,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803693+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:10:05.803693+0000","last_peered":"2024-04-26T22:10:05.803693+0000","last_clean":"2024-04-26T22:10:05.803693+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:10:05.803693+0000","last_undegraded":"2024-04-26T22:10:05.803693+0000","last_fullsized":"2024-04-26T22:10:05.803693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71808,"num_write_kb":921006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"40'7","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.825804+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:10:05.825804+0000","last_peered":"2024-04-26T22:10:05.825804+0000","last_clean":"2024-04-26T22:10:05.825804+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:10:05.825804+0000","last_undegraded":"2024-04-26T22:10:05.825804+0000","last_fullsized":"2024-04-26T22:10:05.825804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"40'183","reported_seq":241,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966669+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:10:11.966669+0000","last_peered":"2024-04-26T22:10:11.966669+0000","last_clean":"2024-04-26T22:10:11.966669+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:10:11.966669+0000","last_undegraded":"2024-04-26T22:10:11.966669+0000","last_fullsized":"2024-04-26T22:10:11.966669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":27847,"num_write":173,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"40'3541","reported_seq":3610,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940587+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:10:11.940587+0000","last_peered":"2024-04-26T22:10:11.940587+0000","last_clean":"2024-04-26T22:10:11.940587+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:10:11.940587+0000","last_undegraded":"2024-04-26T22:10:11.940587+0000","last_fullsized":"2024-04-26T22:10:11.940587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":53073,"num_write_kb":661602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'5106","reported_seq":5164,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.813565+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:10:05.813565+0000","last_peered":"2024-04-26T22:10:05.813565+0000","last_clean":"2024-04-26T22:10:05.813565+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:10:05.813565+0000","last_undegraded":"2024-04-26T22:10:05.813565+0000","last_fullsized":"2024-04-26T22:10:05.813565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":77293,"num_write_kb":982493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"40'221","reported_seq":286,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850755+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:10:05.850755+0000","last_peered":"2024-04-26T22:10:05.850755+0000","last_clean":"2024-04-26T22:10:05.850755+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:10:05.850755+0000","last_undegraded":"2024-04-26T22:10:05.850755+0000","last_fullsized":"2024-04-26T22:10:05.850755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":213,"num_write_kb":8367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"40'5","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.945512+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:10:11.945512+0000","last_peered":"2024-04-26T22:10:11.945512+0000","last_clean":"2024-04-26T22:10:11.945512+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:10:11.945512+0000","last_undegraded":"2024-04-26T22:10:11.945512+0000","last_fullsized":"2024-04-26T22:10:11.945512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"40'3541","reported_seq":3614,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.910214+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:10:11.910214+0000","last_peered":"2024-04-26T22:10:11.910214+0000","last_clean":"2024-04-26T22:10:11.910214+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:10:11.910214+0000","last_undegraded":"2024-04-26T22:10:11.910214+0000","last_fullsized":"2024-04-26T22:10:11.910214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":52805,"num_write_kb":629973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'5114","reported_seq":5174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857162+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:10:05.857162+0000","last_peered":"2024-04-26T22:10:05.857162+0000","last_clean":"2024-04-26T22:10:05.857162+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:10:05.857162+0000","last_undegraded":"2024-04-26T22:10:05.857162+0000","last_fullsized":"2024-04-26T22:10:05.857162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":77095,"num_write_kb":967489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"40'19","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.889177+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:10:14.889177+0000","last_peered":"2024-04-26T22:10:14.889177+0000","last_clean":"2024-04-26T22:10:14.889177+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:10:14.889177+0000","last_undegraded":"2024-04-26T22:10:14.889177+0000","last_fullsized":"2024-04-26T22:10:14.889177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88181,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":134,"num_write":15,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"40'5","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954456+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:10:11.954456+0000","last_peered":"2024-04-26T22:10:11.954456+0000","last_clean":"2024-04-26T22:10:11.954456+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:10:11.954456+0000","last_undegraded":"2024-04-26T22:10:11.954456+0000","last_fullsized":"2024-04-26T22:10:11.954456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'4139","reported_seq":4216,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.947123+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:10:11.947123+0000","last_peered":"2024-04-26T22:10:11.947123+0000","last_clean":"2024-04-26T22:10:11.947123+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:10:11.947123+0000","last_undegraded":"2024-04-26T22:10:11.947123+0000","last_fullsized":"2024-04-26T22:10:11.947123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":62199,"num_write_kb":728313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"40'5239","reported_seq":5297,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810597+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:10:05.810597+0000","last_peered":"2024-04-26T22:10:05.810597+0000","last_clean":"2024-04-26T22:10:05.810597+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:10:05.810597+0000","last_undegraded":"2024-04-26T22:10:05.810597+0000","last_fullsized":"2024-04-26T22:10:05.810597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":79195,"num_write_kb":994121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"40'30","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.863131+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:10:05.863131+0000","last_peered":"2024-04-26T22:10:05.863131+0000","last_clean":"2024-04-26T22:10:05.863131+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:10:05.863131+0000","last_undegraded":"2024-04-26T22:10:05.863131+0000","last_fullsized":"2024-04-26T22:10:05.863131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":8201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"40'211","reported_seq":268,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.956625+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:10:11.956625+0000","last_peered":"2024-04-26T22:10:11.956625+0000","last_clean":"2024-04-26T22:10:11.956625+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:10:11.956625+0000","last_undegraded":"2024-04-26T22:10:11.956625+0000","last_fullsized":"2024-04-26T22:10:11.956625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":74,"num_write":424,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"40'4145","reported_seq":4221,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.957211+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:10:11.957211+0000","last_peered":"2024-04-26T22:10:11.957211+0000","last_clean":"2024-04-26T22:10:11.957211+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:10:11.957211+0000","last_undegraded":"2024-04-26T22:10:11.957211+0000","last_fullsized":"2024-04-26T22:10:11.957211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":62372,"num_write_kb":775201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"40'5748","reported_seq":5805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.762804+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:10:05.762804+0000","last_peered":"2024-04-26T22:10:05.762804+0000","last_clean":"2024-04-26T22:10:05.762804+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:10:05.762804+0000","last_undegraded":"2024-04-26T22:10:05.762804+0000","last_fullsized":"2024-04-26T22:10:05.762804+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":87080,"num_write_kb":1065746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"39'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940251+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:10:00.940251+0000","last_peered":"2024-04-26T22:10:00.940251+0000","last_clean":"2024-04-26T22:10:00.940251+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:10:00.940251+0000","last_undegraded":"2024-04-26T22:10:00.940251+0000","last_fullsized":"2024-04-26T22:10:00.940251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.221589+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:10:01.221589+0000","last_peered":"2024-04-26T22:10:01.221589+0000","last_clean":"2024-04-26T22:10:01.221589+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:10:01.221589+0000","last_undegraded":"2024-04-26T22:10:01.221589+0000","last_fullsized":"2024-04-26T22:10:01.221589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.944318+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:10:11.944318+0000","last_peered":"2024-04-26T22:10:11.944318+0000","last_clean":"2024-04-26T22:10:11.944318+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:10:11.944318+0000","last_undegraded":"2024-04-26T22:10:11.944318+0000","last_fullsized":"2024-04-26T22:10:11.944318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"40'3379","reported_seq":3460,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959450+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:10:11.959450+0000","last_peered":"2024-04-26T22:10:11.959450+0000","last_clean":"2024-04-26T22:10:11.959450+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:10:11.959450+0000","last_undegraded":"2024-04-26T22:10:11.959450+0000","last_fullsized":"2024-04-26T22:10:11.959450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":50792,"num_write_kb":610820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"40'5433","reported_seq":5494,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.869142+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:10:05.869142+0000","last_peered":"2024-04-26T22:10:05.869142+0000","last_clean":"2024-04-26T22:10:05.869142+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:10:05.869142+0000","last_undegraded":"2024-04-26T22:10:05.869142+0000","last_fullsized":"2024-04-26T22:10:05.869142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":81848,"num_write_kb":1014508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"40'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810386+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:10:05.810386+0000","last_peered":"2024-04-26T22:10:05.810386+0000","last_clean":"2024-04-26T22:10:05.810386+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:10:05.810386+0000","last_undegraded":"2024-04-26T22:10:05.810386+0000","last_fullsized":"2024-04-26T22:10:05.810386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"40'201","reported_seq":252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.948751+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:10:11.948751+0000","last_peered":"2024-04-26T22:10:11.948751+0000","last_clean":"2024-04-26T22:10:11.948751+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:10:11.948751+0000","last_undegraded":"2024-04-26T22:10:11.948751+0000","last_fullsized":"2024-04-26T22:10:11.948751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":196,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"40'3780","reported_seq":3857,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.815165+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:10:11.815165+0000","last_peered":"2024-04-26T22:10:11.815165+0000","last_clean":"2024-04-26T22:10:11.815165+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:10:11.815165+0000","last_undegraded":"2024-04-26T22:10:11.815165+0000","last_fullsized":"2024-04-26T22:10:11.815165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":56837,"num_write_kb":699720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"40'5733","reported_seq":5794,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849434+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:10:05.849434+0000","last_peered":"2024-04-26T22:10:05.849434+0000","last_clean":"2024-04-26T22:10:05.849434+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:10:05.849434+0000","last_undegraded":"2024-04-26T22:10:05.849434+0000","last_fullsized":"2024-04-26T22:10:05.849434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5733,"log_dups_size":0,"ondisk_log_size":5733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":86585,"num_write_kb":1068440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"40'19","reported_seq":88,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.943192+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:10:00.943192+0000","last_peered":"2024-04-26T22:10:00.943192+0000","last_clean":"2024-04-26T22:10:00.943192+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:10:00.943192+0000","last_undegraded":"2024-04-26T22:10:00.943192+0000","last_fullsized":"2024-04-26T22:10:00.943192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16390,"num_write":17,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"40'379","reported_seq":446,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955307+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:10:11.955307+0000","last_peered":"2024-04-26T22:10:11.955307+0000","last_clean":"2024-04-26T22:10:11.955307+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:10:11.955307+0000","last_undegraded":"2024-04-26T22:10:11.955307+0000","last_fullsized":"2024-04-26T22:10:11.955307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":25852,"num_write":364,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"40'3909","reported_seq":3987,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946493+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:10:11.946493+0000","last_peered":"2024-04-26T22:10:11.946493+0000","last_clean":"2024-04-26T22:10:11.946493+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:10:11.946493+0000","last_undegraded":"2024-04-26T22:10:11.946493+0000","last_fullsized":"2024-04-26T22:10:11.946493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":58819,"num_write_kb":699051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"40'6248","reported_seq":6306,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823315+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:10:05.823315+0000","last_peered":"2024-04-26T22:10:05.823315+0000","last_clean":"2024-04-26T22:10:05.823315+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:10:05.823315+0000","last_undegraded":"2024-04-26T22:10:05.823315+0000","last_fullsized":"2024-04-26T22:10:05.823315+0000","mapping_epoch":18,"log_start":"28'400","ondisk_log_start":"28'400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":94569,"num_write_kb":1160134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"40'111","reported_seq":183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850721+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:10:05.850721+0000","last_peered":"2024-04-26T22:10:05.850721+0000","last_clean":"2024-04-26T22:10:05.850721+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:10:05.850721+0000","last_undegraded":"2024-04-26T22:10:05.850721+0000","last_fullsized":"2024-04-26T22:10:05.850721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":106,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"40'13","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.935130+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:10:11.935130+0000","last_peered":"2024-04-26T22:10:11.935130+0000","last_clean":"2024-04-26T22:10:11.935130+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:10:11.935130+0000","last_undegraded":"2024-04-26T22:10:11.935130+0000","last_fullsized":"2024-04-26T22:10:11.935130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"40'4399","reported_seq":4473,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.897004+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:10:11.897004+0000","last_peered":"2024-04-26T22:10:11.897004+0000","last_clean":"2024-04-26T22:10:11.897004+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:10:11.897004+0000","last_undegraded":"2024-04-26T22:10:11.897004+0000","last_fullsized":"2024-04-26T22:10:11.897004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":66147,"num_write_kb":822005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"40'5224","reported_seq":5282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822320+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:10:05.822320+0000","last_peered":"2024-04-26T22:10:05.822320+0000","last_clean":"2024-04-26T22:10:05.822320+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:10:05.822320+0000","last_undegraded":"2024-04-26T22:10:05.822320+0000","last_fullsized":"2024-04-26T22:10:05.822320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":78957,"num_write_kb":961695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"40'16","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849672+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:10:05.849672+0000","last_peered":"2024-04-26T22:10:05.849672+0000","last_clean":"2024-04-26T22:10:05.849672+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:10:05.849672+0000","last_undegraded":"2024-04-26T22:10:05.849672+0000","last_fullsized":"2024-04-26T22:10:05.849672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":12294,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951413+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:10:11.951413+0000","last_peered":"2024-04-26T22:10:11.951413+0000","last_clean":"2024-04-26T22:10:11.951413+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:10:11.951413+0000","last_undegraded":"2024-04-26T22:10:11.951413+0000","last_fullsized":"2024-04-26T22:10:11.951413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"40'3562","reported_seq":3638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982357+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:10:11.982357+0000","last_peered":"2024-04-26T22:10:11.982357+0000","last_clean":"2024-04-26T22:10:11.982357+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:10:11.982357+0000","last_undegraded":"2024-04-26T22:10:11.982357+0000","last_fullsized":"2024-04-26T22:10:11.982357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3562,"log_dups_size":0,"ondisk_log_size":3562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":53350,"num_write_kb":651982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"40'5336","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848168+0000","last_change":"2024-04-26T22:00:39.151213+0000","last_active":"2024-04-26T22:10:05.848168+0000","last_peered":"2024-04-26T22:10:05.848168+0000","last_clean":"2024-04-26T22:10:05.848168+0000","last_became_active":"2024-04-26T22:00:39.150992+0000","last_became_peered":"2024-04-26T22:00:39.150992+0000","last_unstale":"2024-04-26T22:10:05.848168+0000","last_undegraded":"2024-04-26T22:10:05.848168+0000","last_fullsized":"2024-04-26T22:10:05.848168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:26.086210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":80461,"num_write_kb":1019956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"40'287","reported_seq":383,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.839145+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:10:05.839145+0000","last_peered":"2024-04-26T22:10:05.839145+0000","last_clean":"2024-04-26T22:10:05.839145+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:10:05.839145+0000","last_undegraded":"2024-04-26T22:10:05.839145+0000","last_fullsized":"2024-04-26T22:10:05.839145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":12387,"num_write":284,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"40'4","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954588+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:10:11.954588+0000","last_peered":"2024-04-26T22:10:11.954588+0000","last_clean":"2024-04-26T22:10:11.954588+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:10:11.954588+0000","last_undegraded":"2024-04-26T22:10:11.954588+0000","last_fullsized":"2024-04-26T22:10:11.954588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"40'4185","reported_seq":4270,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946111+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:10:11.946111+0000","last_peered":"2024-04-26T22:10:11.946111+0000","last_clean":"2024-04-26T22:10:11.946111+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:10:11.946111+0000","last_undegraded":"2024-04-26T22:10:11.946111+0000","last_fullsized":"2024-04-26T22:10:11.946111+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":62914,"num_write_kb":762533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'5750","reported_seq":5810,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.854474+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:10:05.854474+0000","last_peered":"2024-04-26T22:10:05.854474+0000","last_clean":"2024-04-26T22:10:05.854474+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:10:05.854474+0000","last_undegraded":"2024-04-26T22:10:05.854474+0000","last_fullsized":"2024-04-26T22:10:05.854474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":86794,"num_write_kb":1073964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"40'6","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836648+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:10:05.836648+0000","last_peered":"2024-04-26T22:10:05.836648+0000","last_clean":"2024-04-26T22:10:05.836648+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:10:05.836648+0000","last_undegraded":"2024-04-26T22:10:05.836648+0000","last_fullsized":"2024-04-26T22:10:05.836648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.943365+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:10:07.943365+0000","last_peered":"2024-04-26T22:10:07.943365+0000","last_clean":"2024-04-26T22:10:07.943365+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:10:07.943365+0000","last_undegraded":"2024-04-26T22:10:07.943365+0000","last_fullsized":"2024-04-26T22:10:07.943365+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"40'3717","reported_seq":3786,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.778102+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:10:11.778102+0000","last_peered":"2024-04-26T22:10:11.778102+0000","last_clean":"2024-04-26T22:10:11.778102+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:10:11.778102+0000","last_undegraded":"2024-04-26T22:10:11.778102+0000","last_fullsized":"2024-04-26T22:10:11.778102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3717,"log_dups_size":0,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":55831,"num_write_kb":673382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"40'6101","reported_seq":6161,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868996+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:10:05.868996+0000","last_peered":"2024-04-26T22:10:05.868996+0000","last_clean":"2024-04-26T22:10:05.868996+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:10:05.868996+0000","last_undegraded":"2024-04-26T22:10:05.868996+0000","last_fullsized":"2024-04-26T22:10:05.868996+0000","mapping_epoch":18,"log_start":"28'300","ondisk_log_start":"28'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":92157,"num_write_kb":1133252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"40'5","reported_seq":63,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867571+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:10:05.867571+0000","last_peered":"2024-04-26T22:10:05.867571+0000","last_clean":"2024-04-26T22:10:05.867571+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:10:05.867571+0000","last_undegraded":"2024-04-26T22:10:05.867571+0000","last_fullsized":"2024-04-26T22:10:05.867571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"40'374","reported_seq":467,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953183+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:10:11.953183+0000","last_peered":"2024-04-26T22:10:11.953183+0000","last_clean":"2024-04-26T22:10:11.953183+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:10:11.953183+0000","last_undegraded":"2024-04-26T22:10:11.953183+0000","last_fullsized":"2024-04-26T22:10:11.953183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":31157,"num_write":363,"num_write_kb":8442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"40'5464","reported_seq":5545,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.917366+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:10:11.917366+0000","last_peered":"2024-04-26T22:10:11.917366+0000","last_clean":"2024-04-26T22:10:11.917366+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:10:11.917366+0000","last_undegraded":"2024-04-26T22:10:11.917366+0000","last_fullsized":"2024-04-26T22:10:11.917366+0000","mapping_epoch":22,"log_start":"31'200","ondisk_log_start":"31'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":82133,"num_write_kb":980051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'5077","reported_seq":5133,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673410+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:10:05.673410+0000","last_peered":"2024-04-26T22:10:05.673410+0000","last_clean":"2024-04-26T22:10:05.673410+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:10:05.673410+0000","last_undegraded":"2024-04-26T22:10:05.673410+0000","last_fullsized":"2024-04-26T22:10:05.673410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":76618,"num_write_kb":958363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"40'19","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852434+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:10:05.852434+0000","last_peered":"2024-04-26T22:10:05.852434+0000","last_clean":"2024-04-26T22:10:05.852434+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:10:05.852434+0000","last_undegraded":"2024-04-26T22:10:05.852434+0000","last_fullsized":"2024-04-26T22:10:05.852434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":32816,"num_write":13,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"40'15","reported_seq":89,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.994445+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:10:11.994445+0000","last_peered":"2024-04-26T22:10:11.994445+0000","last_clean":"2024-04-26T22:10:11.994445+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:10:11.994445+0000","last_undegraded":"2024-04-26T22:10:11.994445+0000","last_fullsized":"2024-04-26T22:10:11.994445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"40'4714","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982824+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:10:11.982824+0000","last_peered":"2024-04-26T22:10:11.982824+0000","last_clean":"2024-04-26T22:10:11.982824+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:10:11.982824+0000","last_undegraded":"2024-04-26T22:10:11.982824+0000","last_fullsized":"2024-04-26T22:10:11.982824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":70866,"num_write_kb":843772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"40'5114","reported_seq":5170,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.579883+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:10:05.579883+0000","last_peered":"2024-04-26T22:10:05.579883+0000","last_clean":"2024-04-26T22:10:05.579883+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:10:05.579883+0000","last_undegraded":"2024-04-26T22:10:05.579883+0000","last_fullsized":"2024-04-26T22:10:05.579883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77180,"num_write_kb":958350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"40'824","reported_seq":917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848033+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:10:05.848033+0000","last_peered":"2024-04-26T22:10:05.848033+0000","last_clean":"2024-04-26T22:10:05.848033+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:10:05.848033+0000","last_undegraded":"2024-04-26T22:10:05.848033+0000","last_fullsized":"2024-04-26T22:10:05.848033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":36914,"num_write":813,"num_write_kb":13003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"40'5","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977919+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:10:11.977919+0000","last_peered":"2024-04-26T22:10:11.977919+0000","last_clean":"2024-04-26T22:10:11.977919+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:10:11.977919+0000","last_undegraded":"2024-04-26T22:10:11.977919+0000","last_fullsized":"2024-04-26T22:10:11.977919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"40'3745","reported_seq":3823,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.841511+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:10:11.841511+0000","last_peered":"2024-04-26T22:10:11.841511+0000","last_clean":"2024-04-26T22:10:11.841511+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:10:11.841511+0000","last_undegraded":"2024-04-26T22:10:11.841511+0000","last_fullsized":"2024-04-26T22:10:11.841511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56184,"num_write_kb":680067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"40'7267","reported_seq":7326,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824603+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:10:05.824603+0000","last_peered":"2024-04-26T22:10:05.824603+0000","last_clean":"2024-04-26T22:10:05.824603+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:10:05.824603+0000","last_undegraded":"2024-04-26T22:10:05.824603+0000","last_fullsized":"2024-04-26T22:10:05.824603+0000","mapping_epoch":18,"log_start":"30'1800","ondisk_log_start":"30'1800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":109924,"num_write_kb":1323267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"40'370","reported_seq":447,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.881135+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:10:05.881135+0000","last_peered":"2024-04-26T22:10:05.881135+0000","last_clean":"2024-04-26T22:10:05.881135+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:10:05.881135+0000","last_undegraded":"2024-04-26T22:10:05.881135+0000","last_fullsized":"2024-04-26T22:10:05.881135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":367,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"40'231","reported_seq":307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966124+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:10:11.966124+0000","last_peered":"2024-04-26T22:10:11.966124+0000","last_clean":"2024-04-26T22:10:11.966124+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:10:11.966124+0000","last_undegraded":"2024-04-26T22:10:11.966124+0000","last_fullsized":"2024-04-26T22:10:11.966124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":33257,"num_write":223,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"40'4111","reported_seq":4188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967266+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:10:11.967266+0000","last_peered":"2024-04-26T22:10:11.967266+0000","last_clean":"2024-04-26T22:10:11.967266+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:10:11.967266+0000","last_undegraded":"2024-04-26T22:10:11.967266+0000","last_fullsized":"2024-04-26T22:10:11.967266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4111,"log_dups_size":0,"ondisk_log_size":4111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61736,"num_write_kb":766472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"40'4628","reported_seq":4688,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867072+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:10:05.867072+0000","last_peered":"2024-04-26T22:10:05.867072+0000","last_clean":"2024-04-26T22:10:05.867072+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:10:05.867072+0000","last_undegraded":"2024-04-26T22:10:05.867072+0000","last_fullsized":"2024-04-26T22:10:05.867072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":69686,"num_write_kb":866678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"40'102","reported_seq":162,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842367+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:10:05.842367+0000","last_peered":"2024-04-26T22:10:05.842367+0000","last_clean":"2024-04-26T22:10:05.842367+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:10:05.842367+0000","last_undegraded":"2024-04-26T22:10:05.842367+0000","last_fullsized":"2024-04-26T22:10:05.842367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":98,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"40'12","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.943439+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:10:11.943439+0000","last_peered":"2024-04-26T22:10:11.943439+0000","last_clean":"2024-04-26T22:10:11.943439+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:10:11.943439+0000","last_undegraded":"2024-04-26T22:10:11.943439+0000","last_fullsized":"2024-04-26T22:10:11.943439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'4365","reported_seq":4453,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.968703+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:10:11.968703+0000","last_peered":"2024-04-26T22:10:11.968703+0000","last_clean":"2024-04-26T22:10:11.968703+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:10:11.968703+0000","last_undegraded":"2024-04-26T22:10:11.968703+0000","last_fullsized":"2024-04-26T22:10:11.968703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4365,"log_dups_size":0,"ondisk_log_size":4365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":65682,"num_write_kb":806692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"40'5549","reported_seq":5613,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824597+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:10:05.824597+0000","last_peered":"2024-04-26T22:10:05.824597+0000","last_clean":"2024-04-26T22:10:05.824597+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:10:05.824597+0000","last_undegraded":"2024-04-26T22:10:05.824597+0000","last_fullsized":"2024-04-26T22:10:05.824597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":83744,"num_write_kb":1000289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"40'4","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.846233+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:10:05.846233+0000","last_peered":"2024-04-26T22:10:05.846233+0000","last_clean":"2024-04-26T22:10:05.846233+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:10:05.846233+0000","last_undegraded":"2024-04-26T22:10:05.846233+0000","last_fullsized":"2024-04-26T22:10:05.846233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953872+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:10:11.953872+0000","last_peered":"2024-04-26T22:10:11.953872+0000","last_clean":"2024-04-26T22:10:11.953872+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:10:11.953872+0000","last_undegraded":"2024-04-26T22:10:11.953872+0000","last_fullsized":"2024-04-26T22:10:11.953872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4980","reported_seq":5062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967201+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:10:11.967201+0000","last_peered":"2024-04-26T22:10:11.967201+0000","last_clean":"2024-04-26T22:10:11.967201+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:10:11.967201+0000","last_undegraded":"2024-04-26T22:10:11.967201+0000","last_fullsized":"2024-04-26T22:10:11.967201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":74941,"num_write_kb":925745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"40'16","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.970996+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:10:11.970996+0000","last_peered":"2024-04-26T22:10:11.970996+0000","last_clean":"2024-04-26T22:10:11.970996+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:10:11.970996+0000","last_undegraded":"2024-04-26T22:10:11.970996+0000","last_fullsized":"2024-04-26T22:10:11.970996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":8,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"40'6345","reported_seq":6406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.871078+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:10:05.871078+0000","last_peered":"2024-04-26T22:10:05.871078+0000","last_clean":"2024-04-26T22:10:05.871078+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:10:05.871078+0000","last_undegraded":"2024-04-26T22:10:05.871078+0000","last_fullsized":"2024-04-26T22:10:05.871078+0000","mapping_epoch":18,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":95977,"num_write_kb":1176758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"40'19","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.772388+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:10:05.772388+0000","last_peered":"2024-04-26T22:10:05.772388+0000","last_clean":"2024-04-26T22:10:05.772388+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:10:05.772388+0000","last_undegraded":"2024-04-26T22:10:05.772388+0000","last_fullsized":"2024-04-26T22:10:05.772388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":28672,"num_write":16,"num_write_kb":8206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"40'4260","reported_seq":4329,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.876539+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:10:11.876539+0000","last_peered":"2024-04-26T22:10:11.876539+0000","last_clean":"2024-04-26T22:10:11.876539+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:10:11.876539+0000","last_undegraded":"2024-04-26T22:10:11.876539+0000","last_fullsized":"2024-04-26T22:10:11.876539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":64151,"num_write_kb":776804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'25","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.953756+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:10:00.953756+0000","last_peered":"2024-04-26T22:10:00.953756+0000","last_clean":"2024-04-26T22:10:00.953756+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:10:00.953756+0000","last_undegraded":"2024-04-26T22:10:00.953756+0000","last_fullsized":"2024-04-26T22:10:00.953756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20512,"num_write":20,"num_write_kb":4102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"40'5323","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874479+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:10:05.874479+0000","last_peered":"2024-04-26T22:10:05.874479+0000","last_clean":"2024-04-26T22:10:05.874479+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:10:05.874479+0000","last_undegraded":"2024-04-26T22:10:05.874479+0000","last_fullsized":"2024-04-26T22:10:05.874479+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":80432,"num_write_kb":1004371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"40'46","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.934549+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:10:00.934549+0000","last_peered":"2024-04-26T22:10:00.934549+0000","last_clean":"2024-04-26T22:10:00.934549+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:10:00.934549+0000","last_undegraded":"2024-04-26T22:10:00.934549+0000","last_fullsized":"2024-04-26T22:10:00.934549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":41,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'4905","reported_seq":4991,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940116+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:10:11.940116+0000","last_peered":"2024-04-26T22:10:11.940116+0000","last_clean":"2024-04-26T22:10:11.940116+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:10:11.940116+0000","last_undegraded":"2024-04-26T22:10:11.940116+0000","last_fullsized":"2024-04-26T22:10:11.940116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":74018,"num_write_kb":905043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"40'145","reported_seq":213,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.833871+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:10:05.833871+0000","last_peered":"2024-04-26T22:10:05.833871+0000","last_clean":"2024-04-26T22:10:05.833871+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:10:05.833871+0000","last_undegraded":"2024-04-26T22:10:05.833871+0000","last_fullsized":"2024-04-26T22:10:05.833871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":136,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"40'4250","reported_seq":4307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.802351+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:10:05.802351+0000","last_peered":"2024-04-26T22:10:05.802351+0000","last_clean":"2024-04-26T22:10:05.802351+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:10:05.802351+0000","last_undegraded":"2024-04-26T22:10:05.802351+0000","last_fullsized":"2024-04-26T22:10:05.802351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":64229,"num_write_kb":785523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"40'38","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.026729+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:10:07.026729+0000","last_peered":"2024-04-26T22:10:07.026729+0000","last_clean":"2024-04-26T22:10:07.026729+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:10:07.026729+0000","last_undegraded":"2024-04-26T22:10:07.026729+0000","last_fullsized":"2024-04-26T22:10:07.026729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":35,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2966","reported_seq":3026,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.990722+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:10:11.990722+0000","last_peered":"2024-04-26T22:10:11.990722+0000","last_clean":"2024-04-26T22:10:11.990722+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:10:11.990722+0000","last_undegraded":"2024-04-26T22:10:11.990722+0000","last_fullsized":"2024-04-26T22:10:11.990722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":2966,"log_dups_size":0,"ondisk_log_size":2966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":44534,"num_write_kb":522679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"40'1255","reported_seq":1319,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.935885+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:10:00.935885+0000","last_peered":"2024-04-26T22:10:00.935885+0000","last_clean":"2024-04-26T22:10:00.935885+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:10:00.935885+0000","last_undegraded":"2024-04-26T22:10:00.935885+0000","last_fullsized":"2024-04-26T22:10:00.935885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1255,"log_dups_size":0,"ondisk_log_size":1255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":20480,"num_write":1252,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"40'4813","reported_seq":4872,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836289+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:10:05.836289+0000","last_peered":"2024-04-26T22:10:05.836289+0000","last_clean":"2024-04-26T22:10:05.836289+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:10:05.836289+0000","last_undegraded":"2024-04-26T22:10:05.836289+0000","last_fullsized":"2024-04-26T22:10:05.836289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":72672,"num_write_kb":866290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"40'86","reported_seq":166,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953507+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:10:11.953507+0000","last_peered":"2024-04-26T22:10:11.953507+0000","last_clean":"2024-04-26T22:10:11.953507+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:10:11.953507+0000","last_undegraded":"2024-04-26T22:10:11.953507+0000","last_fullsized":"2024-04-26T22:10:11.953507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":62,"num_read_kb":49200,"num_write":70,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"40'3892","reported_seq":3964,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954209+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:10:11.954209+0000","last_peered":"2024-04-26T22:10:11.954209+0000","last_clean":"2024-04-26T22:10:11.954209+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:10:11.954209+0000","last_undegraded":"2024-04-26T22:10:11.954209+0000","last_fullsized":"2024-04-26T22:10:11.954209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":58573,"num_write_kb":703698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"40'6654","reported_seq":6712,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836315+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:10:05.836315+0000","last_peered":"2024-04-26T22:10:05.836315+0000","last_clean":"2024-04-26T22:10:05.836315+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:10:05.836315+0000","last_undegraded":"2024-04-26T22:10:05.836315+0000","last_fullsized":"2024-04-26T22:10:05.836315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":100552,"num_write_kb":1253937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"40'4775","reported_seq":4859,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979203+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:10:11.979203+0000","last_peered":"2024-04-26T22:10:11.979203+0000","last_clean":"2024-04-26T22:10:11.979203+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:10:11.979203+0000","last_undegraded":"2024-04-26T22:10:11.979203+0000","last_fullsized":"2024-04-26T22:10:11.979203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":71741,"num_write_kb":892635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"40'5861","reported_seq":5917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673275+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:10:05.673275+0000","last_peered":"2024-04-26T22:10:05.673275+0000","last_clean":"2024-04-26T22:10:05.673275+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:10:05.673275+0000","last_undegraded":"2024-04-26T22:10:05.673275+0000","last_fullsized":"2024-04-26T22:10:05.673275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88543,"num_write_kb":1086952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"40'3712","reported_seq":3779,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937208+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:10:11.937208+0000","last_peered":"2024-04-26T22:10:11.937208+0000","last_clean":"2024-04-26T22:10:11.937208+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:10:11.937208+0000","last_undegraded":"2024-04-26T22:10:11.937208+0000","last_fullsized":"2024-04-26T22:10:11.937208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":55802,"num_write_kb":677006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"40'6511","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.793662+0000","last_change":"2024-04-26T22:00:39.200941+0000","last_active":"2024-04-26T22:10:05.793662+0000","last_peered":"2024-04-26T22:10:05.793662+0000","last_clean":"2024-04-26T22:10:05.793662+0000","last_became_active":"2024-04-26T22:00:39.200592+0000","last_became_peered":"2024-04-26T22:00:39.200592+0000","last_unstale":"2024-04-26T22:10:05.793662+0000","last_undegraded":"2024-04-26T22:10:05.793662+0000","last_fullsized":"2024-04-26T22:10:05.793662+0000","mapping_epoch":25,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:59.988559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":98396,"num_write_kb":1218831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"40'4288","reported_seq":4364,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.886723+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:10:11.886723+0000","last_peered":"2024-04-26T22:10:11.886723+0000","last_clean":"2024-04-26T22:10:11.886723+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:10:11.886723+0000","last_undegraded":"2024-04-26T22:10:11.886723+0000","last_fullsized":"2024-04-26T22:10:11.886723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":64735,"num_write_kb":800999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"40'4650","reported_seq":4707,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852016+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:10:05.852016+0000","last_peered":"2024-04-26T22:10:05.852016+0000","last_clean":"2024-04-26T22:10:05.852016+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:10:05.852016+0000","last_undegraded":"2024-04-26T22:10:05.852016+0000","last_fullsized":"2024-04-26T22:10:05.852016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":70263,"num_write_kb":869775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"40'5113","reported_seq":5194,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.915535+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:10:11.915535+0000","last_peered":"2024-04-26T22:10:11.915535+0000","last_clean":"2024-04-26T22:10:11.915535+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:10:11.915535+0000","last_undegraded":"2024-04-26T22:10:11.915535+0000","last_fullsized":"2024-04-26T22:10:11.915535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":76970,"num_write_kb":914783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"40'4980","reported_seq":5038,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.818657+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:10:05.818657+0000","last_peered":"2024-04-26T22:10:05.818657+0000","last_clean":"2024-04-26T22:10:05.818657+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:10:05.818657+0000","last_undegraded":"2024-04-26T22:10:05.818657+0000","last_fullsized":"2024-04-26T22:10:05.818657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":75382,"num_write_kb":914602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"40'3556","reported_seq":3628,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.958144+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:10:11.958144+0000","last_peered":"2024-04-26T22:10:11.958144+0000","last_clean":"2024-04-26T22:10:11.958144+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:10:11.958144+0000","last_undegraded":"2024-04-26T22:10:11.958144+0000","last_fullsized":"2024-04-26T22:10:11.958144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":53572,"num_write_kb":653683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"40'5738","reported_seq":5795,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.760118+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:10:05.760118+0000","last_peered":"2024-04-26T22:10:05.760118+0000","last_clean":"2024-04-26T22:10:05.760118+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:10:05.760118+0000","last_undegraded":"2024-04-26T22:10:05.760118+0000","last_fullsized":"2024-04-26T22:10:05.760118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":86675,"num_write_kb":1103247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"40'5159","reported_seq":5238,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951137+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:10:11.951137+0000","last_peered":"2024-04-26T22:10:11.951137+0000","last_clean":"2024-04-26T22:10:11.951137+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:10:11.951137+0000","last_undegraded":"2024-04-26T22:10:11.951137+0000","last_fullsized":"2024-04-26T22:10:11.951137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":77704,"num_write_kb":942965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"40'4895","reported_seq":4967,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842652+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:10:05.842652+0000","last_peered":"2024-04-26T22:10:05.842652+0000","last_clean":"2024-04-26T22:10:05.842652+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:10:05.842652+0000","last_undegraded":"2024-04-26T22:10:05.842652+0000","last_fullsized":"2024-04-26T22:10:05.842652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":363,"num_write":74201,"num_write_kb":908910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"40'4370","reported_seq":4439,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.838189+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:10:11.838189+0000","last_peered":"2024-04-26T22:10:11.838189+0000","last_clean":"2024-04-26T22:10:11.838189+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:10:11.838189+0000","last_undegraded":"2024-04-26T22:10:11.838189+0000","last_fullsized":"2024-04-26T22:10:11.838189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4370,"log_dups_size":0,"ondisk_log_size":4370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":65733,"num_write_kb":804896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"40'6465","reported_seq":6525,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.853311+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:10:05.853311+0000","last_peered":"2024-04-26T22:10:05.853311+0000","last_clean":"2024-04-26T22:10:05.853311+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:10:05.853311+0000","last_undegraded":"2024-04-26T22:10:05.853311+0000","last_fullsized":"2024-04-26T22:10:05.853311+0000","mapping_epoch":18,"log_start":"30'1200","ondisk_log_start":"30'1200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5265,"log_dups_size":0,"ondisk_log_size":5265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":97620,"num_write_kb":1192207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"40'3334","reported_seq":3406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982941+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:10:11.982941+0000","last_peered":"2024-04-26T22:10:11.982941+0000","last_clean":"2024-04-26T22:10:11.982941+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:10:11.982941+0000","last_undegraded":"2024-04-26T22:10:11.982941+0000","last_fullsized":"2024-04-26T22:10:11.982941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":49724,"num_write_kb":588291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"40'4839","reported_seq":4896,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857008+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:10:05.857008+0000","last_peered":"2024-04-26T22:10:05.857008+0000","last_clean":"2024-04-26T22:10:05.857008+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:10:05.857008+0000","last_undegraded":"2024-04-26T22:10:05.857008+0000","last_fullsized":"2024-04-26T22:10:05.857008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":73090,"num_write_kb":888426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"40'5009","reported_seq":5079,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.919327+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:10:11.919327+0000","last_peered":"2024-04-26T22:10:11.919327+0000","last_clean":"2024-04-26T22:10:11.919327+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:10:11.919327+0000","last_undegraded":"2024-04-26T22:10:11.919327+0000","last_fullsized":"2024-04-26T22:10:11.919327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":75285,"num_write_kb":916871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"40'5333","reported_seq":5390,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.697837+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:10:05.697837+0000","last_peered":"2024-04-26T22:10:05.697837+0000","last_clean":"2024-04-26T22:10:05.697837+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:10:05.697837+0000","last_undegraded":"2024-04-26T22:10:05.697837+0000","last_fullsized":"2024-04-26T22:10:05.697837+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80596,"num_write_kb":994453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"40'4157","reported_seq":4231,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959972+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:10:11.959972+0000","last_peered":"2024-04-26T22:10:11.959972+0000","last_clean":"2024-04-26T22:10:11.959972+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:10:11.959972+0000","last_undegraded":"2024-04-26T22:10:11.959972+0000","last_fullsized":"2024-04-26T22:10:11.959972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":62491,"num_write_kb":760147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"40'5861","reported_seq":5904,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835233+0000","last_change":"2024-04-26T22:00:38.947156+0000","last_active":"2024-04-26T22:10:05.835233+0000","last_peered":"2024-04-26T22:10:05.835233+0000","last_clean":"2024-04-26T22:10:05.835233+0000","last_became_active":"2024-04-26T22:00:38.946980+0000","last_became_peered":"2024-04-26T22:00:38.946980+0000","last_unstale":"2024-04-26T22:10:05.835233+0000","last_undegraded":"2024-04-26T22:10:05.835233+0000","last_fullsized":"2024-04-26T22:10:05.835233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:02.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88698,"num_write_kb":1108811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"40'4156","reported_seq":4228,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977006+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:10:11.977006+0000","last_peered":"2024-04-26T22:10:11.977006+0000","last_clean":"2024-04-26T22:10:11.977006+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:10:11.977006+0000","last_undegraded":"2024-04-26T22:10:11.977006+0000","last_fullsized":"2024-04-26T22:10:11.977006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62195,"num_write_kb":734457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"40'5569","reported_seq":5629,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868845+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:10:05.868845+0000","last_peered":"2024-04-26T22:10:05.868845+0000","last_clean":"2024-04-26T22:10:05.868845+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:10:05.868845+0000","last_undegraded":"2024-04-26T22:10:05.868845+0000","last_fullsized":"2024-04-26T22:10:05.868845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5569,"log_dups_size":0,"ondisk_log_size":5569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":83899,"num_write_kb":1024385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"40'4460","reported_seq":4531,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955590+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:10:11.955590+0000","last_peered":"2024-04-26T22:10:11.955590+0000","last_clean":"2024-04-26T22:10:11.955590+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:10:11.955590+0000","last_undegraded":"2024-04-26T22:10:11.955590+0000","last_fullsized":"2024-04-26T22:10:11.955590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":67080,"num_write_kb":827171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"40'5284","reported_seq":5342,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.845154+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:10:05.845154+0000","last_peered":"2024-04-26T22:10:05.845154+0000","last_clean":"2024-04-26T22:10:05.845154+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:10:05.845154+0000","last_undegraded":"2024-04-26T22:10:05.845154+0000","last_fullsized":"2024-04-26T22:10:05.845154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":79924,"num_write_kb":1000899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"40'4361","reported_seq":4441,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972500+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:10:11.972500+0000","last_peered":"2024-04-26T22:10:11.972500+0000","last_clean":"2024-04-26T22:10:11.972500+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:10:11.972500+0000","last_undegraded":"2024-04-26T22:10:11.972500+0000","last_fullsized":"2024-04-26T22:10:11.972500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65464,"num_write_kb":794873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"40'5458","reported_seq":5516,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.860815+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:10:05.860815+0000","last_peered":"2024-04-26T22:10:05.860815+0000","last_clean":"2024-04-26T22:10:05.860815+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:10:05.860815+0000","last_undegraded":"2024-04-26T22:10:05.860815+0000","last_fullsized":"2024-04-26T22:10:05.860815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":82585,"num_write_kb":1000929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"40'3624","reported_seq":3690,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966367+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:10:11.966367+0000","last_peered":"2024-04-26T22:10:11.966367+0000","last_clean":"2024-04-26T22:10:11.966367+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:10:11.966367+0000","last_undegraded":"2024-04-26T22:10:11.966367+0000","last_fullsized":"2024-04-26T22:10:11.966367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":54495,"num_write_kb":662506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"40'4431","reported_seq":4495,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.870020+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:10:05.870020+0000","last_peered":"2024-04-26T22:10:05.870020+0000","last_clean":"2024-04-26T22:10:05.870020+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:10:05.870020+0000","last_undegraded":"2024-04-26T22:10:05.870020+0000","last_fullsized":"2024-04-26T22:10:05.870020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":66582,"num_write_kb":814643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"40'4089","reported_seq":4165,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.832376+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:10:11.832376+0000","last_peered":"2024-04-26T22:10:11.832376+0000","last_clean":"2024-04-26T22:10:11.832376+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:10:11.832376+0000","last_undegraded":"2024-04-26T22:10:11.832376+0000","last_fullsized":"2024-04-26T22:10:11.832376+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4089,"log_dups_size":0,"ondisk_log_size":4089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61370,"num_write_kb":735117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"40'6428","reported_seq":6487,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822613+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:10:05.822613+0000","last_peered":"2024-04-26T22:10:05.822613+0000","last_clean":"2024-04-26T22:10:05.822613+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:10:05.822613+0000","last_undegraded":"2024-04-26T22:10:05.822613+0000","last_fullsized":"2024-04-26T22:10:05.822613+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":97094,"num_write_kb":1219735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"40'4506","reported_seq":4588,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.945454+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:10:11.945454+0000","last_peered":"2024-04-26T22:10:11.945454+0000","last_clean":"2024-04-26T22:10:11.945454+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:10:11.945454+0000","last_undegraded":"2024-04-26T22:10:11.945454+0000","last_fullsized":"2024-04-26T22:10:11.945454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":67596,"num_write_kb":862862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"40'4747","reported_seq":4805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803411+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:10:05.803411+0000","last_peered":"2024-04-26T22:10:05.803411+0000","last_clean":"2024-04-26T22:10:05.803411+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:10:05.803411+0000","last_undegraded":"2024-04-26T22:10:05.803411+0000","last_fullsized":"2024-04-26T22:10:05.803411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71795,"num_write_kb":917779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"40'4901","reported_seq":4976,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.894392+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:10:11.894392+0000","last_peered":"2024-04-26T22:10:11.894392+0000","last_clean":"2024-04-26T22:10:11.894392+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:10:11.894392+0000","last_undegraded":"2024-04-26T22:10:11.894392+0000","last_fullsized":"2024-04-26T22:10:11.894392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4901,"log_dups_size":0,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":73795,"num_write_kb":926135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"40'5581","reported_seq":5639,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.843203+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:10:05.843203+0000","last_peered":"2024-04-26T22:10:05.843203+0000","last_clean":"2024-04-26T22:10:05.843203+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:10:05.843203+0000","last_undegraded":"2024-04-26T22:10:05.843203+0000","last_fullsized":"2024-04-26T22:10:05.843203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5581,"log_dups_size":0,"ondisk_log_size":5581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":84297,"num_write_kb":1026829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"40'3600","reported_seq":3671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.933325+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:10:11.933325+0000","last_peered":"2024-04-26T22:10:11.933325+0000","last_clean":"2024-04-26T22:10:11.933325+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:10:11.933325+0000","last_undegraded":"2024-04-26T22:10:11.933325+0000","last_fullsized":"2024-04-26T22:10:11.933325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":54091,"num_write_kb":649095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"40'5612","reported_seq":5671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.789467+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:10:05.789467+0000","last_peered":"2024-04-26T22:10:05.789467+0000","last_clean":"2024-04-26T22:10:05.789467+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:10:05.789467+0000","last_undegraded":"2024-04-26T22:10:05.789467+0000","last_fullsized":"2024-04-26T22:10:05.789467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":84765,"num_write_kb":1047612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"40'3974","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.882273+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:10:11.882273+0000","last_peered":"2024-04-26T22:10:11.882273+0000","last_clean":"2024-04-26T22:10:11.882273+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:10:11.882273+0000","last_undegraded":"2024-04-26T22:10:11.882273+0000","last_fullsized":"2024-04-26T22:10:11.882273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3974,"log_dups_size":0,"ondisk_log_size":3974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":59852,"num_write_kb":718261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"40'4819","reported_seq":4881,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.858034+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:10:05.858034+0000","last_peered":"2024-04-26T22:10:05.858034+0000","last_clean":"2024-04-26T22:10:05.858034+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:10:05.858034+0000","last_undegraded":"2024-04-26T22:10:05.858034+0000","last_fullsized":"2024-04-26T22:10:05.858034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":72748,"num_write_kb":932622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"40'4140","reported_seq":4220,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.931602+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:10:11.931602+0000","last_peered":"2024-04-26T22:10:11.931602+0000","last_clean":"2024-04-26T22:10:11.931602+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:10:11.931602+0000","last_undegraded":"2024-04-26T22:10:11.931602+0000","last_fullsized":"2024-04-26T22:10:11.931602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":62174,"num_write_kb":775596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"40'4503","reported_seq":4562,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.797708+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:10:05.797708+0000","last_peered":"2024-04-26T22:10:05.797708+0000","last_clean":"2024-04-26T22:10:05.797708+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:10:05.797708+0000","last_undegraded":"2024-04-26T22:10:05.797708+0000","last_fullsized":"2024-04-26T22:10:05.797708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":68209,"num_write_kb":848035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"40'3768","reported_seq":3848,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.978793+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:10:11.978793+0000","last_peered":"2024-04-26T22:10:11.978793+0000","last_clean":"2024-04-26T22:10:11.978793+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:10:11.978793+0000","last_undegraded":"2024-04-26T22:10:11.978793+0000","last_fullsized":"2024-04-26T22:10:11.978793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":56583,"num_write_kb":708038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"40'5303","reported_seq":5359,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.717590+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:10:05.717590+0000","last_peered":"2024-04-26T22:10:05.717590+0000","last_clean":"2024-04-26T22:10:05.717590+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:10:05.717590+0000","last_undegraded":"2024-04-26T22:10:05.717590+0000","last_fullsized":"2024-04-26T22:10:05.717590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80229,"num_write_kb":963753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"40'3667","reported_seq":3745,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.927871+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:10:11.927871+0000","last_peered":"2024-04-26T22:10:11.927871+0000","last_clean":"2024-04-26T22:10:11.927871+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:10:11.927871+0000","last_undegraded":"2024-04-26T22:10:11.927871+0000","last_fullsized":"2024-04-26T22:10:11.927871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3667,"log_dups_size":0,"ondisk_log_size":3667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":55047,"num_write_kb":687611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"40'6069","reported_seq":6128,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810736+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:10:05.810736+0000","last_peered":"2024-04-26T22:10:05.810736+0000","last_clean":"2024-04-26T22:10:05.810736+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:10:05.810736+0000","last_undegraded":"2024-04-26T22:10:05.810736+0000","last_fullsized":"2024-04-26T22:10:05.810736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":91606,"num_write_kb":1169870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"40'4173","reported_seq":4252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953675+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:10:11.953675+0000","last_peered":"2024-04-26T22:10:11.953675+0000","last_clean":"2024-04-26T22:10:11.953675+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:10:11.953675+0000","last_undegraded":"2024-04-26T22:10:11.953675+0000","last_fullsized":"2024-04-26T22:10:11.953675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":62684,"num_write_kb":755589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"40'6512","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.763471+0000","last_change":"2024-04-26T22:00:39.201110+0000","last_active":"2024-04-26T22:10:05.763471+0000","last_peered":"2024-04-26T22:10:05.763471+0000","last_clean":"2024-04-26T22:10:05.763471+0000","last_became_active":"2024-04-26T22:00:39.200691+0000","last_became_peered":"2024-04-26T22:00:39.200691+0000","last_unstale":"2024-04-26T22:10:05.763471+0000","last_undegraded":"2024-04-26T22:10:05.763471+0000","last_fullsized":"2024-04-26T22:10:05.763471+0000","mapping_epoch":25,"log_start":"29'700","ondisk_log_start":"29'700","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:06.181131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":98505,"num_write_kb":1266779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"40'5161","reported_seq":5247,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982946+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:10:11.982946+0000","last_peered":"2024-04-26T22:10:11.982946+0000","last_clean":"2024-04-26T22:10:11.982946+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:10:11.982946+0000","last_undegraded":"2024-04-26T22:10:11.982946+0000","last_fullsized":"2024-04-26T22:10:11.982946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":77820,"num_write_kb":957388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"40'5846","reported_seq":5903,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.706015+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:10:05.706015+0000","last_peered":"2024-04-26T22:10:05.706015+0000","last_clean":"2024-04-26T22:10:05.706015+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:10:05.706015+0000","last_undegraded":"2024-04-26T22:10:05.706015+0000","last_fullsized":"2024-04-26T22:10:05.706015+0000","mapping_epoch":18,"log_start":"29'600","ondisk_log_start":"29'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88593,"num_write_kb":1072922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"40'3484","reported_seq":3552,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972160+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:10:11.972160+0000","last_peered":"2024-04-26T22:10:11.972160+0000","last_clean":"2024-04-26T22:10:11.972160+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:10:11.972160+0000","last_undegraded":"2024-04-26T22:10:11.972160+0000","last_fullsized":"2024-04-26T22:10:11.972160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3484,"log_dups_size":0,"ondisk_log_size":3484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":52218,"num_write_kb":635931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"40'5213","reported_seq":5273,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874711+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:10:05.874711+0000","last_peered":"2024-04-26T22:10:05.874711+0000","last_clean":"2024-04-26T22:10:05.874711+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:10:05.874711+0000","last_undegraded":"2024-04-26T22:10:05.874711+0000","last_fullsized":"2024-04-26T22:10:05.874711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":78898,"num_write_kb":987567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"40'3779","reported_seq":3858,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.698994+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:10:11.698994+0000","last_peered":"2024-04-26T22:10:11.698994+0000","last_clean":"2024-04-26T22:10:11.698994+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:10:11.698994+0000","last_undegraded":"2024-04-26T22:10:11.698994+0000","last_fullsized":"2024-04-26T22:10:11.698994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56904,"num_write_kb":701117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"40'4635","reported_seq":4692,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.724357+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:10:05.724357+0000","last_peered":"2024-04-26T22:10:05.724357+0000","last_clean":"2024-04-26T22:10:05.724357+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:10:05.724357+0000","last_undegraded":"2024-04-26T22:10:05.724357+0000","last_fullsized":"2024-04-26T22:10:05.724357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4635,"log_dups_size":0,"ondisk_log_size":4635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":70184,"num_write_kb":869081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"40'4125","reported_seq":4197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954407+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:10:11.954407+0000","last_peered":"2024-04-26T22:10:11.954407+0000","last_clean":"2024-04-26T22:10:11.954407+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:10:11.954407+0000","last_undegraded":"2024-04-26T22:10:11.954407+0000","last_fullsized":"2024-04-26T22:10:11.954407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62115,"num_write_kb":743778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"40'6598","reported_seq":6661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.878241+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:10:05.878241+0000","last_peered":"2024-04-26T22:10:05.878241+0000","last_clean":"2024-04-26T22:10:05.878241+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:10:05.878241+0000","last_undegraded":"2024-04-26T22:10:05.878241+0000","last_fullsized":"2024-04-26T22:10:05.878241+0000","mapping_epoch":18,"log_start":"29'800","ondisk_log_start":"29'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":99507,"num_write_kb":1174132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"40'3584","reported_seq":3653,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979143+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:10:11.979143+0000","last_peered":"2024-04-26T22:10:11.979143+0000","last_clean":"2024-04-26T22:10:11.979143+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:10:11.979143+0000","last_undegraded":"2024-04-26T22:10:11.979143+0000","last_fullsized":"2024-04-26T22:10:11.979143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3584,"log_dups_size":0,"ondisk_log_size":3584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":54000,"num_write_kb":647349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"40'4854","reported_seq":4911,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.499989+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:10:05.499989+0000","last_peered":"2024-04-26T22:10:05.499989+0000","last_clean":"2024-04-26T22:10:05.499989+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:10:05.499989+0000","last_undegraded":"2024-04-26T22:10:05.499989+0000","last_fullsized":"2024-04-26T22:10:05.499989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73549,"num_write_kb":940613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"40'4744","reported_seq":4833,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.993205+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:10:11.993205+0000","last_peered":"2024-04-26T22:10:11.993205+0000","last_clean":"2024-04-26T22:10:11.993205+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:10:11.993205+0000","last_undegraded":"2024-04-26T22:10:11.993205+0000","last_fullsized":"2024-04-26T22:10:11.993205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":71239,"num_write_kb":843376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"40'5453","reported_seq":5511,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835411+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:10:05.835411+0000","last_peered":"2024-04-26T22:10:05.835411+0000","last_clean":"2024-04-26T22:10:05.835411+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:10:05.835411+0000","last_undegraded":"2024-04-26T22:10:05.835411+0000","last_fullsized":"2024-04-26T22:10:05.835411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5453,"log_dups_size":0,"ondisk_log_size":5453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":82428,"num_write_kb":1036078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"40'4470","reported_seq":4548,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.844612+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:10:11.844612+0000","last_peered":"2024-04-26T22:10:11.844612+0000","last_clean":"2024-04-26T22:10:11.844612+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:10:11.844612+0000","last_undegraded":"2024-04-26T22:10:11.844612+0000","last_fullsized":"2024-04-26T22:10:11.844612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":67360,"num_write_kb":793318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"40'6816","reported_seq":6875,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.873639+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:10:05.873639+0000","last_peered":"2024-04-26T22:10:05.873639+0000","last_clean":"2024-04-26T22:10:05.873639+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:10:05.873639+0000","last_undegraded":"2024-04-26T22:10:05.873639+0000","last_fullsized":"2024-04-26T22:10:05.873639+0000","mapping_epoch":18,"log_start":"29'1100","ondisk_log_start":"29'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":103074,"num_write_kb":1294412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"40'3980","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986399+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:10:11.986399+0000","last_peered":"2024-04-26T22:10:11.986399+0000","last_clean":"2024-04-26T22:10:11.986399+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:10:11.986399+0000","last_undegraded":"2024-04-26T22:10:11.986399+0000","last_fullsized":"2024-04-26T22:10:11.986399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":59871,"num_write_kb":728175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"40'4761","reported_seq":4821,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.861444+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:10:05.861444+0000","last_peered":"2024-04-26T22:10:05.861444+0000","last_clean":"2024-04-26T22:10:05.861444+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:10:05.861444+0000","last_undegraded":"2024-04-26T22:10:05.861444+0000","last_fullsized":"2024-04-26T22:10:05.861444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":72121,"num_write_kb":895283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1800,"num_read_kb":0,"num_write":3978586,"num_write_kb":48381196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264479,"ondisk_log_size":264479,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":109330781,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":691,"num_read_kb":381190,"num_write":3542,"num_write_kb":109337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218726400,"data_stored":218661562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3491,"ondisk_log_size":3491,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":363,"num_write":5266740,"num_write_kb":65270508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338431,"ondisk_log_size":338431,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":143059102,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":553,"num_read_kb":447893,"num_write":4271,"num_write_kb":143037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286179328,"data_stored":286117605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4237,"ondisk_log_size":4237,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525528,"kb_used_data":21200,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":93846312,"statfs":{"total":96636764160,"available":96098623488,"internally_reserved":0,"allocated":21708800,"data_stored":21468517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516422085},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848624,"kb_used_data":66920,"kb_used_omap":1,"kb_used_meta":781694,"kb_avail":93523216,"statfs":{"total":96636764160,"available":95767773184,"internally_reserved":0,"allocated":68526080,"data_stored":68281767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":800455114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574352,"kb_used_data":74184,"kb_used_omap":1,"kb_used_meta":500158,"kb_avail":93797488,"statfs":{"total":96636764160,"available":96048627712,"internally_reserved":0,"allocated":75964416,"data_stored":75718280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":512162246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789860,"kb_used_data":45596,"kb_used_omap":1,"kb_used_meta":744254,"kb_avail":93581980,"statfs":{"total":96636764160,"available":95827947520,"internally_reserved":0,"allocated":46690304,"data_stored":46445504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":762116549},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882668,"kb_used_data":82468,"kb_used_omap":1,"kb_used_meta":800190,"kb_avail":93489172,"statfs":{"total":96636764160,"available":95732912128,"internally_reserved":0,"allocated":84447232,"data_stored":84193910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":819395014},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900648,"kb_used_data":78560,"kb_used_omap":1,"kb_used_meta":822078,"kb_avail":93471192,"statfs":{"total":96636764160,"available":95714500608,"internally_reserved":0,"allocated":80445440,"data_stored":80190170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":841808327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767416,"kb_used_data":86832,"kb_used_omap":1,"kb_used_meta":680574,"kb_avail":93604424,"statfs":{"total":96636764160,"available":95850930176,"internally_reserved":0,"allocated":88915968,"data_stored":88663274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":696908229},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740368,"kb_used_data":41992,"kb_used_omap":1,"kb_used_meta":698366,"kb_avail":93631472,"statfs":{"total":96636764160,"available":95878627328,"internally_reserved":0,"allocated":42999808,"data_stored":42753566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715127237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29372416,"data_stored":29360766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":38112011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12673024,"data_stored":12671093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33652736,"data_stored":33643285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29364224,"data_stored":29360218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":46145536,"data_stored":46138016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50700288,"data_stored":50694746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8585216,"data_stored":8578005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46342144,"data_stored":46327365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37847040,"data_stored":37836920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16875520,"data_stored":16865968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:10:17.440 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:10:17.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.567+0000 7f493332c640 1 Processor -- start 2024-04-26T22:10:17.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.567+0000 7f493332c640 1 -- start start 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f493332c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f492c1574b0 0x7f492c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f493332c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f492c0c3810 con 0x7f492c0ab3f0 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f493332c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f492c0c3ab0 con 0x7f492c1570d0 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f493332c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f492c157e20 con 0x7f492c1574b0 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f49318c9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f492c1574b0 0x7f492c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f49308c7640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f492c0ab3f0 0x7f492c0c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47660/0 (socket says 172.21.15.135:47660) 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f49308c7640 1 -- 172.21.15.135:0/1156332248 learned_addr learned my addr 172.21.15.135:0/1156332248 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f49318c9640 1 -- 172.21.15.135:0/1156332248 >> v1:172.21.15.139:6789/0 conn(0x7f492c1570d0 legacy=0x7f492c150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f49318c9640 1 -- 172.21.15.135:0/1156332248 >> v1:172.21.15.135:6789/0 conn(0x7f492c0ab3f0 legacy=0x7f492c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.568+0000 7f49318c9640 1 -- 172.21.15.135:0/1156332248 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c0c4580 con 0x7f492c1574b0 2024-04-26T22:10:17.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f49318c9640 1 --2- 172.21.15.135:0/1156332248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c1574b0 0x7f492c1578e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4924009d90 tx=0x7f492402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e2f494b6c6c06d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f4923fff640 1 -- 172.21.15.135:0/1156332248 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924031600 con 0x7f492c1574b0 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f4923fff640 1 -- 172.21.15.135:0/1156332248 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49240317a0 con 0x7f492c1574b0 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f4923fff640 1 -- 172.21.15.135:0/1156332248 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924031b70 con 0x7f492c1574b0 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 -- 172.21.15.135:0/1156332248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c1574b0 msgr2=0x7f492c1578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 --2- 172.21.15.135:0/1156332248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c1574b0 0x7f492c1578e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4924009d90 tx=0x7f492402a5d0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 -- 172.21.15.135:0/1156332248 shutdown_connections 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 --2- 172.21.15.135:0/1156332248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c1574b0 0x7f492c1578e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 -- 172.21.15.135:0/1156332248 >> 172.21.15.135:0/1156332248 conn(0x7f492c06c310 msgr2=0x7f492c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 -- 172.21.15.135:0/1156332248 shutdown_connections 2024-04-26T22:10:17.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.569+0000 7f493332c640 1 -- 172.21.15.135:0/1156332248 wait complete. 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.570+0000 7f493332c640 1 Processor -- start 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.570+0000 7f493332c640 1 -- start start 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f493332c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 0x7f492c072a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f493332c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c1570d0 0x7f492c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f493332c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c1574b0 0x7f492c1edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f493332c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f492c151160 con 0x7f492c0ab3f0 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f493332c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f492c0c4330 con 0x7f492c1570d0 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f493332c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f492c1512e0 con 0x7f492c1574b0 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 0x7f492c072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 0x7f492c072a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51542/0 (socket says 172.21.15.135:51542) 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 -- 172.21.15.135:0/651410018 learned_addr learned my addr 172.21.15.135:0/651410018 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:17.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49308c7640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c1570d0 0x7f492c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49318c9640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c1574b0 0x7f492c1edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 -- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c1574b0 msgr2=0x7f492c1edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c1574b0 0x7f492c1edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 -- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c1570d0 msgr2=0x7f492c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c1570d0 0x7f492c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49310c8640 1 -- 172.21.15.135:0/651410018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c1569c0 con 0x7f492c0ab3f0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.571+0000 7f49318c9640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c1574b0 0x7f492c1edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.572+0000 7f49310c8640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 0x7f492c072a60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f492800c430 tx=0x7f492800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.572+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49280052d0 con 0x7f492c0ab3f0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.572+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4928005490 con 0x7f492c0ab3f0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.572+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c148cb0 con 0x7f492c0ab3f0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.572+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928007260 con 0x7f492c0ab3f0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.572+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c156590 con 0x7f492c0ab3f0 2024-04-26T22:10:17.574 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.573+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c0c2900 con 0x7f492c0ab3f0 2024-04-26T22:10:17.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.573+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4928007400 con 0x7f492c0ab3f0 2024-04-26T22:10:17.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.574+0000 7f4921ffb640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f491006c8f0 0x7f491006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.574+0000 7f49308c7640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f491006c8f0 0x7f491006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.574+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f492808b890 con 0x7f492c0ab3f0 2024-04-26T22:10:17.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.574+0000 7f49308c7640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f491006c8f0 0x7f491006edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f492c14d930 tx=0x7f491c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.578+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f492c1569c0 con 0x7f492c0ab3f0 2024-04-26T22:10:17.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.747+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f492c148a60 con 0x7f491006c8f0 2024-04-26T22:10:17.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.758+0000 7f4921ffb640 1 -- 172.21.15.135:0/651410018 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494622 (secure 0 0 0) 0x7f492c148a60 con 0x7f491006c8f0 2024-04-26T22:10:17.767 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:17.772 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.762+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f491006c8f0 msgr2=0x7f491006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.762+0000 7f493332c640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f491006c8f0 0x7f491006edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f492c14d930 tx=0x7f491c009040 comp rx=0 tx=0).stop 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.762+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 msgr2=0x7f492c072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.762+0000 7f493332c640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 0x7f492c072a60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f492800c430 tx=0x7f492800c8f0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 shutdown_connections 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f491006c8f0 0x7f491006edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c1574b0 0x7f492c1edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c1570d0 0x7f492c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 --2- 172.21.15.135:0/651410018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f492c0ab3f0 0x7f492c072a60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 >> 172.21.15.135:0/651410018 conn(0x7f492c06c310 msgr2=0x7f492c14b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:17.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 shutdown_connections 2024-04-26T22:10:17.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.763+0000 7f493332c640 1 -- 172.21.15.135:0/651410018 wait complete. 2024-04-26T22:10:17.775 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-04-26T22:10:17.308801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252980251,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":3350,"num_read_kb":829546,"num_write":9253298,"num_write_kb":113906494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610755,"ondisk_log_size":610755,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6029464,"kb_used_data":497752,"kb_used_omap":12,"kb_used_meta":5531635,"kb_avail":748945256,"statfs":{"total":773094113280,"available":766919942144,"internally_reserved":0,"allocated":509698048,"data_stored":507714988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":5664394801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20190116710,"num_objects":-5212,"num_object_clones":0,"num_object_copies":-10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5212,"num_whiteouts":0,"num_read":581,"num_read_kb":84526,"num_write":5421,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003200"},"pg_stats":[{"pgid":"5.18","version":"40'4026","reported_seq":4100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.983679+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:10:11.983679+0000","last_peered":"2024-04-26T22:10:11.983679+0000","last_clean":"2024-04-26T22:10:11.983679+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:10:11.983679+0000","last_undegraded":"2024-04-26T22:10:11.983679+0000","last_fullsized":"2024-04-26T22:10:11.983679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":60598,"num_write_kb":740487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"39'27","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.941276+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:10:00.941276+0000","last_peered":"2024-04-26T22:10:00.941276+0000","last_clean":"2024-04-26T22:10:00.941276+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:10:00.941276+0000","last_undegraded":"2024-04-26T22:10:00.941276+0000","last_fullsized":"2024-04-26T22:10:00.941276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":16408,"num_write":20,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'5701","reported_seq":5758,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.764348+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:10:05.764348+0000","last_peered":"2024-04-26T22:10:05.764348+0000","last_clean":"2024-04-26T22:10:05.764348+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:10:05.764348+0000","last_undegraded":"2024-04-26T22:10:05.764348+0000","last_fullsized":"2024-04-26T22:10:05.764348+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86528,"num_write_kb":1106988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"40'19","reported_seq":69,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.727806+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:10:10.727806+0000","last_peered":"2024-04-26T22:10:10.727806+0000","last_clean":"2024-04-26T22:10:10.727806+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:10:10.727806+0000","last_undegraded":"2024-04-26T22:10:10.727806+0000","last_fullsized":"2024-04-26T22:10:10.727806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189327,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"40'4869","reported_seq":4957,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988095+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:10:11.988095+0000","last_peered":"2024-04-26T22:10:11.988095+0000","last_clean":"2024-04-26T22:10:11.988095+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:10:11.988095+0000","last_undegraded":"2024-04-26T22:10:11.988095+0000","last_fullsized":"2024-04-26T22:10:11.988095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":73297,"num_write_kb":899566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"40'20","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857789+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:10:05.857789+0000","last_peered":"2024-04-26T22:10:05.857789+0000","last_clean":"2024-04-26T22:10:05.857789+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:10:05.857789+0000","last_undegraded":"2024-04-26T22:10:05.857789+0000","last_fullsized":"2024-04-26T22:10:05.857789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"40'6511","reported_seq":6573,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848823+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:10:05.848823+0000","last_peered":"2024-04-26T22:10:05.848823+0000","last_clean":"2024-04-26T22:10:05.848823+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:10:05.848823+0000","last_undegraded":"2024-04-26T22:10:05.848823+0000","last_fullsized":"2024-04-26T22:10:05.848823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6511,"log_dups_size":0,"ondisk_log_size":6511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":98412,"num_write_kb":1227645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"40'132","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.904919+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:10:11.904919+0000","last_peered":"2024-04-26T22:10:11.904919+0000","last_clean":"2024-04-26T22:10:11.904919+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:10:11.904919+0000","last_undegraded":"2024-04-26T22:10:11.904919+0000","last_fullsized":"2024-04-26T22:10:11.904919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":127,"num_write_kb":8276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"40'3508","reported_seq":3574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.949165+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:10:11.949165+0000","last_peered":"2024-04-26T22:10:11.949165+0000","last_clean":"2024-04-26T22:10:11.949165+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:10:11.949165+0000","last_undegraded":"2024-04-26T22:10:11.949165+0000","last_fullsized":"2024-04-26T22:10:11.949165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3508,"log_dups_size":0,"ondisk_log_size":3508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":52619,"num_write_kb":632470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"40'116","reported_seq":197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.110609+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:10:10.110609+0000","last_peered":"2024-04-26T22:10:10.110609+0000","last_clean":"2024-04-26T22:10:10.110609+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:10:10.110609+0000","last_undegraded":"2024-04-26T22:10:10.110609+0000","last_fullsized":"2024-04-26T22:10:10.110609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":45092,"num_write":102,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"40'3918","reported_seq":3975,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.754455+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:10:05.754455+0000","last_peered":"2024-04-26T22:10:05.754455+0000","last_clean":"2024-04-26T22:10:05.754455+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:10:05.754455+0000","last_undegraded":"2024-04-26T22:10:05.754455+0000","last_fullsized":"2024-04-26T22:10:05.754455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":59308,"num_write_kb":714963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"40'1268","reported_seq":1325,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.959833+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:10:00.959833+0000","last_peered":"2024-04-26T22:10:00.959833+0000","last_clean":"2024-04-26T22:10:00.959833+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:10:00.959833+0000","last_undegraded":"2024-04-26T22:10:00.959833+0000","last_fullsized":"2024-04-26T22:10:00.959833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1268,"log_dups_size":0,"ondisk_log_size":1268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":1265,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'3315","reported_seq":3384,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.932922+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:10:11.932922+0000","last_peered":"2024-04-26T22:10:11.932922+0000","last_clean":"2024-04-26T22:10:11.932922+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:10:11.932922+0000","last_undegraded":"2024-04-26T22:10:11.932922+0000","last_fullsized":"2024-04-26T22:10:11.932922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3315,"log_dups_size":0,"ondisk_log_size":3315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":49749,"num_write_kb":591259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"40'20","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.954736+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:10:01.954736+0000","last_peered":"2024-04-26T22:10:01.954736+0000","last_clean":"2024-04-26T22:10:01.954736+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:10:01.954736+0000","last_undegraded":"2024-04-26T22:10:01.954736+0000","last_fullsized":"2024-04-26T22:10:01.954736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":18,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"40'6620","reported_seq":6664,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.806952+0000","last_change":"2024-04-26T22:00:39.151257+0000","last_active":"2024-04-26T22:10:05.806952+0000","last_peered":"2024-04-26T22:10:05.806952+0000","last_clean":"2024-04-26T22:10:05.806952+0000","last_became_active":"2024-04-26T22:00:39.151025+0000","last_became_peered":"2024-04-26T22:00:39.151025+0000","last_unstale":"2024-04-26T22:10:05.806952+0000","last_undegraded":"2024-04-26T22:10:05.806952+0000","last_fullsized":"2024-04-26T22:10:05.806952+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:06.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":100009,"num_write_kb":1224131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"40'41","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955354+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:10:11.955354+0000","last_peered":"2024-04-26T22:10:11.955354+0000","last_clean":"2024-04-26T22:10:11.955354+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:10:11.955354+0000","last_undegraded":"2024-04-26T22:10:11.955354+0000","last_fullsized":"2024-04-26T22:10:11.955354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":30,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"40'4367","reported_seq":4447,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.965935+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:10:11.965935+0000","last_peered":"2024-04-26T22:10:11.965935+0000","last_clean":"2024-04-26T22:10:11.965935+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:10:11.965935+0000","last_undegraded":"2024-04-26T22:10:11.965935+0000","last_fullsized":"2024-04-26T22:10:11.965935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65356,"num_write_kb":790948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"40'4740","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.661582+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:10:05.661582+0000","last_peered":"2024-04-26T22:10:05.661582+0000","last_clean":"2024-04-26T22:10:05.661582+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:10:05.661582+0000","last_undegraded":"2024-04-26T22:10:05.661582+0000","last_fullsized":"2024-04-26T22:10:05.661582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":71515,"num_write_kb":926848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"40'206","reported_seq":280,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.865453+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:10:05.865453+0000","last_peered":"2024-04-26T22:10:05.865453+0000","last_clean":"2024-04-26T22:10:05.865453+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:10:05.865453+0000","last_undegraded":"2024-04-26T22:10:05.865453+0000","last_fullsized":"2024-04-26T22:10:05.865453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":29169,"num_write":192,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"40'39","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.495965+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:10:01.495965+0000","last_peered":"2024-04-26T22:10:01.495965+0000","last_clean":"2024-04-26T22:10:01.495965+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:10:01.495965+0000","last_undegraded":"2024-04-26T22:10:01.495965+0000","last_fullsized":"2024-04-26T22:10:01.495965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":13991,"num_write":36,"num_write_kb":4124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'3984","reported_seq":4061,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986082+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:10:11.986082+0000","last_peered":"2024-04-26T22:10:11.986082+0000","last_clean":"2024-04-26T22:10:11.986082+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:10:11.986082+0000","last_undegraded":"2024-04-26T22:10:11.986082+0000","last_fullsized":"2024-04-26T22:10:11.986082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59797,"num_write_kb":746756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"40'4602","reported_seq":4661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852754+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:10:05.852754+0000","last_peered":"2024-04-26T22:10:05.852754+0000","last_clean":"2024-04-26T22:10:05.852754+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:10:05.852754+0000","last_undegraded":"2024-04-26T22:10:05.852754+0000","last_fullsized":"2024-04-26T22:10:05.852754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69468,"num_write_kb":843448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"40'24","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.828651+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:10:05.828651+0000","last_peered":"2024-04-26T22:10:05.828651+0000","last_clean":"2024-04-26T22:10:05.828651+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:10:05.828651+0000","last_undegraded":"2024-04-26T22:10:05.828651+0000","last_fullsized":"2024-04-26T22:10:05.828651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":20,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937503+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:10:11.937503+0000","last_peered":"2024-04-26T22:10:11.937503+0000","last_clean":"2024-04-26T22:10:11.937503+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:10:11.937503+0000","last_undegraded":"2024-04-26T22:10:11.937503+0000","last_fullsized":"2024-04-26T22:10:11.937503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"40'3976","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.916627+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:10:11.916627+0000","last_peered":"2024-04-26T22:10:11.916627+0000","last_clean":"2024-04-26T22:10:11.916627+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:10:11.916627+0000","last_undegraded":"2024-04-26T22:10:11.916627+0000","last_fullsized":"2024-04-26T22:10:11.916627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3976,"log_dups_size":0,"ondisk_log_size":3976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59917,"num_write_kb":706723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"40'5423","reported_seq":5480,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.761465+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:10:05.761465+0000","last_peered":"2024-04-26T22:10:05.761465+0000","last_clean":"2024-04-26T22:10:05.761465+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:10:05.761465+0000","last_undegraded":"2024-04-26T22:10:05.761465+0000","last_fullsized":"2024-04-26T22:10:05.761465+0000","mapping_epoch":18,"log_start":"28'200","ondisk_log_start":"28'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":81931,"num_write_kb":1035269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'44","reported_seq":104,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.415234+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:10:14.415234+0000","last_peered":"2024-04-26T22:10:14.415234+0000","last_clean":"2024-04-26T22:10:14.415234+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:10:14.415234+0000","last_undegraded":"2024-04-26T22:10:14.415234+0000","last_fullsized":"2024-04-26T22:10:14.415234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363115,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":324,"num_write":38,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"40'85","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.973183+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:10:11.973183+0000","last_peered":"2024-04-26T22:10:11.973183+0000","last_clean":"2024-04-26T22:10:11.973183+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:10:11.973183+0000","last_undegraded":"2024-04-26T22:10:11.973183+0000","last_fullsized":"2024-04-26T22:10:11.973183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":6841,"num_write":81,"num_write_kb":4168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"40'4751","reported_seq":4829,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937603+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:10:11.937603+0000","last_peered":"2024-04-26T22:10:11.937603+0000","last_clean":"2024-04-26T22:10:11.937603+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:10:11.937603+0000","last_undegraded":"2024-04-26T22:10:11.937603+0000","last_fullsized":"2024-04-26T22:10:11.937603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":71494,"num_write_kb":858502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"40'5174","reported_seq":5233,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.821492+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:10:05.821492+0000","last_peered":"2024-04-26T22:10:05.821492+0000","last_clean":"2024-04-26T22:10:05.821492+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:10:05.821492+0000","last_undegraded":"2024-04-26T22:10:05.821492+0000","last_fullsized":"2024-04-26T22:10:05.821492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":78246,"num_write_kb":941647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"40'8","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823374+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:10:05.823374+0000","last_peered":"2024-04-26T22:10:05.823374+0000","last_clean":"2024-04-26T22:10:05.823374+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:10:05.823374+0000","last_undegraded":"2024-04-26T22:10:05.823374+0000","last_fullsized":"2024-04-26T22:10:05.823374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":4,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"40'37","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962744+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:10:11.962744+0000","last_peered":"2024-04-26T22:10:11.962744+0000","last_clean":"2024-04-26T22:10:11.962744+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:10:11.962744+0000","last_undegraded":"2024-04-26T22:10:11.962744+0000","last_fullsized":"2024-04-26T22:10:11.962744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":31,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"40'3913","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962293+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:10:11.962293+0000","last_peered":"2024-04-26T22:10:11.962293+0000","last_clean":"2024-04-26T22:10:11.962293+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:10:11.962293+0000","last_undegraded":"2024-04-26T22:10:11.962293+0000","last_fullsized":"2024-04-26T22:10:11.962293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":58791,"num_write_kb":740377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'4599","reported_seq":4657,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.774768+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:10:05.774768+0000","last_peered":"2024-04-26T22:10:05.774768+0000","last_clean":"2024-04-26T22:10:05.774768+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:10:05.774768+0000","last_undegraded":"2024-04-26T22:10:05.774768+0000","last_fullsized":"2024-04-26T22:10:05.774768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69140,"num_write_kb":847801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"40'8","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836838+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:10:05.836838+0000","last_peered":"2024-04-26T22:10:05.836838+0000","last_clean":"2024-04-26T22:10:05.836838+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:10:05.836838+0000","last_undegraded":"2024-04-26T22:10:05.836838+0000","last_fullsized":"2024-04-26T22:10:05.836838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"40'23","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.979062+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:10:14.979062+0000","last_peered":"2024-04-26T22:10:14.979062+0000","last_clean":"2024-04-26T22:10:14.979062+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:10:14.979062+0000","last_undegraded":"2024-04-26T22:10:14.979062+0000","last_fullsized":"2024-04-26T22:10:14.979062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":98,"num_write":20,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"40'3929","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.938066+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:10:11.938066+0000","last_peered":"2024-04-26T22:10:11.938066+0000","last_clean":"2024-04-26T22:10:11.938066+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:10:11.938066+0000","last_undegraded":"2024-04-26T22:10:11.938066+0000","last_fullsized":"2024-04-26T22:10:11.938066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":59023,"num_write_kb":718389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'5311","reported_seq":5368,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.831429+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:10:05.831429+0000","last_peered":"2024-04-26T22:10:05.831429+0000","last_clean":"2024-04-26T22:10:05.831429+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:10:05.831429+0000","last_undegraded":"2024-04-26T22:10:05.831429+0000","last_fullsized":"2024-04-26T22:10:05.831429+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80379,"num_write_kb":1004221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'218","reported_seq":282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.885693+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:10:14.885693+0000","last_peered":"2024-04-26T22:10:14.885693+0000","last_clean":"2024-04-26T22:10:14.885693+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:10:14.885693+0000","last_undegraded":"2024-04-26T22:10:14.885693+0000","last_fullsized":"2024-04-26T22:10:14.885693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":61,"num_write":416,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"40'22","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.961590+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:10:11.961590+0000","last_peered":"2024-04-26T22:10:11.961590+0000","last_clean":"2024-04-26T22:10:11.961590+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:10:11.961590+0000","last_undegraded":"2024-04-26T22:10:11.961590+0000","last_fullsized":"2024-04-26T22:10:11.961590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":15,"num_write_kb":8199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"40'4334","reported_seq":4409,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.892208+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:10:11.892208+0000","last_peered":"2024-04-26T22:10:11.892208+0000","last_clean":"2024-04-26T22:10:11.892208+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:10:11.892208+0000","last_undegraded":"2024-04-26T22:10:11.892208+0000","last_fullsized":"2024-04-26T22:10:11.892208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":65365,"num_write_kb":782662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"40'5496","reported_seq":5556,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.837207+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:10:05.837207+0000","last_peered":"2024-04-26T22:10:05.837207+0000","last_clean":"2024-04-26T22:10:05.837207+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:10:05.837207+0000","last_undegraded":"2024-04-26T22:10:05.837207+0000","last_fullsized":"2024-04-26T22:10:05.837207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":82892,"num_write_kb":1050745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'35","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.834069+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:10:05.834069+0000","last_peered":"2024-04-26T22:10:05.834069+0000","last_clean":"2024-04-26T22:10:05.834069+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:10:05.834069+0000","last_undegraded":"2024-04-26T22:10:05.834069+0000","last_fullsized":"2024-04-26T22:10:05.834069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":29,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"28'1","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940808+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:10:00.940808+0000","last_peered":"2024-04-26T22:10:00.940808+0000","last_clean":"2024-04-26T22:10:00.940808+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:10:00.940808+0000","last_undegraded":"2024-04-26T22:10:00.940808+0000","last_fullsized":"2024-04-26T22:10:00.940808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"40'3789","reported_seq":3862,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937218+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:10:11.937218+0000","last_peered":"2024-04-26T22:10:11.937218+0000","last_clean":"2024-04-26T22:10:11.937218+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:10:11.937218+0000","last_undegraded":"2024-04-26T22:10:11.937218+0000","last_fullsized":"2024-04-26T22:10:11.937218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":56913,"num_write_kb":707433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"40'4759","reported_seq":4817,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803693+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:10:05.803693+0000","last_peered":"2024-04-26T22:10:05.803693+0000","last_clean":"2024-04-26T22:10:05.803693+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:10:05.803693+0000","last_undegraded":"2024-04-26T22:10:05.803693+0000","last_fullsized":"2024-04-26T22:10:05.803693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71808,"num_write_kb":921006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"40'7","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.825804+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:10:05.825804+0000","last_peered":"2024-04-26T22:10:05.825804+0000","last_clean":"2024-04-26T22:10:05.825804+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:10:05.825804+0000","last_undegraded":"2024-04-26T22:10:05.825804+0000","last_fullsized":"2024-04-26T22:10:05.825804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"40'183","reported_seq":241,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966669+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:10:11.966669+0000","last_peered":"2024-04-26T22:10:11.966669+0000","last_clean":"2024-04-26T22:10:11.966669+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:10:11.966669+0000","last_undegraded":"2024-04-26T22:10:11.966669+0000","last_fullsized":"2024-04-26T22:10:11.966669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":27847,"num_write":173,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"40'3541","reported_seq":3610,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940587+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:10:11.940587+0000","last_peered":"2024-04-26T22:10:11.940587+0000","last_clean":"2024-04-26T22:10:11.940587+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:10:11.940587+0000","last_undegraded":"2024-04-26T22:10:11.940587+0000","last_fullsized":"2024-04-26T22:10:11.940587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":53073,"num_write_kb":661602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'5106","reported_seq":5164,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.813565+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:10:05.813565+0000","last_peered":"2024-04-26T22:10:05.813565+0000","last_clean":"2024-04-26T22:10:05.813565+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:10:05.813565+0000","last_undegraded":"2024-04-26T22:10:05.813565+0000","last_fullsized":"2024-04-26T22:10:05.813565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":77293,"num_write_kb":982493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"40'221","reported_seq":286,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850755+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:10:05.850755+0000","last_peered":"2024-04-26T22:10:05.850755+0000","last_clean":"2024-04-26T22:10:05.850755+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:10:05.850755+0000","last_undegraded":"2024-04-26T22:10:05.850755+0000","last_fullsized":"2024-04-26T22:10:05.850755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":213,"num_write_kb":8367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"40'5","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.945512+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:10:11.945512+0000","last_peered":"2024-04-26T22:10:11.945512+0000","last_clean":"2024-04-26T22:10:11.945512+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:10:11.945512+0000","last_undegraded":"2024-04-26T22:10:11.945512+0000","last_fullsized":"2024-04-26T22:10:11.945512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"40'3541","reported_seq":3614,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.910214+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:10:11.910214+0000","last_peered":"2024-04-26T22:10:11.910214+0000","last_clean":"2024-04-26T22:10:11.910214+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:10:11.910214+0000","last_undegraded":"2024-04-26T22:10:11.910214+0000","last_fullsized":"2024-04-26T22:10:11.910214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":52805,"num_write_kb":629973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'5114","reported_seq":5174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857162+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:10:05.857162+0000","last_peered":"2024-04-26T22:10:05.857162+0000","last_clean":"2024-04-26T22:10:05.857162+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:10:05.857162+0000","last_undegraded":"2024-04-26T22:10:05.857162+0000","last_fullsized":"2024-04-26T22:10:05.857162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":77095,"num_write_kb":967489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"40'19","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.889177+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:10:14.889177+0000","last_peered":"2024-04-26T22:10:14.889177+0000","last_clean":"2024-04-26T22:10:14.889177+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:10:14.889177+0000","last_undegraded":"2024-04-26T22:10:14.889177+0000","last_fullsized":"2024-04-26T22:10:14.889177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88181,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":134,"num_write":15,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"40'5","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954456+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:10:11.954456+0000","last_peered":"2024-04-26T22:10:11.954456+0000","last_clean":"2024-04-26T22:10:11.954456+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:10:11.954456+0000","last_undegraded":"2024-04-26T22:10:11.954456+0000","last_fullsized":"2024-04-26T22:10:11.954456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'4139","reported_seq":4216,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.947123+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:10:11.947123+0000","last_peered":"2024-04-26T22:10:11.947123+0000","last_clean":"2024-04-26T22:10:11.947123+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:10:11.947123+0000","last_undegraded":"2024-04-26T22:10:11.947123+0000","last_fullsized":"2024-04-26T22:10:11.947123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":62199,"num_write_kb":728313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"40'5239","reported_seq":5297,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810597+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:10:05.810597+0000","last_peered":"2024-04-26T22:10:05.810597+0000","last_clean":"2024-04-26T22:10:05.810597+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:10:05.810597+0000","last_undegraded":"2024-04-26T22:10:05.810597+0000","last_fullsized":"2024-04-26T22:10:05.810597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":79195,"num_write_kb":994121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"40'30","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.863131+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:10:05.863131+0000","last_peered":"2024-04-26T22:10:05.863131+0000","last_clean":"2024-04-26T22:10:05.863131+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:10:05.863131+0000","last_undegraded":"2024-04-26T22:10:05.863131+0000","last_fullsized":"2024-04-26T22:10:05.863131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":8201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"40'211","reported_seq":268,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.956625+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:10:11.956625+0000","last_peered":"2024-04-26T22:10:11.956625+0000","last_clean":"2024-04-26T22:10:11.956625+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:10:11.956625+0000","last_undegraded":"2024-04-26T22:10:11.956625+0000","last_fullsized":"2024-04-26T22:10:11.956625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":74,"num_write":424,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"40'4145","reported_seq":4221,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.957211+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:10:11.957211+0000","last_peered":"2024-04-26T22:10:11.957211+0000","last_clean":"2024-04-26T22:10:11.957211+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:10:11.957211+0000","last_undegraded":"2024-04-26T22:10:11.957211+0000","last_fullsized":"2024-04-26T22:10:11.957211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":62372,"num_write_kb":775201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"40'5748","reported_seq":5805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.762804+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:10:05.762804+0000","last_peered":"2024-04-26T22:10:05.762804+0000","last_clean":"2024-04-26T22:10:05.762804+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:10:05.762804+0000","last_undegraded":"2024-04-26T22:10:05.762804+0000","last_fullsized":"2024-04-26T22:10:05.762804+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":87080,"num_write_kb":1065746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"39'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940251+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:10:00.940251+0000","last_peered":"2024-04-26T22:10:00.940251+0000","last_clean":"2024-04-26T22:10:00.940251+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:10:00.940251+0000","last_undegraded":"2024-04-26T22:10:00.940251+0000","last_fullsized":"2024-04-26T22:10:00.940251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.221589+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:10:01.221589+0000","last_peered":"2024-04-26T22:10:01.221589+0000","last_clean":"2024-04-26T22:10:01.221589+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:10:01.221589+0000","last_undegraded":"2024-04-26T22:10:01.221589+0000","last_fullsized":"2024-04-26T22:10:01.221589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.944318+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:10:11.944318+0000","last_peered":"2024-04-26T22:10:11.944318+0000","last_clean":"2024-04-26T22:10:11.944318+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:10:11.944318+0000","last_undegraded":"2024-04-26T22:10:11.944318+0000","last_fullsized":"2024-04-26T22:10:11.944318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"40'3379","reported_seq":3460,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959450+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:10:11.959450+0000","last_peered":"2024-04-26T22:10:11.959450+0000","last_clean":"2024-04-26T22:10:11.959450+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:10:11.959450+0000","last_undegraded":"2024-04-26T22:10:11.959450+0000","last_fullsized":"2024-04-26T22:10:11.959450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":50792,"num_write_kb":610820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"40'5433","reported_seq":5494,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.869142+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:10:05.869142+0000","last_peered":"2024-04-26T22:10:05.869142+0000","last_clean":"2024-04-26T22:10:05.869142+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:10:05.869142+0000","last_undegraded":"2024-04-26T22:10:05.869142+0000","last_fullsized":"2024-04-26T22:10:05.869142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":81848,"num_write_kb":1014508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"40'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810386+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:10:05.810386+0000","last_peered":"2024-04-26T22:10:05.810386+0000","last_clean":"2024-04-26T22:10:05.810386+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:10:05.810386+0000","last_undegraded":"2024-04-26T22:10:05.810386+0000","last_fullsized":"2024-04-26T22:10:05.810386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"40'201","reported_seq":252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.948751+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:10:11.948751+0000","last_peered":"2024-04-26T22:10:11.948751+0000","last_clean":"2024-04-26T22:10:11.948751+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:10:11.948751+0000","last_undegraded":"2024-04-26T22:10:11.948751+0000","last_fullsized":"2024-04-26T22:10:11.948751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":196,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"40'3780","reported_seq":3857,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.815165+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:10:11.815165+0000","last_peered":"2024-04-26T22:10:11.815165+0000","last_clean":"2024-04-26T22:10:11.815165+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:10:11.815165+0000","last_undegraded":"2024-04-26T22:10:11.815165+0000","last_fullsized":"2024-04-26T22:10:11.815165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":56837,"num_write_kb":699720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"40'5733","reported_seq":5794,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849434+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:10:05.849434+0000","last_peered":"2024-04-26T22:10:05.849434+0000","last_clean":"2024-04-26T22:10:05.849434+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:10:05.849434+0000","last_undegraded":"2024-04-26T22:10:05.849434+0000","last_fullsized":"2024-04-26T22:10:05.849434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5733,"log_dups_size":0,"ondisk_log_size":5733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":86585,"num_write_kb":1068440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"40'19","reported_seq":88,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.943192+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:10:00.943192+0000","last_peered":"2024-04-26T22:10:00.943192+0000","last_clean":"2024-04-26T22:10:00.943192+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:10:00.943192+0000","last_undegraded":"2024-04-26T22:10:00.943192+0000","last_fullsized":"2024-04-26T22:10:00.943192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16390,"num_write":17,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"40'379","reported_seq":446,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955307+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:10:11.955307+0000","last_peered":"2024-04-26T22:10:11.955307+0000","last_clean":"2024-04-26T22:10:11.955307+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:10:11.955307+0000","last_undegraded":"2024-04-26T22:10:11.955307+0000","last_fullsized":"2024-04-26T22:10:11.955307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":25852,"num_write":364,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"40'3909","reported_seq":3987,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946493+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:10:11.946493+0000","last_peered":"2024-04-26T22:10:11.946493+0000","last_clean":"2024-04-26T22:10:11.946493+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:10:11.946493+0000","last_undegraded":"2024-04-26T22:10:11.946493+0000","last_fullsized":"2024-04-26T22:10:11.946493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":58819,"num_write_kb":699051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"40'6248","reported_seq":6306,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823315+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:10:05.823315+0000","last_peered":"2024-04-26T22:10:05.823315+0000","last_clean":"2024-04-26T22:10:05.823315+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:10:05.823315+0000","last_undegraded":"2024-04-26T22:10:05.823315+0000","last_fullsized":"2024-04-26T22:10:05.823315+0000","mapping_epoch":18,"log_start":"28'400","ondisk_log_start":"28'400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":94569,"num_write_kb":1160134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"40'111","reported_seq":183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850721+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:10:05.850721+0000","last_peered":"2024-04-26T22:10:05.850721+0000","last_clean":"2024-04-26T22:10:05.850721+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:10:05.850721+0000","last_undegraded":"2024-04-26T22:10:05.850721+0000","last_fullsized":"2024-04-26T22:10:05.850721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":106,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"40'13","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.935130+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:10:11.935130+0000","last_peered":"2024-04-26T22:10:11.935130+0000","last_clean":"2024-04-26T22:10:11.935130+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:10:11.935130+0000","last_undegraded":"2024-04-26T22:10:11.935130+0000","last_fullsized":"2024-04-26T22:10:11.935130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"40'4399","reported_seq":4473,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.897004+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:10:11.897004+0000","last_peered":"2024-04-26T22:10:11.897004+0000","last_clean":"2024-04-26T22:10:11.897004+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:10:11.897004+0000","last_undegraded":"2024-04-26T22:10:11.897004+0000","last_fullsized":"2024-04-26T22:10:11.897004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":66147,"num_write_kb":822005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"40'5224","reported_seq":5282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822320+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:10:05.822320+0000","last_peered":"2024-04-26T22:10:05.822320+0000","last_clean":"2024-04-26T22:10:05.822320+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:10:05.822320+0000","last_undegraded":"2024-04-26T22:10:05.822320+0000","last_fullsized":"2024-04-26T22:10:05.822320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":78957,"num_write_kb":961695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"40'16","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849672+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:10:05.849672+0000","last_peered":"2024-04-26T22:10:05.849672+0000","last_clean":"2024-04-26T22:10:05.849672+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:10:05.849672+0000","last_undegraded":"2024-04-26T22:10:05.849672+0000","last_fullsized":"2024-04-26T22:10:05.849672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":12294,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951413+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:10:11.951413+0000","last_peered":"2024-04-26T22:10:11.951413+0000","last_clean":"2024-04-26T22:10:11.951413+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:10:11.951413+0000","last_undegraded":"2024-04-26T22:10:11.951413+0000","last_fullsized":"2024-04-26T22:10:11.951413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"40'3562","reported_seq":3638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982357+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:10:11.982357+0000","last_peered":"2024-04-26T22:10:11.982357+0000","last_clean":"2024-04-26T22:10:11.982357+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:10:11.982357+0000","last_undegraded":"2024-04-26T22:10:11.982357+0000","last_fullsized":"2024-04-26T22:10:11.982357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3562,"log_dups_size":0,"ondisk_log_size":3562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":53350,"num_write_kb":651982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"40'5336","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848168+0000","last_change":"2024-04-26T22:00:39.151213+0000","last_active":"2024-04-26T22:10:05.848168+0000","last_peered":"2024-04-26T22:10:05.848168+0000","last_clean":"2024-04-26T22:10:05.848168+0000","last_became_active":"2024-04-26T22:00:39.150992+0000","last_became_peered":"2024-04-26T22:00:39.150992+0000","last_unstale":"2024-04-26T22:10:05.848168+0000","last_undegraded":"2024-04-26T22:10:05.848168+0000","last_fullsized":"2024-04-26T22:10:05.848168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:26.086210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":80461,"num_write_kb":1019956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"40'287","reported_seq":383,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.839145+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:10:05.839145+0000","last_peered":"2024-04-26T22:10:05.839145+0000","last_clean":"2024-04-26T22:10:05.839145+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:10:05.839145+0000","last_undegraded":"2024-04-26T22:10:05.839145+0000","last_fullsized":"2024-04-26T22:10:05.839145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":12387,"num_write":284,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"40'4","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954588+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:10:11.954588+0000","last_peered":"2024-04-26T22:10:11.954588+0000","last_clean":"2024-04-26T22:10:11.954588+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:10:11.954588+0000","last_undegraded":"2024-04-26T22:10:11.954588+0000","last_fullsized":"2024-04-26T22:10:11.954588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"40'4185","reported_seq":4270,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946111+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:10:11.946111+0000","last_peered":"2024-04-26T22:10:11.946111+0000","last_clean":"2024-04-26T22:10:11.946111+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:10:11.946111+0000","last_undegraded":"2024-04-26T22:10:11.946111+0000","last_fullsized":"2024-04-26T22:10:11.946111+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":62914,"num_write_kb":762533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'5750","reported_seq":5810,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.854474+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:10:05.854474+0000","last_peered":"2024-04-26T22:10:05.854474+0000","last_clean":"2024-04-26T22:10:05.854474+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:10:05.854474+0000","last_undegraded":"2024-04-26T22:10:05.854474+0000","last_fullsized":"2024-04-26T22:10:05.854474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":86794,"num_write_kb":1073964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"40'6","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836648+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:10:05.836648+0000","last_peered":"2024-04-26T22:10:05.836648+0000","last_clean":"2024-04-26T22:10:05.836648+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:10:05.836648+0000","last_undegraded":"2024-04-26T22:10:05.836648+0000","last_fullsized":"2024-04-26T22:10:05.836648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.943365+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:10:07.943365+0000","last_peered":"2024-04-26T22:10:07.943365+0000","last_clean":"2024-04-26T22:10:07.943365+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:10:07.943365+0000","last_undegraded":"2024-04-26T22:10:07.943365+0000","last_fullsized":"2024-04-26T22:10:07.943365+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"40'3717","reported_seq":3786,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.778102+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:10:11.778102+0000","last_peered":"2024-04-26T22:10:11.778102+0000","last_clean":"2024-04-26T22:10:11.778102+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:10:11.778102+0000","last_undegraded":"2024-04-26T22:10:11.778102+0000","last_fullsized":"2024-04-26T22:10:11.778102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3717,"log_dups_size":0,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":55831,"num_write_kb":673382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"40'6101","reported_seq":6161,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868996+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:10:05.868996+0000","last_peered":"2024-04-26T22:10:05.868996+0000","last_clean":"2024-04-26T22:10:05.868996+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:10:05.868996+0000","last_undegraded":"2024-04-26T22:10:05.868996+0000","last_fullsized":"2024-04-26T22:10:05.868996+0000","mapping_epoch":18,"log_start":"28'300","ondisk_log_start":"28'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":92157,"num_write_kb":1133252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"40'5","reported_seq":63,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867571+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:10:05.867571+0000","last_peered":"2024-04-26T22:10:05.867571+0000","last_clean":"2024-04-26T22:10:05.867571+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:10:05.867571+0000","last_undegraded":"2024-04-26T22:10:05.867571+0000","last_fullsized":"2024-04-26T22:10:05.867571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"40'374","reported_seq":467,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953183+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:10:11.953183+0000","last_peered":"2024-04-26T22:10:11.953183+0000","last_clean":"2024-04-26T22:10:11.953183+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:10:11.953183+0000","last_undegraded":"2024-04-26T22:10:11.953183+0000","last_fullsized":"2024-04-26T22:10:11.953183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":31157,"num_write":363,"num_write_kb":8442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"40'5464","reported_seq":5545,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.917366+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:10:11.917366+0000","last_peered":"2024-04-26T22:10:11.917366+0000","last_clean":"2024-04-26T22:10:11.917366+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:10:11.917366+0000","last_undegraded":"2024-04-26T22:10:11.917366+0000","last_fullsized":"2024-04-26T22:10:11.917366+0000","mapping_epoch":22,"log_start":"31'200","ondisk_log_start":"31'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":82133,"num_write_kb":980051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'5077","reported_seq":5133,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673410+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:10:05.673410+0000","last_peered":"2024-04-26T22:10:05.673410+0000","last_clean":"2024-04-26T22:10:05.673410+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:10:05.673410+0000","last_undegraded":"2024-04-26T22:10:05.673410+0000","last_fullsized":"2024-04-26T22:10:05.673410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":76618,"num_write_kb":958363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"40'19","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852434+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:10:05.852434+0000","last_peered":"2024-04-26T22:10:05.852434+0000","last_clean":"2024-04-26T22:10:05.852434+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:10:05.852434+0000","last_undegraded":"2024-04-26T22:10:05.852434+0000","last_fullsized":"2024-04-26T22:10:05.852434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":32816,"num_write":13,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"40'15","reported_seq":89,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.994445+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:10:11.994445+0000","last_peered":"2024-04-26T22:10:11.994445+0000","last_clean":"2024-04-26T22:10:11.994445+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:10:11.994445+0000","last_undegraded":"2024-04-26T22:10:11.994445+0000","last_fullsized":"2024-04-26T22:10:11.994445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"40'4714","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982824+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:10:11.982824+0000","last_peered":"2024-04-26T22:10:11.982824+0000","last_clean":"2024-04-26T22:10:11.982824+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:10:11.982824+0000","last_undegraded":"2024-04-26T22:10:11.982824+0000","last_fullsized":"2024-04-26T22:10:11.982824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":70866,"num_write_kb":843772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"40'5114","reported_seq":5170,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.579883+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:10:05.579883+0000","last_peered":"2024-04-26T22:10:05.579883+0000","last_clean":"2024-04-26T22:10:05.579883+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:10:05.579883+0000","last_undegraded":"2024-04-26T22:10:05.579883+0000","last_fullsized":"2024-04-26T22:10:05.579883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77180,"num_write_kb":958350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"40'824","reported_seq":917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848033+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:10:05.848033+0000","last_peered":"2024-04-26T22:10:05.848033+0000","last_clean":"2024-04-26T22:10:05.848033+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:10:05.848033+0000","last_undegraded":"2024-04-26T22:10:05.848033+0000","last_fullsized":"2024-04-26T22:10:05.848033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":36914,"num_write":813,"num_write_kb":13003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"40'5","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977919+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:10:11.977919+0000","last_peered":"2024-04-26T22:10:11.977919+0000","last_clean":"2024-04-26T22:10:11.977919+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:10:11.977919+0000","last_undegraded":"2024-04-26T22:10:11.977919+0000","last_fullsized":"2024-04-26T22:10:11.977919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"40'3745","reported_seq":3823,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.841511+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:10:11.841511+0000","last_peered":"2024-04-26T22:10:11.841511+0000","last_clean":"2024-04-26T22:10:11.841511+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:10:11.841511+0000","last_undegraded":"2024-04-26T22:10:11.841511+0000","last_fullsized":"2024-04-26T22:10:11.841511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56184,"num_write_kb":680067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"40'7267","reported_seq":7326,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824603+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:10:05.824603+0000","last_peered":"2024-04-26T22:10:05.824603+0000","last_clean":"2024-04-26T22:10:05.824603+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:10:05.824603+0000","last_undegraded":"2024-04-26T22:10:05.824603+0000","last_fullsized":"2024-04-26T22:10:05.824603+0000","mapping_epoch":18,"log_start":"30'1800","ondisk_log_start":"30'1800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":109924,"num_write_kb":1323267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"40'370","reported_seq":447,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.881135+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:10:05.881135+0000","last_peered":"2024-04-26T22:10:05.881135+0000","last_clean":"2024-04-26T22:10:05.881135+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:10:05.881135+0000","last_undegraded":"2024-04-26T22:10:05.881135+0000","last_fullsized":"2024-04-26T22:10:05.881135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":367,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"40'231","reported_seq":307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966124+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:10:11.966124+0000","last_peered":"2024-04-26T22:10:11.966124+0000","last_clean":"2024-04-26T22:10:11.966124+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:10:11.966124+0000","last_undegraded":"2024-04-26T22:10:11.966124+0000","last_fullsized":"2024-04-26T22:10:11.966124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":33257,"num_write":223,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"40'4111","reported_seq":4188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967266+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:10:11.967266+0000","last_peered":"2024-04-26T22:10:11.967266+0000","last_clean":"2024-04-26T22:10:11.967266+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:10:11.967266+0000","last_undegraded":"2024-04-26T22:10:11.967266+0000","last_fullsized":"2024-04-26T22:10:11.967266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4111,"log_dups_size":0,"ondisk_log_size":4111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61736,"num_write_kb":766472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"40'4628","reported_seq":4688,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867072+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:10:05.867072+0000","last_peered":"2024-04-26T22:10:05.867072+0000","last_clean":"2024-04-26T22:10:05.867072+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:10:05.867072+0000","last_undegraded":"2024-04-26T22:10:05.867072+0000","last_fullsized":"2024-04-26T22:10:05.867072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":69686,"num_write_kb":866678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"40'102","reported_seq":162,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842367+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:10:05.842367+0000","last_peered":"2024-04-26T22:10:05.842367+0000","last_clean":"2024-04-26T22:10:05.842367+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:10:05.842367+0000","last_undegraded":"2024-04-26T22:10:05.842367+0000","last_fullsized":"2024-04-26T22:10:05.842367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":98,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"40'12","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.943439+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:10:11.943439+0000","last_peered":"2024-04-26T22:10:11.943439+0000","last_clean":"2024-04-26T22:10:11.943439+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:10:11.943439+0000","last_undegraded":"2024-04-26T22:10:11.943439+0000","last_fullsized":"2024-04-26T22:10:11.943439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'4365","reported_seq":4453,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.968703+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:10:11.968703+0000","last_peered":"2024-04-26T22:10:11.968703+0000","last_clean":"2024-04-26T22:10:11.968703+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:10:11.968703+0000","last_undegraded":"2024-04-26T22:10:11.968703+0000","last_fullsized":"2024-04-26T22:10:11.968703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4365,"log_dups_size":0,"ondisk_log_size":4365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":65682,"num_write_kb":806692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"40'5549","reported_seq":5613,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824597+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:10:05.824597+0000","last_peered":"2024-04-26T22:10:05.824597+0000","last_clean":"2024-04-26T22:10:05.824597+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:10:05.824597+0000","last_undegraded":"2024-04-26T22:10:05.824597+0000","last_fullsized":"2024-04-26T22:10:05.824597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":83744,"num_write_kb":1000289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"40'4","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.846233+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:10:05.846233+0000","last_peered":"2024-04-26T22:10:05.846233+0000","last_clean":"2024-04-26T22:10:05.846233+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:10:05.846233+0000","last_undegraded":"2024-04-26T22:10:05.846233+0000","last_fullsized":"2024-04-26T22:10:05.846233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953872+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:10:11.953872+0000","last_peered":"2024-04-26T22:10:11.953872+0000","last_clean":"2024-04-26T22:10:11.953872+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:10:11.953872+0000","last_undegraded":"2024-04-26T22:10:11.953872+0000","last_fullsized":"2024-04-26T22:10:11.953872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4980","reported_seq":5062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967201+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:10:11.967201+0000","last_peered":"2024-04-26T22:10:11.967201+0000","last_clean":"2024-04-26T22:10:11.967201+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:10:11.967201+0000","last_undegraded":"2024-04-26T22:10:11.967201+0000","last_fullsized":"2024-04-26T22:10:11.967201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":74941,"num_write_kb":925745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"40'16","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.970996+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:10:11.970996+0000","last_peered":"2024-04-26T22:10:11.970996+0000","last_clean":"2024-04-26T22:10:11.970996+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:10:11.970996+0000","last_undegraded":"2024-04-26T22:10:11.970996+0000","last_fullsized":"2024-04-26T22:10:11.970996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":8,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"40'6345","reported_seq":6406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.871078+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:10:05.871078+0000","last_peered":"2024-04-26T22:10:05.871078+0000","last_clean":"2024-04-26T22:10:05.871078+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:10:05.871078+0000","last_undegraded":"2024-04-26T22:10:05.871078+0000","last_fullsized":"2024-04-26T22:10:05.871078+0000","mapping_epoch":18,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":95977,"num_write_kb":1176758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"40'19","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.772388+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:10:05.772388+0000","last_peered":"2024-04-26T22:10:05.772388+0000","last_clean":"2024-04-26T22:10:05.772388+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:10:05.772388+0000","last_undegraded":"2024-04-26T22:10:05.772388+0000","last_fullsized":"2024-04-26T22:10:05.772388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":28672,"num_write":16,"num_write_kb":8206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"40'4260","reported_seq":4329,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.876539+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:10:11.876539+0000","last_peered":"2024-04-26T22:10:11.876539+0000","last_clean":"2024-04-26T22:10:11.876539+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:10:11.876539+0000","last_undegraded":"2024-04-26T22:10:11.876539+0000","last_fullsized":"2024-04-26T22:10:11.876539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":64151,"num_write_kb":776804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'25","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.953756+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:10:00.953756+0000","last_peered":"2024-04-26T22:10:00.953756+0000","last_clean":"2024-04-26T22:10:00.953756+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:10:00.953756+0000","last_undegraded":"2024-04-26T22:10:00.953756+0000","last_fullsized":"2024-04-26T22:10:00.953756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20512,"num_write":20,"num_write_kb":4102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"40'5323","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874479+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:10:05.874479+0000","last_peered":"2024-04-26T22:10:05.874479+0000","last_clean":"2024-04-26T22:10:05.874479+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:10:05.874479+0000","last_undegraded":"2024-04-26T22:10:05.874479+0000","last_fullsized":"2024-04-26T22:10:05.874479+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":80432,"num_write_kb":1004371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"40'46","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.934549+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:10:00.934549+0000","last_peered":"2024-04-26T22:10:00.934549+0000","last_clean":"2024-04-26T22:10:00.934549+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:10:00.934549+0000","last_undegraded":"2024-04-26T22:10:00.934549+0000","last_fullsized":"2024-04-26T22:10:00.934549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":41,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'4905","reported_seq":4991,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940116+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:10:11.940116+0000","last_peered":"2024-04-26T22:10:11.940116+0000","last_clean":"2024-04-26T22:10:11.940116+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:10:11.940116+0000","last_undegraded":"2024-04-26T22:10:11.940116+0000","last_fullsized":"2024-04-26T22:10:11.940116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":74018,"num_write_kb":905043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"40'145","reported_seq":213,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.833871+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:10:05.833871+0000","last_peered":"2024-04-26T22:10:05.833871+0000","last_clean":"2024-04-26T22:10:05.833871+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:10:05.833871+0000","last_undegraded":"2024-04-26T22:10:05.833871+0000","last_fullsized":"2024-04-26T22:10:05.833871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":136,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"40'4250","reported_seq":4307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.802351+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:10:05.802351+0000","last_peered":"2024-04-26T22:10:05.802351+0000","last_clean":"2024-04-26T22:10:05.802351+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:10:05.802351+0000","last_undegraded":"2024-04-26T22:10:05.802351+0000","last_fullsized":"2024-04-26T22:10:05.802351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":64229,"num_write_kb":785523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"40'38","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.026729+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:10:07.026729+0000","last_peered":"2024-04-26T22:10:07.026729+0000","last_clean":"2024-04-26T22:10:07.026729+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:10:07.026729+0000","last_undegraded":"2024-04-26T22:10:07.026729+0000","last_fullsized":"2024-04-26T22:10:07.026729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":35,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2966","reported_seq":3026,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.990722+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:10:11.990722+0000","last_peered":"2024-04-26T22:10:11.990722+0000","last_clean":"2024-04-26T22:10:11.990722+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:10:11.990722+0000","last_undegraded":"2024-04-26T22:10:11.990722+0000","last_fullsized":"2024-04-26T22:10:11.990722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":2966,"log_dups_size":0,"ondisk_log_size":2966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":44534,"num_write_kb":522679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"40'1255","reported_seq":1319,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.935885+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:10:00.935885+0000","last_peered":"2024-04-26T22:10:00.935885+0000","last_clean":"2024-04-26T22:10:00.935885+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:10:00.935885+0000","last_undegraded":"2024-04-26T22:10:00.935885+0000","last_fullsized":"2024-04-26T22:10:00.935885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1255,"log_dups_size":0,"ondisk_log_size":1255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":20480,"num_write":1252,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"40'4813","reported_seq":4872,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836289+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:10:05.836289+0000","last_peered":"2024-04-26T22:10:05.836289+0000","last_clean":"2024-04-26T22:10:05.836289+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:10:05.836289+0000","last_undegraded":"2024-04-26T22:10:05.836289+0000","last_fullsized":"2024-04-26T22:10:05.836289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":72672,"num_write_kb":866290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"40'86","reported_seq":166,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953507+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:10:11.953507+0000","last_peered":"2024-04-26T22:10:11.953507+0000","last_clean":"2024-04-26T22:10:11.953507+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:10:11.953507+0000","last_undegraded":"2024-04-26T22:10:11.953507+0000","last_fullsized":"2024-04-26T22:10:11.953507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":62,"num_read_kb":49200,"num_write":70,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"40'3892","reported_seq":3964,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954209+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:10:11.954209+0000","last_peered":"2024-04-26T22:10:11.954209+0000","last_clean":"2024-04-26T22:10:11.954209+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:10:11.954209+0000","last_undegraded":"2024-04-26T22:10:11.954209+0000","last_fullsized":"2024-04-26T22:10:11.954209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":58573,"num_write_kb":703698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"40'6654","reported_seq":6712,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836315+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:10:05.836315+0000","last_peered":"2024-04-26T22:10:05.836315+0000","last_clean":"2024-04-26T22:10:05.836315+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:10:05.836315+0000","last_undegraded":"2024-04-26T22:10:05.836315+0000","last_fullsized":"2024-04-26T22:10:05.836315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":100552,"num_write_kb":1253937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"40'4775","reported_seq":4859,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979203+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:10:11.979203+0000","last_peered":"2024-04-26T22:10:11.979203+0000","last_clean":"2024-04-26T22:10:11.979203+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:10:11.979203+0000","last_undegraded":"2024-04-26T22:10:11.979203+0000","last_fullsized":"2024-04-26T22:10:11.979203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":71741,"num_write_kb":892635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"40'5861","reported_seq":5917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673275+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:10:05.673275+0000","last_peered":"2024-04-26T22:10:05.673275+0000","last_clean":"2024-04-26T22:10:05.673275+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:10:05.673275+0000","last_undegraded":"2024-04-26T22:10:05.673275+0000","last_fullsized":"2024-04-26T22:10:05.673275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88543,"num_write_kb":1086952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"40'3712","reported_seq":3779,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937208+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:10:11.937208+0000","last_peered":"2024-04-26T22:10:11.937208+0000","last_clean":"2024-04-26T22:10:11.937208+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:10:11.937208+0000","last_undegraded":"2024-04-26T22:10:11.937208+0000","last_fullsized":"2024-04-26T22:10:11.937208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":55802,"num_write_kb":677006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"40'6511","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.793662+0000","last_change":"2024-04-26T22:00:39.200941+0000","last_active":"2024-04-26T22:10:05.793662+0000","last_peered":"2024-04-26T22:10:05.793662+0000","last_clean":"2024-04-26T22:10:05.793662+0000","last_became_active":"2024-04-26T22:00:39.200592+0000","last_became_peered":"2024-04-26T22:00:39.200592+0000","last_unstale":"2024-04-26T22:10:05.793662+0000","last_undegraded":"2024-04-26T22:10:05.793662+0000","last_fullsized":"2024-04-26T22:10:05.793662+0000","mapping_epoch":25,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:59.988559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":98396,"num_write_kb":1218831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"40'4288","reported_seq":4364,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.886723+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:10:11.886723+0000","last_peered":"2024-04-26T22:10:11.886723+0000","last_clean":"2024-04-26T22:10:11.886723+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:10:11.886723+0000","last_undegraded":"2024-04-26T22:10:11.886723+0000","last_fullsized":"2024-04-26T22:10:11.886723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":64735,"num_write_kb":800999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"40'4650","reported_seq":4707,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852016+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:10:05.852016+0000","last_peered":"2024-04-26T22:10:05.852016+0000","last_clean":"2024-04-26T22:10:05.852016+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:10:05.852016+0000","last_undegraded":"2024-04-26T22:10:05.852016+0000","last_fullsized":"2024-04-26T22:10:05.852016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":70263,"num_write_kb":869775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"40'5113","reported_seq":5194,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.915535+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:10:11.915535+0000","last_peered":"2024-04-26T22:10:11.915535+0000","last_clean":"2024-04-26T22:10:11.915535+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:10:11.915535+0000","last_undegraded":"2024-04-26T22:10:11.915535+0000","last_fullsized":"2024-04-26T22:10:11.915535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":76970,"num_write_kb":914783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"40'4980","reported_seq":5038,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.818657+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:10:05.818657+0000","last_peered":"2024-04-26T22:10:05.818657+0000","last_clean":"2024-04-26T22:10:05.818657+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:10:05.818657+0000","last_undegraded":"2024-04-26T22:10:05.818657+0000","last_fullsized":"2024-04-26T22:10:05.818657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":75382,"num_write_kb":914602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"40'3556","reported_seq":3628,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.958144+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:10:11.958144+0000","last_peered":"2024-04-26T22:10:11.958144+0000","last_clean":"2024-04-26T22:10:11.958144+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:10:11.958144+0000","last_undegraded":"2024-04-26T22:10:11.958144+0000","last_fullsized":"2024-04-26T22:10:11.958144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":53572,"num_write_kb":653683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"40'5738","reported_seq":5795,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.760118+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:10:05.760118+0000","last_peered":"2024-04-26T22:10:05.760118+0000","last_clean":"2024-04-26T22:10:05.760118+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:10:05.760118+0000","last_undegraded":"2024-04-26T22:10:05.760118+0000","last_fullsized":"2024-04-26T22:10:05.760118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":86675,"num_write_kb":1103247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"40'5159","reported_seq":5238,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951137+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:10:11.951137+0000","last_peered":"2024-04-26T22:10:11.951137+0000","last_clean":"2024-04-26T22:10:11.951137+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:10:11.951137+0000","last_undegraded":"2024-04-26T22:10:11.951137+0000","last_fullsized":"2024-04-26T22:10:11.951137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":77704,"num_write_kb":942965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"40'4895","reported_seq":4967,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842652+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:10:05.842652+0000","last_peered":"2024-04-26T22:10:05.842652+0000","last_clean":"2024-04-26T22:10:05.842652+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:10:05.842652+0000","last_undegraded":"2024-04-26T22:10:05.842652+0000","last_fullsized":"2024-04-26T22:10:05.842652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":363,"num_write":74201,"num_write_kb":908910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"40'4370","reported_seq":4439,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.838189+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:10:11.838189+0000","last_peered":"2024-04-26T22:10:11.838189+0000","last_clean":"2024-04-26T22:10:11.838189+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:10:11.838189+0000","last_undegraded":"2024-04-26T22:10:11.838189+0000","last_fullsized":"2024-04-26T22:10:11.838189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4370,"log_dups_size":0,"ondisk_log_size":4370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":65733,"num_write_kb":804896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"40'6465","reported_seq":6525,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.853311+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:10:05.853311+0000","last_peered":"2024-04-26T22:10:05.853311+0000","last_clean":"2024-04-26T22:10:05.853311+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:10:05.853311+0000","last_undegraded":"2024-04-26T22:10:05.853311+0000","last_fullsized":"2024-04-26T22:10:05.853311+0000","mapping_epoch":18,"log_start":"30'1200","ondisk_log_start":"30'1200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5265,"log_dups_size":0,"ondisk_log_size":5265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":97620,"num_write_kb":1192207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"40'3334","reported_seq":3406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982941+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:10:11.982941+0000","last_peered":"2024-04-26T22:10:11.982941+0000","last_clean":"2024-04-26T22:10:11.982941+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:10:11.982941+0000","last_undegraded":"2024-04-26T22:10:11.982941+0000","last_fullsized":"2024-04-26T22:10:11.982941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":49724,"num_write_kb":588291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"40'4839","reported_seq":4896,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857008+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:10:05.857008+0000","last_peered":"2024-04-26T22:10:05.857008+0000","last_clean":"2024-04-26T22:10:05.857008+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:10:05.857008+0000","last_undegraded":"2024-04-26T22:10:05.857008+0000","last_fullsized":"2024-04-26T22:10:05.857008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":73090,"num_write_kb":888426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"40'5009","reported_seq":5079,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.919327+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:10:11.919327+0000","last_peered":"2024-04-26T22:10:11.919327+0000","last_clean":"2024-04-26T22:10:11.919327+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:10:11.919327+0000","last_undegraded":"2024-04-26T22:10:11.919327+0000","last_fullsized":"2024-04-26T22:10:11.919327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":75285,"num_write_kb":916871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"40'5333","reported_seq":5390,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.697837+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:10:05.697837+0000","last_peered":"2024-04-26T22:10:05.697837+0000","last_clean":"2024-04-26T22:10:05.697837+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:10:05.697837+0000","last_undegraded":"2024-04-26T22:10:05.697837+0000","last_fullsized":"2024-04-26T22:10:05.697837+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80596,"num_write_kb":994453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"40'4157","reported_seq":4231,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959972+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:10:11.959972+0000","last_peered":"2024-04-26T22:10:11.959972+0000","last_clean":"2024-04-26T22:10:11.959972+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:10:11.959972+0000","last_undegraded":"2024-04-26T22:10:11.959972+0000","last_fullsized":"2024-04-26T22:10:11.959972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":62491,"num_write_kb":760147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"40'5861","reported_seq":5904,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835233+0000","last_change":"2024-04-26T22:00:38.947156+0000","last_active":"2024-04-26T22:10:05.835233+0000","last_peered":"2024-04-26T22:10:05.835233+0000","last_clean":"2024-04-26T22:10:05.835233+0000","last_became_active":"2024-04-26T22:00:38.946980+0000","last_became_peered":"2024-04-26T22:00:38.946980+0000","last_unstale":"2024-04-26T22:10:05.835233+0000","last_undegraded":"2024-04-26T22:10:05.835233+0000","last_fullsized":"2024-04-26T22:10:05.835233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:02.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88698,"num_write_kb":1108811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"40'4156","reported_seq":4228,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977006+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:10:11.977006+0000","last_peered":"2024-04-26T22:10:11.977006+0000","last_clean":"2024-04-26T22:10:11.977006+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:10:11.977006+0000","last_undegraded":"2024-04-26T22:10:11.977006+0000","last_fullsized":"2024-04-26T22:10:11.977006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62195,"num_write_kb":734457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"40'5569","reported_seq":5629,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868845+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:10:05.868845+0000","last_peered":"2024-04-26T22:10:05.868845+0000","last_clean":"2024-04-26T22:10:05.868845+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:10:05.868845+0000","last_undegraded":"2024-04-26T22:10:05.868845+0000","last_fullsized":"2024-04-26T22:10:05.868845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5569,"log_dups_size":0,"ondisk_log_size":5569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":83899,"num_write_kb":1024385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"40'4460","reported_seq":4531,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955590+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:10:11.955590+0000","last_peered":"2024-04-26T22:10:11.955590+0000","last_clean":"2024-04-26T22:10:11.955590+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:10:11.955590+0000","last_undegraded":"2024-04-26T22:10:11.955590+0000","last_fullsized":"2024-04-26T22:10:11.955590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":67080,"num_write_kb":827171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"40'5284","reported_seq":5342,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.845154+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:10:05.845154+0000","last_peered":"2024-04-26T22:10:05.845154+0000","last_clean":"2024-04-26T22:10:05.845154+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:10:05.845154+0000","last_undegraded":"2024-04-26T22:10:05.845154+0000","last_fullsized":"2024-04-26T22:10:05.845154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":79924,"num_write_kb":1000899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"40'4361","reported_seq":4441,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972500+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:10:11.972500+0000","last_peered":"2024-04-26T22:10:11.972500+0000","last_clean":"2024-04-26T22:10:11.972500+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:10:11.972500+0000","last_undegraded":"2024-04-26T22:10:11.972500+0000","last_fullsized":"2024-04-26T22:10:11.972500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65464,"num_write_kb":794873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"40'5458","reported_seq":5516,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.860815+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:10:05.860815+0000","last_peered":"2024-04-26T22:10:05.860815+0000","last_clean":"2024-04-26T22:10:05.860815+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:10:05.860815+0000","last_undegraded":"2024-04-26T22:10:05.860815+0000","last_fullsized":"2024-04-26T22:10:05.860815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":82585,"num_write_kb":1000929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"40'3624","reported_seq":3690,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966367+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:10:11.966367+0000","last_peered":"2024-04-26T22:10:11.966367+0000","last_clean":"2024-04-26T22:10:11.966367+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:10:11.966367+0000","last_undegraded":"2024-04-26T22:10:11.966367+0000","last_fullsized":"2024-04-26T22:10:11.966367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":54495,"num_write_kb":662506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"40'4431","reported_seq":4495,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.870020+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:10:05.870020+0000","last_peered":"2024-04-26T22:10:05.870020+0000","last_clean":"2024-04-26T22:10:05.870020+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:10:05.870020+0000","last_undegraded":"2024-04-26T22:10:05.870020+0000","last_fullsized":"2024-04-26T22:10:05.870020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":66582,"num_write_kb":814643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"40'4089","reported_seq":4165,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.832376+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:10:11.832376+0000","last_peered":"2024-04-26T22:10:11.832376+0000","last_clean":"2024-04-26T22:10:11.832376+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:10:11.832376+0000","last_undegraded":"2024-04-26T22:10:11.832376+0000","last_fullsized":"2024-04-26T22:10:11.832376+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4089,"log_dups_size":0,"ondisk_log_size":4089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61370,"num_write_kb":735117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"40'6428","reported_seq":6487,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822613+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:10:05.822613+0000","last_peered":"2024-04-26T22:10:05.822613+0000","last_clean":"2024-04-26T22:10:05.822613+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:10:05.822613+0000","last_undegraded":"2024-04-26T22:10:05.822613+0000","last_fullsized":"2024-04-26T22:10:05.822613+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":97094,"num_write_kb":1219735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"40'4506","reported_seq":4588,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.945454+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:10:11.945454+0000","last_peered":"2024-04-26T22:10:11.945454+0000","last_clean":"2024-04-26T22:10:11.945454+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:10:11.945454+0000","last_undegraded":"2024-04-26T22:10:11.945454+0000","last_fullsized":"2024-04-26T22:10:11.945454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":67596,"num_write_kb":862862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"40'4747","reported_seq":4805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803411+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:10:05.803411+0000","last_peered":"2024-04-26T22:10:05.803411+0000","last_clean":"2024-04-26T22:10:05.803411+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:10:05.803411+0000","last_undegraded":"2024-04-26T22:10:05.803411+0000","last_fullsized":"2024-04-26T22:10:05.803411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71795,"num_write_kb":917779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"40'4901","reported_seq":4976,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.894392+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:10:11.894392+0000","last_peered":"2024-04-26T22:10:11.894392+0000","last_clean":"2024-04-26T22:10:11.894392+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:10:11.894392+0000","last_undegraded":"2024-04-26T22:10:11.894392+0000","last_fullsized":"2024-04-26T22:10:11.894392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4901,"log_dups_size":0,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":73795,"num_write_kb":926135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"40'5581","reported_seq":5639,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.843203+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:10:05.843203+0000","last_peered":"2024-04-26T22:10:05.843203+0000","last_clean":"2024-04-26T22:10:05.843203+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:10:05.843203+0000","last_undegraded":"2024-04-26T22:10:05.843203+0000","last_fullsized":"2024-04-26T22:10:05.843203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5581,"log_dups_size":0,"ondisk_log_size":5581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":84297,"num_write_kb":1026829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"40'3600","reported_seq":3671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.933325+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:10:11.933325+0000","last_peered":"2024-04-26T22:10:11.933325+0000","last_clean":"2024-04-26T22:10:11.933325+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:10:11.933325+0000","last_undegraded":"2024-04-26T22:10:11.933325+0000","last_fullsized":"2024-04-26T22:10:11.933325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":54091,"num_write_kb":649095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"40'5612","reported_seq":5671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.789467+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:10:05.789467+0000","last_peered":"2024-04-26T22:10:05.789467+0000","last_clean":"2024-04-26T22:10:05.789467+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:10:05.789467+0000","last_undegraded":"2024-04-26T22:10:05.789467+0000","last_fullsized":"2024-04-26T22:10:05.789467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":84765,"num_write_kb":1047612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"40'3974","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.882273+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:10:11.882273+0000","last_peered":"2024-04-26T22:10:11.882273+0000","last_clean":"2024-04-26T22:10:11.882273+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:10:11.882273+0000","last_undegraded":"2024-04-26T22:10:11.882273+0000","last_fullsized":"2024-04-26T22:10:11.882273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3974,"log_dups_size":0,"ondisk_log_size":3974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":59852,"num_write_kb":718261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"40'4819","reported_seq":4881,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.858034+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:10:05.858034+0000","last_peered":"2024-04-26T22:10:05.858034+0000","last_clean":"2024-04-26T22:10:05.858034+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:10:05.858034+0000","last_undegraded":"2024-04-26T22:10:05.858034+0000","last_fullsized":"2024-04-26T22:10:05.858034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":72748,"num_write_kb":932622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"40'4140","reported_seq":4220,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.931602+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:10:11.931602+0000","last_peered":"2024-04-26T22:10:11.931602+0000","last_clean":"2024-04-26T22:10:11.931602+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:10:11.931602+0000","last_undegraded":"2024-04-26T22:10:11.931602+0000","last_fullsized":"2024-04-26T22:10:11.931602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":62174,"num_write_kb":775596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"40'4503","reported_seq":4562,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.797708+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:10:05.797708+0000","last_peered":"2024-04-26T22:10:05.797708+0000","last_clean":"2024-04-26T22:10:05.797708+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:10:05.797708+0000","last_undegraded":"2024-04-26T22:10:05.797708+0000","last_fullsized":"2024-04-26T22:10:05.797708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":68209,"num_write_kb":848035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"40'3768","reported_seq":3848,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.978793+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:10:11.978793+0000","last_peered":"2024-04-26T22:10:11.978793+0000","last_clean":"2024-04-26T22:10:11.978793+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:10:11.978793+0000","last_undegraded":"2024-04-26T22:10:11.978793+0000","last_fullsized":"2024-04-26T22:10:11.978793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":56583,"num_write_kb":708038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"40'5303","reported_seq":5359,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.717590+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:10:05.717590+0000","last_peered":"2024-04-26T22:10:05.717590+0000","last_clean":"2024-04-26T22:10:05.717590+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:10:05.717590+0000","last_undegraded":"2024-04-26T22:10:05.717590+0000","last_fullsized":"2024-04-26T22:10:05.717590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80229,"num_write_kb":963753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"40'3667","reported_seq":3745,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.927871+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:10:11.927871+0000","last_peered":"2024-04-26T22:10:11.927871+0000","last_clean":"2024-04-26T22:10:11.927871+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:10:11.927871+0000","last_undegraded":"2024-04-26T22:10:11.927871+0000","last_fullsized":"2024-04-26T22:10:11.927871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3667,"log_dups_size":0,"ondisk_log_size":3667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":55047,"num_write_kb":687611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"40'6069","reported_seq":6128,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810736+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:10:05.810736+0000","last_peered":"2024-04-26T22:10:05.810736+0000","last_clean":"2024-04-26T22:10:05.810736+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:10:05.810736+0000","last_undegraded":"2024-04-26T22:10:05.810736+0000","last_fullsized":"2024-04-26T22:10:05.810736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":91606,"num_write_kb":1169870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"40'4173","reported_seq":4252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953675+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:10:11.953675+0000","last_peered":"2024-04-26T22:10:11.953675+0000","last_clean":"2024-04-26T22:10:11.953675+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:10:11.953675+0000","last_undegraded":"2024-04-26T22:10:11.953675+0000","last_fullsized":"2024-04-26T22:10:11.953675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":62684,"num_write_kb":755589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"40'6512","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.763471+0000","last_change":"2024-04-26T22:00:39.201110+0000","last_active":"2024-04-26T22:10:05.763471+0000","last_peered":"2024-04-26T22:10:05.763471+0000","last_clean":"2024-04-26T22:10:05.763471+0000","last_became_active":"2024-04-26T22:00:39.200691+0000","last_became_peered":"2024-04-26T22:00:39.200691+0000","last_unstale":"2024-04-26T22:10:05.763471+0000","last_undegraded":"2024-04-26T22:10:05.763471+0000","last_fullsized":"2024-04-26T22:10:05.763471+0000","mapping_epoch":25,"log_start":"29'700","ondisk_log_start":"29'700","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:06.181131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":98505,"num_write_kb":1266779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"40'5161","reported_seq":5247,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982946+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:10:11.982946+0000","last_peered":"2024-04-26T22:10:11.982946+0000","last_clean":"2024-04-26T22:10:11.982946+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:10:11.982946+0000","last_undegraded":"2024-04-26T22:10:11.982946+0000","last_fullsized":"2024-04-26T22:10:11.982946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":77820,"num_write_kb":957388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"40'5846","reported_seq":5903,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.706015+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:10:05.706015+0000","last_peered":"2024-04-26T22:10:05.706015+0000","last_clean":"2024-04-26T22:10:05.706015+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:10:05.706015+0000","last_undegraded":"2024-04-26T22:10:05.706015+0000","last_fullsized":"2024-04-26T22:10:05.706015+0000","mapping_epoch":18,"log_start":"29'600","ondisk_log_start":"29'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88593,"num_write_kb":1072922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"40'3484","reported_seq":3552,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972160+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:10:11.972160+0000","last_peered":"2024-04-26T22:10:11.972160+0000","last_clean":"2024-04-26T22:10:11.972160+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:10:11.972160+0000","last_undegraded":"2024-04-26T22:10:11.972160+0000","last_fullsized":"2024-04-26T22:10:11.972160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3484,"log_dups_size":0,"ondisk_log_size":3484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":52218,"num_write_kb":635931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"40'5213","reported_seq":5273,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874711+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:10:05.874711+0000","last_peered":"2024-04-26T22:10:05.874711+0000","last_clean":"2024-04-26T22:10:05.874711+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:10:05.874711+0000","last_undegraded":"2024-04-26T22:10:05.874711+0000","last_fullsized":"2024-04-26T22:10:05.874711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":78898,"num_write_kb":987567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"40'3779","reported_seq":3858,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.698994+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:10:11.698994+0000","last_peered":"2024-04-26T22:10:11.698994+0000","last_clean":"2024-04-26T22:10:11.698994+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:10:11.698994+0000","last_undegraded":"2024-04-26T22:10:11.698994+0000","last_fullsized":"2024-04-26T22:10:11.698994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56904,"num_write_kb":701117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"40'4635","reported_seq":4692,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.724357+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:10:05.724357+0000","last_peered":"2024-04-26T22:10:05.724357+0000","last_clean":"2024-04-26T22:10:05.724357+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:10:05.724357+0000","last_undegraded":"2024-04-26T22:10:05.724357+0000","last_fullsized":"2024-04-26T22:10:05.724357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4635,"log_dups_size":0,"ondisk_log_size":4635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":70184,"num_write_kb":869081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"40'4125","reported_seq":4197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954407+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:10:11.954407+0000","last_peered":"2024-04-26T22:10:11.954407+0000","last_clean":"2024-04-26T22:10:11.954407+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:10:11.954407+0000","last_undegraded":"2024-04-26T22:10:11.954407+0000","last_fullsized":"2024-04-26T22:10:11.954407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62115,"num_write_kb":743778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"40'6598","reported_seq":6661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.878241+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:10:05.878241+0000","last_peered":"2024-04-26T22:10:05.878241+0000","last_clean":"2024-04-26T22:10:05.878241+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:10:05.878241+0000","last_undegraded":"2024-04-26T22:10:05.878241+0000","last_fullsized":"2024-04-26T22:10:05.878241+0000","mapping_epoch":18,"log_start":"29'800","ondisk_log_start":"29'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":99507,"num_write_kb":1174132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"40'3584","reported_seq":3653,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979143+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:10:11.979143+0000","last_peered":"2024-04-26T22:10:11.979143+0000","last_clean":"2024-04-26T22:10:11.979143+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:10:11.979143+0000","last_undegraded":"2024-04-26T22:10:11.979143+0000","last_fullsized":"2024-04-26T22:10:11.979143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3584,"log_dups_size":0,"ondisk_log_size":3584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":54000,"num_write_kb":647349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"40'4854","reported_seq":4911,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.499989+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:10:05.499989+0000","last_peered":"2024-04-26T22:10:05.499989+0000","last_clean":"2024-04-26T22:10:05.499989+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:10:05.499989+0000","last_undegraded":"2024-04-26T22:10:05.499989+0000","last_fullsized":"2024-04-26T22:10:05.499989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73549,"num_write_kb":940613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"40'4744","reported_seq":4833,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.993205+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:10:11.993205+0000","last_peered":"2024-04-26T22:10:11.993205+0000","last_clean":"2024-04-26T22:10:11.993205+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:10:11.993205+0000","last_undegraded":"2024-04-26T22:10:11.993205+0000","last_fullsized":"2024-04-26T22:10:11.993205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":71239,"num_write_kb":843376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"40'5453","reported_seq":5511,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835411+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:10:05.835411+0000","last_peered":"2024-04-26T22:10:05.835411+0000","last_clean":"2024-04-26T22:10:05.835411+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:10:05.835411+0000","last_undegraded":"2024-04-26T22:10:05.835411+0000","last_fullsized":"2024-04-26T22:10:05.835411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5453,"log_dups_size":0,"ondisk_log_size":5453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":82428,"num_write_kb":1036078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"40'4470","reported_seq":4548,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.844612+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:10:11.844612+0000","last_peered":"2024-04-26T22:10:11.844612+0000","last_clean":"2024-04-26T22:10:11.844612+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:10:11.844612+0000","last_undegraded":"2024-04-26T22:10:11.844612+0000","last_fullsized":"2024-04-26T22:10:11.844612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":67360,"num_write_kb":793318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"40'6816","reported_seq":6875,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.873639+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:10:05.873639+0000","last_peered":"2024-04-26T22:10:05.873639+0000","last_clean":"2024-04-26T22:10:05.873639+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:10:05.873639+0000","last_undegraded":"2024-04-26T22:10:05.873639+0000","last_fullsized":"2024-04-26T22:10:05.873639+0000","mapping_epoch":18,"log_start":"29'1100","ondisk_log_start":"29'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":103074,"num_write_kb":1294412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"40'3980","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986399+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:10:11.986399+0000","last_peered":"2024-04-26T22:10:11.986399+0000","last_clean":"2024-04-26T22:10:11.986399+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:10:11.986399+0000","last_undegraded":"2024-04-26T22:10:11.986399+0000","last_fullsized":"2024-04-26T22:10:11.986399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":59871,"num_write_kb":728175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"40'4761","reported_seq":4821,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.861444+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:10:05.861444+0000","last_peered":"2024-04-26T22:10:05.861444+0000","last_clean":"2024-04-26T22:10:05.861444+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:10:05.861444+0000","last_undegraded":"2024-04-26T22:10:05.861444+0000","last_fullsized":"2024-04-26T22:10:05.861444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":72121,"num_write_kb":895283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1800,"num_read_kb":0,"num_write":3978586,"num_write_kb":48381196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264479,"ondisk_log_size":264479,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":109330781,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":691,"num_read_kb":381190,"num_write":3542,"num_write_kb":109337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218726400,"data_stored":218661562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3491,"ondisk_log_size":3491,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":363,"num_write":5266740,"num_write_kb":65270508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338431,"ondisk_log_size":338431,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":143059102,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":553,"num_read_kb":447893,"num_write":4271,"num_write_kb":143037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286179328,"data_stored":286117605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4237,"ondisk_log_size":4237,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525528,"kb_used_data":21200,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":93846312,"statfs":{"total":96636764160,"available":96098623488,"internally_reserved":0,"allocated":21708800,"data_stored":21468517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516422085},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848624,"kb_used_data":66920,"kb_used_omap":1,"kb_used_meta":781694,"kb_avail":93523216,"statfs":{"total":96636764160,"available":95767773184,"internally_reserved":0,"allocated":68526080,"data_stored":68281767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":800455114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574352,"kb_used_data":74184,"kb_used_omap":1,"kb_used_meta":500158,"kb_avail":93797488,"statfs":{"total":96636764160,"available":96048627712,"internally_reserved":0,"allocated":75964416,"data_stored":75718280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":512162246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789860,"kb_used_data":45596,"kb_used_omap":1,"kb_used_meta":744254,"kb_avail":93581980,"statfs":{"total":96636764160,"available":95827947520,"internally_reserved":0,"allocated":46690304,"data_stored":46445504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":762116549},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882668,"kb_used_data":82468,"kb_used_omap":1,"kb_used_meta":800190,"kb_avail":93489172,"statfs":{"total":96636764160,"available":95732912128,"internally_reserved":0,"allocated":84447232,"data_stored":84193910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":819395014},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900648,"kb_used_data":78560,"kb_used_omap":1,"kb_used_meta":822078,"kb_avail":93471192,"statfs":{"total":96636764160,"available":95714500608,"internally_reserved":0,"allocated":80445440,"data_stored":80190170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":841808327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767416,"kb_used_data":86832,"kb_used_omap":1,"kb_used_meta":680574,"kb_avail":93604424,"statfs":{"total":96636764160,"available":95850930176,"internally_reserved":0,"allocated":88915968,"data_stored":88663274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":696908229},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740368,"kb_used_data":41992,"kb_used_omap":1,"kb_used_meta":698366,"kb_avail":93631472,"statfs":{"total":96636764160,"available":95878627328,"internally_reserved":0,"allocated":42999808,"data_stored":42753566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715127237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29372416,"data_stored":29360766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":38112011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12673024,"data_stored":12671093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33652736,"data_stored":33643285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29364224,"data_stored":29360218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":46145536,"data_stored":46138016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50700288,"data_stored":50694746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8585216,"data_stored":8578005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46342144,"data_stored":46327365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37847040,"data_stored":37836920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16875520,"data_stored":16865968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:10:17.782 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T22:10:17.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:10:17.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 Processor -- start 2024-04-26T22:10:17.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 -- start start 2024-04-26T22:10:17.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f28d0154160 0x7f28d0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28d00c6e00 0x7f28d00c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f28d015a6e0 con 0x7f28d0154160 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f28d00c9100 con 0x7f28d00c6e00 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.912+0000 7f28d5e93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f28d00c8df0 con 0x7f28d00ca5a0 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f28d0154160 0x7f28d0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f28d0154160 0x7f28d0156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51550/0 (socket says 172.21.15.135:51550) 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2684279110 learned_addr learned my addr 172.21.15.135:0/2684279110 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cffff640 1 --2- 172.21.15.135:0/2684279110 >> v2:172.21.15.139:3300/0 conn(0x7f28d00c6e00 0x7f28d00c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2684279110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 msgr2=0x7f28d015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28c7fff640 1 --2- 172.21.15.135:0/2684279110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 --2- 172.21.15.135:0/2684279110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2684279110 >> v2:172.21.15.139:3300/0 conn(0x7f28d00c6e00 msgr2=0x7f28d00c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 --2- 172.21.15.135:0/2684279110 >> v2:172.21.15.139:3300/0 conn(0x7f28d00c6e00 0x7f28d00c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.913+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2684279110 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d0154540 con 0x7f28d0154160 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28cf7fe640 1 --2- 172.21.15.135:0/2684279110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d0154160 0x7f28d0156bb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f28d00aeb90 tx=0x7f28c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e1691621f0627e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28ceffd640 1 -- 172.21.15.135:0/2684279110 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0037070 con 0x7f28d0154160 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28ceffd640 1 -- 172.21.15.135:0/2684279110 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c0032440 con 0x7f28d0154160 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28ceffd640 1 -- 172.21.15.135:0/2684279110 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0032810 con 0x7f28d0154160 2024-04-26T22:10:17.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 -- 172.21.15.135:0/2684279110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d0154160 msgr2=0x7f28d0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2684279110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d0154160 0x7f28d0156bb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f28d00aeb90 tx=0x7f28c002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 -- 172.21.15.135:0/2684279110 shutdown_connections 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2684279110 >> v2:172.21.15.139:3300/0 conn(0x7f28d00c6e00 0x7f28d00c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2684279110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2684279110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d0154160 0x7f28d0156bb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 -- 172.21.15.135:0/2684279110 >> 172.21.15.135:0/2684279110 conn(0x7f28d00ae010 msgr2=0x7f28d00ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 -- 172.21.15.135:0/2684279110 shutdown_connections 2024-04-26T22:10:17.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.914+0000 7f28d5e93640 1 -- 172.21.15.135:0/2684279110 wait complete. 2024-04-26T22:10:17.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.915+0000 7f28d5e93640 1 Processor -- start 2024-04-26T22:10:17.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.915+0000 7f28d5e93640 1 -- start start 2024-04-26T22:10:17.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28d5e93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 0x7f28d0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28d5e93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28d5e93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d0154160 0x7f28d014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28d5e93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28d015aed0 con 0x7f28d00c6e00 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28d5e93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28d015abf0 con 0x7f28d0154160 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28d5e93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f28d00c9590 con 0x7f28d00ca5a0 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 0x7f28d0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 0x7f28d0150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51556/0 (socket says 172.21.15.135:51556) 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2661408922 learned_addr learned my addr 172.21.15.135:0/2661408922 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cffff640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d0154160 0x7f28d014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28c7fff640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 msgr2=0x7f28d014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d0154160 msgr2=0x7f28d014acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d0154160 0x7f28d014acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.916+0000 7f28cf7fe640 1 -- 172.21.15.135:0/2661408922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d0147560 con 0x7f28d00c6e00 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.917+0000 7f28cf7fe640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 0x7f28d0150350 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f28c0000c00 tx=0x7f28c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.917+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0041070 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.917+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c003c870 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.917+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d014ba50 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.917+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0037040 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.917+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d014c010 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.918+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d0076650 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.918+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28c0044070 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.919+0000 7f28ccff9640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f28b006c8f0 0x7f28b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.919+0000 7f28c7fff640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f28b006c8f0 0x7f28b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.919+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f28c00ad040 con 0x7f28d00c6e00 2024-04-26T22:10:17.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.919+0000 7f28c7fff640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f28b006c8f0 0x7f28b006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28d0149fb0 tx=0x7f28b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:17.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:17.923+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28c007adb0 con 0x7f28d00c6e00 2024-04-26T22:10:18.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.097+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28d00507a0 con 0x7f28b006c8f0 2024-04-26T22:10:18.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.107+0000 7f28ccff9640 1 -- 172.21.15.135:0/2661408922 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494622 (secure 0 0 0) 0x7f28d00507a0 con 0x7f28b006c8f0 2024-04-26T22:10:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:18.116 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:10:18.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f28b006c8f0 msgr2=0x7f28b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f28b006c8f0 0x7f28b006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28d0149fb0 tx=0x7f28b8008040 comp rx=0 tx=0).stop 2024-04-26T22:10:18.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 msgr2=0x7f28d0150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 0x7f28d0150350 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f28c0000c00 tx=0x7f28c00046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 shutdown_connections 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f28b006c8f0 0x7f28b006edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28d0154160 0x7f28d014acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28d00ca5a0 0x7f28d014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 --2- 172.21.15.135:0/2661408922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28d00c6e00 0x7f28d0150350 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 >> 172.21.15.135:0/2661408922 conn(0x7f28d00ae010 msgr2=0x7f28d00ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.111+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 shutdown_connections 2024-04-26T22:10:18.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.112+0000 7f28d5e93640 1 -- 172.21.15.135:0/2661408922 wait complete. 2024-04-26T22:10:18.122 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-04-26T22:10:17.308801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252980251,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":3350,"num_read_kb":829546,"num_write":9253298,"num_write_kb":113906494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610755,"ondisk_log_size":610755,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6029464,"kb_used_data":497752,"kb_used_omap":12,"kb_used_meta":5531635,"kb_avail":748945256,"statfs":{"total":773094113280,"available":766919942144,"internally_reserved":0,"allocated":509698048,"data_stored":507714988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":5664394801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20190116710,"num_objects":-5212,"num_object_clones":0,"num_object_copies":-10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5212,"num_whiteouts":0,"num_read":581,"num_read_kb":84526,"num_write":5421,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003200"},"pg_stats":[{"pgid":"5.18","version":"40'4026","reported_seq":4100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.983679+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:10:11.983679+0000","last_peered":"2024-04-26T22:10:11.983679+0000","last_clean":"2024-04-26T22:10:11.983679+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:10:11.983679+0000","last_undegraded":"2024-04-26T22:10:11.983679+0000","last_fullsized":"2024-04-26T22:10:11.983679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":60598,"num_write_kb":740487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"39'27","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.941276+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:10:00.941276+0000","last_peered":"2024-04-26T22:10:00.941276+0000","last_clean":"2024-04-26T22:10:00.941276+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:10:00.941276+0000","last_undegraded":"2024-04-26T22:10:00.941276+0000","last_fullsized":"2024-04-26T22:10:00.941276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":16408,"num_write":20,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'5701","reported_seq":5758,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.764348+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:10:05.764348+0000","last_peered":"2024-04-26T22:10:05.764348+0000","last_clean":"2024-04-26T22:10:05.764348+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:10:05.764348+0000","last_undegraded":"2024-04-26T22:10:05.764348+0000","last_fullsized":"2024-04-26T22:10:05.764348+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86528,"num_write_kb":1106988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"40'19","reported_seq":69,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.727806+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:10:10.727806+0000","last_peered":"2024-04-26T22:10:10.727806+0000","last_clean":"2024-04-26T22:10:10.727806+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:10:10.727806+0000","last_undegraded":"2024-04-26T22:10:10.727806+0000","last_fullsized":"2024-04-26T22:10:10.727806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189327,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"40'4869","reported_seq":4957,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988095+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:10:11.988095+0000","last_peered":"2024-04-26T22:10:11.988095+0000","last_clean":"2024-04-26T22:10:11.988095+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:10:11.988095+0000","last_undegraded":"2024-04-26T22:10:11.988095+0000","last_fullsized":"2024-04-26T22:10:11.988095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":73297,"num_write_kb":899566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"40'20","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857789+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:10:05.857789+0000","last_peered":"2024-04-26T22:10:05.857789+0000","last_clean":"2024-04-26T22:10:05.857789+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:10:05.857789+0000","last_undegraded":"2024-04-26T22:10:05.857789+0000","last_fullsized":"2024-04-26T22:10:05.857789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"40'6511","reported_seq":6573,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848823+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:10:05.848823+0000","last_peered":"2024-04-26T22:10:05.848823+0000","last_clean":"2024-04-26T22:10:05.848823+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:10:05.848823+0000","last_undegraded":"2024-04-26T22:10:05.848823+0000","last_fullsized":"2024-04-26T22:10:05.848823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6511,"log_dups_size":0,"ondisk_log_size":6511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":98412,"num_write_kb":1227645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"40'132","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.904919+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:10:11.904919+0000","last_peered":"2024-04-26T22:10:11.904919+0000","last_clean":"2024-04-26T22:10:11.904919+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:10:11.904919+0000","last_undegraded":"2024-04-26T22:10:11.904919+0000","last_fullsized":"2024-04-26T22:10:11.904919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":127,"num_write_kb":8276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"40'3508","reported_seq":3574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.949165+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:10:11.949165+0000","last_peered":"2024-04-26T22:10:11.949165+0000","last_clean":"2024-04-26T22:10:11.949165+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:10:11.949165+0000","last_undegraded":"2024-04-26T22:10:11.949165+0000","last_fullsized":"2024-04-26T22:10:11.949165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3508,"log_dups_size":0,"ondisk_log_size":3508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":52619,"num_write_kb":632470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"40'116","reported_seq":197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.110609+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:10:10.110609+0000","last_peered":"2024-04-26T22:10:10.110609+0000","last_clean":"2024-04-26T22:10:10.110609+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:10:10.110609+0000","last_undegraded":"2024-04-26T22:10:10.110609+0000","last_fullsized":"2024-04-26T22:10:10.110609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":45092,"num_write":102,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"40'3918","reported_seq":3975,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.754455+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:10:05.754455+0000","last_peered":"2024-04-26T22:10:05.754455+0000","last_clean":"2024-04-26T22:10:05.754455+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:10:05.754455+0000","last_undegraded":"2024-04-26T22:10:05.754455+0000","last_fullsized":"2024-04-26T22:10:05.754455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":59308,"num_write_kb":714963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"40'1268","reported_seq":1325,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.959833+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:10:00.959833+0000","last_peered":"2024-04-26T22:10:00.959833+0000","last_clean":"2024-04-26T22:10:00.959833+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:10:00.959833+0000","last_undegraded":"2024-04-26T22:10:00.959833+0000","last_fullsized":"2024-04-26T22:10:00.959833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1268,"log_dups_size":0,"ondisk_log_size":1268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":1265,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'3315","reported_seq":3384,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.932922+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:10:11.932922+0000","last_peered":"2024-04-26T22:10:11.932922+0000","last_clean":"2024-04-26T22:10:11.932922+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:10:11.932922+0000","last_undegraded":"2024-04-26T22:10:11.932922+0000","last_fullsized":"2024-04-26T22:10:11.932922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3315,"log_dups_size":0,"ondisk_log_size":3315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":49749,"num_write_kb":591259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"40'20","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.954736+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:10:01.954736+0000","last_peered":"2024-04-26T22:10:01.954736+0000","last_clean":"2024-04-26T22:10:01.954736+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:10:01.954736+0000","last_undegraded":"2024-04-26T22:10:01.954736+0000","last_fullsized":"2024-04-26T22:10:01.954736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":18,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"40'6620","reported_seq":6664,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.806952+0000","last_change":"2024-04-26T22:00:39.151257+0000","last_active":"2024-04-26T22:10:05.806952+0000","last_peered":"2024-04-26T22:10:05.806952+0000","last_clean":"2024-04-26T22:10:05.806952+0000","last_became_active":"2024-04-26T22:00:39.151025+0000","last_became_peered":"2024-04-26T22:00:39.151025+0000","last_unstale":"2024-04-26T22:10:05.806952+0000","last_undegraded":"2024-04-26T22:10:05.806952+0000","last_fullsized":"2024-04-26T22:10:05.806952+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:06.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":100009,"num_write_kb":1224131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"40'41","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955354+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:10:11.955354+0000","last_peered":"2024-04-26T22:10:11.955354+0000","last_clean":"2024-04-26T22:10:11.955354+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:10:11.955354+0000","last_undegraded":"2024-04-26T22:10:11.955354+0000","last_fullsized":"2024-04-26T22:10:11.955354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":30,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"40'4367","reported_seq":4447,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.965935+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:10:11.965935+0000","last_peered":"2024-04-26T22:10:11.965935+0000","last_clean":"2024-04-26T22:10:11.965935+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:10:11.965935+0000","last_undegraded":"2024-04-26T22:10:11.965935+0000","last_fullsized":"2024-04-26T22:10:11.965935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65356,"num_write_kb":790948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"40'4740","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.661582+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:10:05.661582+0000","last_peered":"2024-04-26T22:10:05.661582+0000","last_clean":"2024-04-26T22:10:05.661582+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:10:05.661582+0000","last_undegraded":"2024-04-26T22:10:05.661582+0000","last_fullsized":"2024-04-26T22:10:05.661582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":71515,"num_write_kb":926848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"40'206","reported_seq":280,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.865453+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:10:05.865453+0000","last_peered":"2024-04-26T22:10:05.865453+0000","last_clean":"2024-04-26T22:10:05.865453+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:10:05.865453+0000","last_undegraded":"2024-04-26T22:10:05.865453+0000","last_fullsized":"2024-04-26T22:10:05.865453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":29169,"num_write":192,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"40'39","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.495965+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:10:01.495965+0000","last_peered":"2024-04-26T22:10:01.495965+0000","last_clean":"2024-04-26T22:10:01.495965+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:10:01.495965+0000","last_undegraded":"2024-04-26T22:10:01.495965+0000","last_fullsized":"2024-04-26T22:10:01.495965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":13991,"num_write":36,"num_write_kb":4124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'3984","reported_seq":4061,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986082+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:10:11.986082+0000","last_peered":"2024-04-26T22:10:11.986082+0000","last_clean":"2024-04-26T22:10:11.986082+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:10:11.986082+0000","last_undegraded":"2024-04-26T22:10:11.986082+0000","last_fullsized":"2024-04-26T22:10:11.986082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59797,"num_write_kb":746756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"40'4602","reported_seq":4661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852754+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:10:05.852754+0000","last_peered":"2024-04-26T22:10:05.852754+0000","last_clean":"2024-04-26T22:10:05.852754+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:10:05.852754+0000","last_undegraded":"2024-04-26T22:10:05.852754+0000","last_fullsized":"2024-04-26T22:10:05.852754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69468,"num_write_kb":843448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"40'24","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.828651+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:10:05.828651+0000","last_peered":"2024-04-26T22:10:05.828651+0000","last_clean":"2024-04-26T22:10:05.828651+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:10:05.828651+0000","last_undegraded":"2024-04-26T22:10:05.828651+0000","last_fullsized":"2024-04-26T22:10:05.828651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":20,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937503+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:10:11.937503+0000","last_peered":"2024-04-26T22:10:11.937503+0000","last_clean":"2024-04-26T22:10:11.937503+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:10:11.937503+0000","last_undegraded":"2024-04-26T22:10:11.937503+0000","last_fullsized":"2024-04-26T22:10:11.937503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"40'3976","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.916627+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:10:11.916627+0000","last_peered":"2024-04-26T22:10:11.916627+0000","last_clean":"2024-04-26T22:10:11.916627+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:10:11.916627+0000","last_undegraded":"2024-04-26T22:10:11.916627+0000","last_fullsized":"2024-04-26T22:10:11.916627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3976,"log_dups_size":0,"ondisk_log_size":3976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59917,"num_write_kb":706723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"40'5423","reported_seq":5480,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.761465+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:10:05.761465+0000","last_peered":"2024-04-26T22:10:05.761465+0000","last_clean":"2024-04-26T22:10:05.761465+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:10:05.761465+0000","last_undegraded":"2024-04-26T22:10:05.761465+0000","last_fullsized":"2024-04-26T22:10:05.761465+0000","mapping_epoch":18,"log_start":"28'200","ondisk_log_start":"28'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":81931,"num_write_kb":1035269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'44","reported_seq":104,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.415234+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:10:14.415234+0000","last_peered":"2024-04-26T22:10:14.415234+0000","last_clean":"2024-04-26T22:10:14.415234+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:10:14.415234+0000","last_undegraded":"2024-04-26T22:10:14.415234+0000","last_fullsized":"2024-04-26T22:10:14.415234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363115,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":324,"num_write":38,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"40'85","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.973183+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:10:11.973183+0000","last_peered":"2024-04-26T22:10:11.973183+0000","last_clean":"2024-04-26T22:10:11.973183+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:10:11.973183+0000","last_undegraded":"2024-04-26T22:10:11.973183+0000","last_fullsized":"2024-04-26T22:10:11.973183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:45.178164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":6841,"num_write":81,"num_write_kb":4168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"40'4751","reported_seq":4829,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937603+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:10:11.937603+0000","last_peered":"2024-04-26T22:10:11.937603+0000","last_clean":"2024-04-26T22:10:11.937603+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:10:11.937603+0000","last_undegraded":"2024-04-26T22:10:11.937603+0000","last_fullsized":"2024-04-26T22:10:11.937603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":71494,"num_write_kb":858502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"40'5174","reported_seq":5233,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.821492+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:10:05.821492+0000","last_peered":"2024-04-26T22:10:05.821492+0000","last_clean":"2024-04-26T22:10:05.821492+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:10:05.821492+0000","last_undegraded":"2024-04-26T22:10:05.821492+0000","last_fullsized":"2024-04-26T22:10:05.821492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":78246,"num_write_kb":941647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"40'8","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823374+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:10:05.823374+0000","last_peered":"2024-04-26T22:10:05.823374+0000","last_clean":"2024-04-26T22:10:05.823374+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:10:05.823374+0000","last_undegraded":"2024-04-26T22:10:05.823374+0000","last_fullsized":"2024-04-26T22:10:05.823374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":4,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"40'37","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962744+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:10:11.962744+0000","last_peered":"2024-04-26T22:10:11.962744+0000","last_clean":"2024-04-26T22:10:11.962744+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:10:11.962744+0000","last_undegraded":"2024-04-26T22:10:11.962744+0000","last_fullsized":"2024-04-26T22:10:11.962744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":31,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"40'3913","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962293+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:10:11.962293+0000","last_peered":"2024-04-26T22:10:11.962293+0000","last_clean":"2024-04-26T22:10:11.962293+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:10:11.962293+0000","last_undegraded":"2024-04-26T22:10:11.962293+0000","last_fullsized":"2024-04-26T22:10:11.962293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":58791,"num_write_kb":740377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'4599","reported_seq":4657,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.774768+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:10:05.774768+0000","last_peered":"2024-04-26T22:10:05.774768+0000","last_clean":"2024-04-26T22:10:05.774768+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:10:05.774768+0000","last_undegraded":"2024-04-26T22:10:05.774768+0000","last_fullsized":"2024-04-26T22:10:05.774768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69140,"num_write_kb":847801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"40'8","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836838+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:10:05.836838+0000","last_peered":"2024-04-26T22:10:05.836838+0000","last_clean":"2024-04-26T22:10:05.836838+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:10:05.836838+0000","last_undegraded":"2024-04-26T22:10:05.836838+0000","last_fullsized":"2024-04-26T22:10:05.836838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"40'23","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.979062+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:10:14.979062+0000","last_peered":"2024-04-26T22:10:14.979062+0000","last_clean":"2024-04-26T22:10:14.979062+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:10:14.979062+0000","last_undegraded":"2024-04-26T22:10:14.979062+0000","last_fullsized":"2024-04-26T22:10:14.979062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":98,"num_write":20,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"40'3929","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.938066+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:10:11.938066+0000","last_peered":"2024-04-26T22:10:11.938066+0000","last_clean":"2024-04-26T22:10:11.938066+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:10:11.938066+0000","last_undegraded":"2024-04-26T22:10:11.938066+0000","last_fullsized":"2024-04-26T22:10:11.938066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":59023,"num_write_kb":718389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'5311","reported_seq":5368,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.831429+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:10:05.831429+0000","last_peered":"2024-04-26T22:10:05.831429+0000","last_clean":"2024-04-26T22:10:05.831429+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:10:05.831429+0000","last_undegraded":"2024-04-26T22:10:05.831429+0000","last_fullsized":"2024-04-26T22:10:05.831429+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80379,"num_write_kb":1004221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'218","reported_seq":282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.885693+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:10:14.885693+0000","last_peered":"2024-04-26T22:10:14.885693+0000","last_clean":"2024-04-26T22:10:14.885693+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:10:14.885693+0000","last_undegraded":"2024-04-26T22:10:14.885693+0000","last_fullsized":"2024-04-26T22:10:14.885693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":61,"num_write":416,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"40'22","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.961590+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:10:11.961590+0000","last_peered":"2024-04-26T22:10:11.961590+0000","last_clean":"2024-04-26T22:10:11.961590+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:10:11.961590+0000","last_undegraded":"2024-04-26T22:10:11.961590+0000","last_fullsized":"2024-04-26T22:10:11.961590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":15,"num_write_kb":8199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"40'4334","reported_seq":4409,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.892208+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:10:11.892208+0000","last_peered":"2024-04-26T22:10:11.892208+0000","last_clean":"2024-04-26T22:10:11.892208+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:10:11.892208+0000","last_undegraded":"2024-04-26T22:10:11.892208+0000","last_fullsized":"2024-04-26T22:10:11.892208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":65365,"num_write_kb":782662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"40'5496","reported_seq":5556,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.837207+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:10:05.837207+0000","last_peered":"2024-04-26T22:10:05.837207+0000","last_clean":"2024-04-26T22:10:05.837207+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:10:05.837207+0000","last_undegraded":"2024-04-26T22:10:05.837207+0000","last_fullsized":"2024-04-26T22:10:05.837207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":82892,"num_write_kb":1050745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'35","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.834069+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:10:05.834069+0000","last_peered":"2024-04-26T22:10:05.834069+0000","last_clean":"2024-04-26T22:10:05.834069+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:10:05.834069+0000","last_undegraded":"2024-04-26T22:10:05.834069+0000","last_fullsized":"2024-04-26T22:10:05.834069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":29,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"28'1","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940808+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:10:00.940808+0000","last_peered":"2024-04-26T22:10:00.940808+0000","last_clean":"2024-04-26T22:10:00.940808+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:10:00.940808+0000","last_undegraded":"2024-04-26T22:10:00.940808+0000","last_fullsized":"2024-04-26T22:10:00.940808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"40'3789","reported_seq":3862,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937218+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:10:11.937218+0000","last_peered":"2024-04-26T22:10:11.937218+0000","last_clean":"2024-04-26T22:10:11.937218+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:10:11.937218+0000","last_undegraded":"2024-04-26T22:10:11.937218+0000","last_fullsized":"2024-04-26T22:10:11.937218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":56913,"num_write_kb":707433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"40'4759","reported_seq":4817,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803693+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:10:05.803693+0000","last_peered":"2024-04-26T22:10:05.803693+0000","last_clean":"2024-04-26T22:10:05.803693+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:10:05.803693+0000","last_undegraded":"2024-04-26T22:10:05.803693+0000","last_fullsized":"2024-04-26T22:10:05.803693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71808,"num_write_kb":921006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"40'7","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.825804+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:10:05.825804+0000","last_peered":"2024-04-26T22:10:05.825804+0000","last_clean":"2024-04-26T22:10:05.825804+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:10:05.825804+0000","last_undegraded":"2024-04-26T22:10:05.825804+0000","last_fullsized":"2024-04-26T22:10:05.825804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"40'183","reported_seq":241,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966669+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:10:11.966669+0000","last_peered":"2024-04-26T22:10:11.966669+0000","last_clean":"2024-04-26T22:10:11.966669+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:10:11.966669+0000","last_undegraded":"2024-04-26T22:10:11.966669+0000","last_fullsized":"2024-04-26T22:10:11.966669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":27847,"num_write":173,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"40'3541","reported_seq":3610,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940587+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:10:11.940587+0000","last_peered":"2024-04-26T22:10:11.940587+0000","last_clean":"2024-04-26T22:10:11.940587+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:10:11.940587+0000","last_undegraded":"2024-04-26T22:10:11.940587+0000","last_fullsized":"2024-04-26T22:10:11.940587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":53073,"num_write_kb":661602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'5106","reported_seq":5164,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.813565+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:10:05.813565+0000","last_peered":"2024-04-26T22:10:05.813565+0000","last_clean":"2024-04-26T22:10:05.813565+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:10:05.813565+0000","last_undegraded":"2024-04-26T22:10:05.813565+0000","last_fullsized":"2024-04-26T22:10:05.813565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":77293,"num_write_kb":982493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"40'221","reported_seq":286,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850755+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:10:05.850755+0000","last_peered":"2024-04-26T22:10:05.850755+0000","last_clean":"2024-04-26T22:10:05.850755+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:10:05.850755+0000","last_undegraded":"2024-04-26T22:10:05.850755+0000","last_fullsized":"2024-04-26T22:10:05.850755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":213,"num_write_kb":8367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"40'5","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.945512+0000","last_change":"2024-04-26T22:00:05.815435+0000","last_active":"2024-04-26T22:10:11.945512+0000","last_peered":"2024-04-26T22:10:11.945512+0000","last_clean":"2024-04-26T22:10:11.945512+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:10:11.945512+0000","last_undegraded":"2024-04-26T22:10:11.945512+0000","last_fullsized":"2024-04-26T22:10:11.945512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"40'3541","reported_seq":3614,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.910214+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:10:11.910214+0000","last_peered":"2024-04-26T22:10:11.910214+0000","last_clean":"2024-04-26T22:10:11.910214+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:10:11.910214+0000","last_undegraded":"2024-04-26T22:10:11.910214+0000","last_fullsized":"2024-04-26T22:10:11.910214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":52805,"num_write_kb":629973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'5114","reported_seq":5174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857162+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:10:05.857162+0000","last_peered":"2024-04-26T22:10:05.857162+0000","last_clean":"2024-04-26T22:10:05.857162+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:10:05.857162+0000","last_undegraded":"2024-04-26T22:10:05.857162+0000","last_fullsized":"2024-04-26T22:10:05.857162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":77095,"num_write_kb":967489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"40'19","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.889177+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:10:14.889177+0000","last_peered":"2024-04-26T22:10:14.889177+0000","last_clean":"2024-04-26T22:10:14.889177+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:10:14.889177+0000","last_undegraded":"2024-04-26T22:10:14.889177+0000","last_fullsized":"2024-04-26T22:10:14.889177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88181,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":134,"num_write":15,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"40'5","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954456+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:10:11.954456+0000","last_peered":"2024-04-26T22:10:11.954456+0000","last_clean":"2024-04-26T22:10:11.954456+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:10:11.954456+0000","last_undegraded":"2024-04-26T22:10:11.954456+0000","last_fullsized":"2024-04-26T22:10:11.954456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:15.036356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'4139","reported_seq":4216,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.947123+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:10:11.947123+0000","last_peered":"2024-04-26T22:10:11.947123+0000","last_clean":"2024-04-26T22:10:11.947123+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:10:11.947123+0000","last_undegraded":"2024-04-26T22:10:11.947123+0000","last_fullsized":"2024-04-26T22:10:11.947123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":62199,"num_write_kb":728313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"40'5239","reported_seq":5297,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810597+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:10:05.810597+0000","last_peered":"2024-04-26T22:10:05.810597+0000","last_clean":"2024-04-26T22:10:05.810597+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:10:05.810597+0000","last_undegraded":"2024-04-26T22:10:05.810597+0000","last_fullsized":"2024-04-26T22:10:05.810597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":79195,"num_write_kb":994121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"40'30","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.863131+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:10:05.863131+0000","last_peered":"2024-04-26T22:10:05.863131+0000","last_clean":"2024-04-26T22:10:05.863131+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:10:05.863131+0000","last_undegraded":"2024-04-26T22:10:05.863131+0000","last_fullsized":"2024-04-26T22:10:05.863131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":8201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"40'211","reported_seq":268,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.956625+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:10:11.956625+0000","last_peered":"2024-04-26T22:10:11.956625+0000","last_clean":"2024-04-26T22:10:11.956625+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:10:11.956625+0000","last_undegraded":"2024-04-26T22:10:11.956625+0000","last_fullsized":"2024-04-26T22:10:11.956625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":74,"num_write":424,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"40'4145","reported_seq":4221,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.957211+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:10:11.957211+0000","last_peered":"2024-04-26T22:10:11.957211+0000","last_clean":"2024-04-26T22:10:11.957211+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:10:11.957211+0000","last_undegraded":"2024-04-26T22:10:11.957211+0000","last_fullsized":"2024-04-26T22:10:11.957211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":62372,"num_write_kb":775201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"40'5748","reported_seq":5805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.762804+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:10:05.762804+0000","last_peered":"2024-04-26T22:10:05.762804+0000","last_clean":"2024-04-26T22:10:05.762804+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:10:05.762804+0000","last_undegraded":"2024-04-26T22:10:05.762804+0000","last_fullsized":"2024-04-26T22:10:05.762804+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":87080,"num_write_kb":1065746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"39'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940251+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:10:00.940251+0000","last_peered":"2024-04-26T22:10:00.940251+0000","last_clean":"2024-04-26T22:10:00.940251+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:10:00.940251+0000","last_undegraded":"2024-04-26T22:10:00.940251+0000","last_fullsized":"2024-04-26T22:10:00.940251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.221589+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:10:01.221589+0000","last_peered":"2024-04-26T22:10:01.221589+0000","last_clean":"2024-04-26T22:10:01.221589+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:10:01.221589+0000","last_undegraded":"2024-04-26T22:10:01.221589+0000","last_fullsized":"2024-04-26T22:10:01.221589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.944318+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:10:11.944318+0000","last_peered":"2024-04-26T22:10:11.944318+0000","last_clean":"2024-04-26T22:10:11.944318+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:10:11.944318+0000","last_undegraded":"2024-04-26T22:10:11.944318+0000","last_fullsized":"2024-04-26T22:10:11.944318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"40'3379","reported_seq":3460,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959450+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:10:11.959450+0000","last_peered":"2024-04-26T22:10:11.959450+0000","last_clean":"2024-04-26T22:10:11.959450+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:10:11.959450+0000","last_undegraded":"2024-04-26T22:10:11.959450+0000","last_fullsized":"2024-04-26T22:10:11.959450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":50792,"num_write_kb":610820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"40'5433","reported_seq":5494,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.869142+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:10:05.869142+0000","last_peered":"2024-04-26T22:10:05.869142+0000","last_clean":"2024-04-26T22:10:05.869142+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:10:05.869142+0000","last_undegraded":"2024-04-26T22:10:05.869142+0000","last_fullsized":"2024-04-26T22:10:05.869142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":81848,"num_write_kb":1014508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"40'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810386+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:10:05.810386+0000","last_peered":"2024-04-26T22:10:05.810386+0000","last_clean":"2024-04-26T22:10:05.810386+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:10:05.810386+0000","last_undegraded":"2024-04-26T22:10:05.810386+0000","last_fullsized":"2024-04-26T22:10:05.810386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"40'201","reported_seq":252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.948751+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:10:11.948751+0000","last_peered":"2024-04-26T22:10:11.948751+0000","last_clean":"2024-04-26T22:10:11.948751+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:10:11.948751+0000","last_undegraded":"2024-04-26T22:10:11.948751+0000","last_fullsized":"2024-04-26T22:10:11.948751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":196,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"40'3780","reported_seq":3857,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.815165+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:10:11.815165+0000","last_peered":"2024-04-26T22:10:11.815165+0000","last_clean":"2024-04-26T22:10:11.815165+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:10:11.815165+0000","last_undegraded":"2024-04-26T22:10:11.815165+0000","last_fullsized":"2024-04-26T22:10:11.815165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":56837,"num_write_kb":699720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"40'5733","reported_seq":5794,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849434+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:10:05.849434+0000","last_peered":"2024-04-26T22:10:05.849434+0000","last_clean":"2024-04-26T22:10:05.849434+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:10:05.849434+0000","last_undegraded":"2024-04-26T22:10:05.849434+0000","last_fullsized":"2024-04-26T22:10:05.849434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5733,"log_dups_size":0,"ondisk_log_size":5733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":86585,"num_write_kb":1068440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"40'19","reported_seq":88,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.943192+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:10:00.943192+0000","last_peered":"2024-04-26T22:10:00.943192+0000","last_clean":"2024-04-26T22:10:00.943192+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:10:00.943192+0000","last_undegraded":"2024-04-26T22:10:00.943192+0000","last_fullsized":"2024-04-26T22:10:00.943192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16390,"num_write":17,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"40'379","reported_seq":446,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955307+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:10:11.955307+0000","last_peered":"2024-04-26T22:10:11.955307+0000","last_clean":"2024-04-26T22:10:11.955307+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:10:11.955307+0000","last_undegraded":"2024-04-26T22:10:11.955307+0000","last_fullsized":"2024-04-26T22:10:11.955307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":25852,"num_write":364,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"40'3909","reported_seq":3987,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946493+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:10:11.946493+0000","last_peered":"2024-04-26T22:10:11.946493+0000","last_clean":"2024-04-26T22:10:11.946493+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:10:11.946493+0000","last_undegraded":"2024-04-26T22:10:11.946493+0000","last_fullsized":"2024-04-26T22:10:11.946493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":58819,"num_write_kb":699051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"40'6248","reported_seq":6306,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823315+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:10:05.823315+0000","last_peered":"2024-04-26T22:10:05.823315+0000","last_clean":"2024-04-26T22:10:05.823315+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:10:05.823315+0000","last_undegraded":"2024-04-26T22:10:05.823315+0000","last_fullsized":"2024-04-26T22:10:05.823315+0000","mapping_epoch":18,"log_start":"28'400","ondisk_log_start":"28'400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":94569,"num_write_kb":1160134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"40'111","reported_seq":183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850721+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:10:05.850721+0000","last_peered":"2024-04-26T22:10:05.850721+0000","last_clean":"2024-04-26T22:10:05.850721+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:10:05.850721+0000","last_undegraded":"2024-04-26T22:10:05.850721+0000","last_fullsized":"2024-04-26T22:10:05.850721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":106,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"40'13","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.935130+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:10:11.935130+0000","last_peered":"2024-04-26T22:10:11.935130+0000","last_clean":"2024-04-26T22:10:11.935130+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:10:11.935130+0000","last_undegraded":"2024-04-26T22:10:11.935130+0000","last_fullsized":"2024-04-26T22:10:11.935130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"40'4399","reported_seq":4473,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.897004+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:10:11.897004+0000","last_peered":"2024-04-26T22:10:11.897004+0000","last_clean":"2024-04-26T22:10:11.897004+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:10:11.897004+0000","last_undegraded":"2024-04-26T22:10:11.897004+0000","last_fullsized":"2024-04-26T22:10:11.897004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":66147,"num_write_kb":822005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"40'5224","reported_seq":5282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822320+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:10:05.822320+0000","last_peered":"2024-04-26T22:10:05.822320+0000","last_clean":"2024-04-26T22:10:05.822320+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:10:05.822320+0000","last_undegraded":"2024-04-26T22:10:05.822320+0000","last_fullsized":"2024-04-26T22:10:05.822320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":78957,"num_write_kb":961695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"40'16","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849672+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:10:05.849672+0000","last_peered":"2024-04-26T22:10:05.849672+0000","last_clean":"2024-04-26T22:10:05.849672+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:10:05.849672+0000","last_undegraded":"2024-04-26T22:10:05.849672+0000","last_fullsized":"2024-04-26T22:10:05.849672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":12294,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951413+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:10:11.951413+0000","last_peered":"2024-04-26T22:10:11.951413+0000","last_clean":"2024-04-26T22:10:11.951413+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:10:11.951413+0000","last_undegraded":"2024-04-26T22:10:11.951413+0000","last_fullsized":"2024-04-26T22:10:11.951413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"40'3562","reported_seq":3638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982357+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:10:11.982357+0000","last_peered":"2024-04-26T22:10:11.982357+0000","last_clean":"2024-04-26T22:10:11.982357+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:10:11.982357+0000","last_undegraded":"2024-04-26T22:10:11.982357+0000","last_fullsized":"2024-04-26T22:10:11.982357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3562,"log_dups_size":0,"ondisk_log_size":3562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":53350,"num_write_kb":651982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"40'5336","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848168+0000","last_change":"2024-04-26T22:00:39.151213+0000","last_active":"2024-04-26T22:10:05.848168+0000","last_peered":"2024-04-26T22:10:05.848168+0000","last_clean":"2024-04-26T22:10:05.848168+0000","last_became_active":"2024-04-26T22:00:39.150992+0000","last_became_peered":"2024-04-26T22:00:39.150992+0000","last_unstale":"2024-04-26T22:10:05.848168+0000","last_undegraded":"2024-04-26T22:10:05.848168+0000","last_fullsized":"2024-04-26T22:10:05.848168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:26.086210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":80461,"num_write_kb":1019956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"40'287","reported_seq":383,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.839145+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:10:05.839145+0000","last_peered":"2024-04-26T22:10:05.839145+0000","last_clean":"2024-04-26T22:10:05.839145+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:10:05.839145+0000","last_undegraded":"2024-04-26T22:10:05.839145+0000","last_fullsized":"2024-04-26T22:10:05.839145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":12387,"num_write":284,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"40'4","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954588+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:10:11.954588+0000","last_peered":"2024-04-26T22:10:11.954588+0000","last_clean":"2024-04-26T22:10:11.954588+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:10:11.954588+0000","last_undegraded":"2024-04-26T22:10:11.954588+0000","last_fullsized":"2024-04-26T22:10:11.954588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"40'4185","reported_seq":4270,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946111+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:10:11.946111+0000","last_peered":"2024-04-26T22:10:11.946111+0000","last_clean":"2024-04-26T22:10:11.946111+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:10:11.946111+0000","last_undegraded":"2024-04-26T22:10:11.946111+0000","last_fullsized":"2024-04-26T22:10:11.946111+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":62914,"num_write_kb":762533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'5750","reported_seq":5810,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.854474+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:10:05.854474+0000","last_peered":"2024-04-26T22:10:05.854474+0000","last_clean":"2024-04-26T22:10:05.854474+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:10:05.854474+0000","last_undegraded":"2024-04-26T22:10:05.854474+0000","last_fullsized":"2024-04-26T22:10:05.854474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":86794,"num_write_kb":1073964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"40'6","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836648+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:10:05.836648+0000","last_peered":"2024-04-26T22:10:05.836648+0000","last_clean":"2024-04-26T22:10:05.836648+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:10:05.836648+0000","last_undegraded":"2024-04-26T22:10:05.836648+0000","last_fullsized":"2024-04-26T22:10:05.836648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.943365+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:10:07.943365+0000","last_peered":"2024-04-26T22:10:07.943365+0000","last_clean":"2024-04-26T22:10:07.943365+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:10:07.943365+0000","last_undegraded":"2024-04-26T22:10:07.943365+0000","last_fullsized":"2024-04-26T22:10:07.943365+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"40'3717","reported_seq":3786,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.778102+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:10:11.778102+0000","last_peered":"2024-04-26T22:10:11.778102+0000","last_clean":"2024-04-26T22:10:11.778102+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:10:11.778102+0000","last_undegraded":"2024-04-26T22:10:11.778102+0000","last_fullsized":"2024-04-26T22:10:11.778102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3717,"log_dups_size":0,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":55831,"num_write_kb":673382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"40'6101","reported_seq":6161,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868996+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:10:05.868996+0000","last_peered":"2024-04-26T22:10:05.868996+0000","last_clean":"2024-04-26T22:10:05.868996+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:10:05.868996+0000","last_undegraded":"2024-04-26T22:10:05.868996+0000","last_fullsized":"2024-04-26T22:10:05.868996+0000","mapping_epoch":18,"log_start":"28'300","ondisk_log_start":"28'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":92157,"num_write_kb":1133252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"40'5","reported_seq":63,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867571+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:10:05.867571+0000","last_peered":"2024-04-26T22:10:05.867571+0000","last_clean":"2024-04-26T22:10:05.867571+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:10:05.867571+0000","last_undegraded":"2024-04-26T22:10:05.867571+0000","last_fullsized":"2024-04-26T22:10:05.867571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"40'374","reported_seq":467,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953183+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:10:11.953183+0000","last_peered":"2024-04-26T22:10:11.953183+0000","last_clean":"2024-04-26T22:10:11.953183+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:10:11.953183+0000","last_undegraded":"2024-04-26T22:10:11.953183+0000","last_fullsized":"2024-04-26T22:10:11.953183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":31157,"num_write":363,"num_write_kb":8442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"40'5464","reported_seq":5545,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.917366+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:10:11.917366+0000","last_peered":"2024-04-26T22:10:11.917366+0000","last_clean":"2024-04-26T22:10:11.917366+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:10:11.917366+0000","last_undegraded":"2024-04-26T22:10:11.917366+0000","last_fullsized":"2024-04-26T22:10:11.917366+0000","mapping_epoch":22,"log_start":"31'200","ondisk_log_start":"31'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":82133,"num_write_kb":980051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'5077","reported_seq":5133,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673410+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:10:05.673410+0000","last_peered":"2024-04-26T22:10:05.673410+0000","last_clean":"2024-04-26T22:10:05.673410+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:10:05.673410+0000","last_undegraded":"2024-04-26T22:10:05.673410+0000","last_fullsized":"2024-04-26T22:10:05.673410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":76618,"num_write_kb":958363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"40'19","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852434+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:10:05.852434+0000","last_peered":"2024-04-26T22:10:05.852434+0000","last_clean":"2024-04-26T22:10:05.852434+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:10:05.852434+0000","last_undegraded":"2024-04-26T22:10:05.852434+0000","last_fullsized":"2024-04-26T22:10:05.852434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":32816,"num_write":13,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"40'15","reported_seq":89,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.994445+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:10:11.994445+0000","last_peered":"2024-04-26T22:10:11.994445+0000","last_clean":"2024-04-26T22:10:11.994445+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:10:11.994445+0000","last_undegraded":"2024-04-26T22:10:11.994445+0000","last_fullsized":"2024-04-26T22:10:11.994445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"40'4714","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982824+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:10:11.982824+0000","last_peered":"2024-04-26T22:10:11.982824+0000","last_clean":"2024-04-26T22:10:11.982824+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:10:11.982824+0000","last_undegraded":"2024-04-26T22:10:11.982824+0000","last_fullsized":"2024-04-26T22:10:11.982824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":70866,"num_write_kb":843772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"40'5114","reported_seq":5170,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.579883+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:10:05.579883+0000","last_peered":"2024-04-26T22:10:05.579883+0000","last_clean":"2024-04-26T22:10:05.579883+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:10:05.579883+0000","last_undegraded":"2024-04-26T22:10:05.579883+0000","last_fullsized":"2024-04-26T22:10:05.579883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77180,"num_write_kb":958350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"40'824","reported_seq":917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848033+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:10:05.848033+0000","last_peered":"2024-04-26T22:10:05.848033+0000","last_clean":"2024-04-26T22:10:05.848033+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:10:05.848033+0000","last_undegraded":"2024-04-26T22:10:05.848033+0000","last_fullsized":"2024-04-26T22:10:05.848033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":36914,"num_write":813,"num_write_kb":13003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"40'5","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977919+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:10:11.977919+0000","last_peered":"2024-04-26T22:10:11.977919+0000","last_clean":"2024-04-26T22:10:11.977919+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:10:11.977919+0000","last_undegraded":"2024-04-26T22:10:11.977919+0000","last_fullsized":"2024-04-26T22:10:11.977919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"40'3745","reported_seq":3823,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.841511+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:10:11.841511+0000","last_peered":"2024-04-26T22:10:11.841511+0000","last_clean":"2024-04-26T22:10:11.841511+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:10:11.841511+0000","last_undegraded":"2024-04-26T22:10:11.841511+0000","last_fullsized":"2024-04-26T22:10:11.841511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56184,"num_write_kb":680067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"40'7267","reported_seq":7326,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824603+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:10:05.824603+0000","last_peered":"2024-04-26T22:10:05.824603+0000","last_clean":"2024-04-26T22:10:05.824603+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:10:05.824603+0000","last_undegraded":"2024-04-26T22:10:05.824603+0000","last_fullsized":"2024-04-26T22:10:05.824603+0000","mapping_epoch":18,"log_start":"30'1800","ondisk_log_start":"30'1800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":109924,"num_write_kb":1323267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"40'370","reported_seq":447,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.881135+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:10:05.881135+0000","last_peered":"2024-04-26T22:10:05.881135+0000","last_clean":"2024-04-26T22:10:05.881135+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:10:05.881135+0000","last_undegraded":"2024-04-26T22:10:05.881135+0000","last_fullsized":"2024-04-26T22:10:05.881135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":367,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"40'231","reported_seq":307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966124+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:10:11.966124+0000","last_peered":"2024-04-26T22:10:11.966124+0000","last_clean":"2024-04-26T22:10:11.966124+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:10:11.966124+0000","last_undegraded":"2024-04-26T22:10:11.966124+0000","last_fullsized":"2024-04-26T22:10:11.966124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":33257,"num_write":223,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"40'4111","reported_seq":4188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967266+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:10:11.967266+0000","last_peered":"2024-04-26T22:10:11.967266+0000","last_clean":"2024-04-26T22:10:11.967266+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:10:11.967266+0000","last_undegraded":"2024-04-26T22:10:11.967266+0000","last_fullsized":"2024-04-26T22:10:11.967266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4111,"log_dups_size":0,"ondisk_log_size":4111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61736,"num_write_kb":766472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"40'4628","reported_seq":4688,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867072+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:10:05.867072+0000","last_peered":"2024-04-26T22:10:05.867072+0000","last_clean":"2024-04-26T22:10:05.867072+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:10:05.867072+0000","last_undegraded":"2024-04-26T22:10:05.867072+0000","last_fullsized":"2024-04-26T22:10:05.867072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":69686,"num_write_kb":866678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"40'102","reported_seq":162,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842367+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:10:05.842367+0000","last_peered":"2024-04-26T22:10:05.842367+0000","last_clean":"2024-04-26T22:10:05.842367+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:10:05.842367+0000","last_undegraded":"2024-04-26T22:10:05.842367+0000","last_fullsized":"2024-04-26T22:10:05.842367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":98,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"40'12","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.943439+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:10:11.943439+0000","last_peered":"2024-04-26T22:10:11.943439+0000","last_clean":"2024-04-26T22:10:11.943439+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:10:11.943439+0000","last_undegraded":"2024-04-26T22:10:11.943439+0000","last_fullsized":"2024-04-26T22:10:11.943439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'4365","reported_seq":4453,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.968703+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:10:11.968703+0000","last_peered":"2024-04-26T22:10:11.968703+0000","last_clean":"2024-04-26T22:10:11.968703+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:10:11.968703+0000","last_undegraded":"2024-04-26T22:10:11.968703+0000","last_fullsized":"2024-04-26T22:10:11.968703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4365,"log_dups_size":0,"ondisk_log_size":4365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":65682,"num_write_kb":806692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"40'5549","reported_seq":5613,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824597+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:10:05.824597+0000","last_peered":"2024-04-26T22:10:05.824597+0000","last_clean":"2024-04-26T22:10:05.824597+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:10:05.824597+0000","last_undegraded":"2024-04-26T22:10:05.824597+0000","last_fullsized":"2024-04-26T22:10:05.824597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":83744,"num_write_kb":1000289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"40'4","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.846233+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:10:05.846233+0000","last_peered":"2024-04-26T22:10:05.846233+0000","last_clean":"2024-04-26T22:10:05.846233+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:10:05.846233+0000","last_undegraded":"2024-04-26T22:10:05.846233+0000","last_fullsized":"2024-04-26T22:10:05.846233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953872+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:10:11.953872+0000","last_peered":"2024-04-26T22:10:11.953872+0000","last_clean":"2024-04-26T22:10:11.953872+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:10:11.953872+0000","last_undegraded":"2024-04-26T22:10:11.953872+0000","last_fullsized":"2024-04-26T22:10:11.953872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4980","reported_seq":5062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967201+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:10:11.967201+0000","last_peered":"2024-04-26T22:10:11.967201+0000","last_clean":"2024-04-26T22:10:11.967201+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:10:11.967201+0000","last_undegraded":"2024-04-26T22:10:11.967201+0000","last_fullsized":"2024-04-26T22:10:11.967201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":74941,"num_write_kb":925745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"40'16","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.970996+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:10:11.970996+0000","last_peered":"2024-04-26T22:10:11.970996+0000","last_clean":"2024-04-26T22:10:11.970996+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:10:11.970996+0000","last_undegraded":"2024-04-26T22:10:11.970996+0000","last_fullsized":"2024-04-26T22:10:11.970996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":8,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"40'6345","reported_seq":6406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.871078+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:10:05.871078+0000","last_peered":"2024-04-26T22:10:05.871078+0000","last_clean":"2024-04-26T22:10:05.871078+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:10:05.871078+0000","last_undegraded":"2024-04-26T22:10:05.871078+0000","last_fullsized":"2024-04-26T22:10:05.871078+0000","mapping_epoch":18,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":95977,"num_write_kb":1176758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"40'19","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.772388+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:10:05.772388+0000","last_peered":"2024-04-26T22:10:05.772388+0000","last_clean":"2024-04-26T22:10:05.772388+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:10:05.772388+0000","last_undegraded":"2024-04-26T22:10:05.772388+0000","last_fullsized":"2024-04-26T22:10:05.772388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":28672,"num_write":16,"num_write_kb":8206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"40'4260","reported_seq":4329,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.876539+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:10:11.876539+0000","last_peered":"2024-04-26T22:10:11.876539+0000","last_clean":"2024-04-26T22:10:11.876539+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:10:11.876539+0000","last_undegraded":"2024-04-26T22:10:11.876539+0000","last_fullsized":"2024-04-26T22:10:11.876539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":64151,"num_write_kb":776804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'25","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.953756+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:10:00.953756+0000","last_peered":"2024-04-26T22:10:00.953756+0000","last_clean":"2024-04-26T22:10:00.953756+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:10:00.953756+0000","last_undegraded":"2024-04-26T22:10:00.953756+0000","last_fullsized":"2024-04-26T22:10:00.953756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20512,"num_write":20,"num_write_kb":4102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"40'5323","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874479+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:10:05.874479+0000","last_peered":"2024-04-26T22:10:05.874479+0000","last_clean":"2024-04-26T22:10:05.874479+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:10:05.874479+0000","last_undegraded":"2024-04-26T22:10:05.874479+0000","last_fullsized":"2024-04-26T22:10:05.874479+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":80432,"num_write_kb":1004371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"40'46","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.934549+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:10:00.934549+0000","last_peered":"2024-04-26T22:10:00.934549+0000","last_clean":"2024-04-26T22:10:00.934549+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:10:00.934549+0000","last_undegraded":"2024-04-26T22:10:00.934549+0000","last_fullsized":"2024-04-26T22:10:00.934549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":41,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'4905","reported_seq":4991,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940116+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:10:11.940116+0000","last_peered":"2024-04-26T22:10:11.940116+0000","last_clean":"2024-04-26T22:10:11.940116+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:10:11.940116+0000","last_undegraded":"2024-04-26T22:10:11.940116+0000","last_fullsized":"2024-04-26T22:10:11.940116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:16.969683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":74018,"num_write_kb":905043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"40'145","reported_seq":213,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.833871+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:10:05.833871+0000","last_peered":"2024-04-26T22:10:05.833871+0000","last_clean":"2024-04-26T22:10:05.833871+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:10:05.833871+0000","last_undegraded":"2024-04-26T22:10:05.833871+0000","last_fullsized":"2024-04-26T22:10:05.833871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":136,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"40'4250","reported_seq":4307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.802351+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:10:05.802351+0000","last_peered":"2024-04-26T22:10:05.802351+0000","last_clean":"2024-04-26T22:10:05.802351+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:10:05.802351+0000","last_undegraded":"2024-04-26T22:10:05.802351+0000","last_fullsized":"2024-04-26T22:10:05.802351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":64229,"num_write_kb":785523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"40'38","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.026729+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:10:07.026729+0000","last_peered":"2024-04-26T22:10:07.026729+0000","last_clean":"2024-04-26T22:10:07.026729+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:10:07.026729+0000","last_undegraded":"2024-04-26T22:10:07.026729+0000","last_fullsized":"2024-04-26T22:10:07.026729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":35,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2966","reported_seq":3026,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.990722+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:10:11.990722+0000","last_peered":"2024-04-26T22:10:11.990722+0000","last_clean":"2024-04-26T22:10:11.990722+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:10:11.990722+0000","last_undegraded":"2024-04-26T22:10:11.990722+0000","last_fullsized":"2024-04-26T22:10:11.990722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":2966,"log_dups_size":0,"ondisk_log_size":2966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":44534,"num_write_kb":522679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"40'1255","reported_seq":1319,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.935885+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:10:00.935885+0000","last_peered":"2024-04-26T22:10:00.935885+0000","last_clean":"2024-04-26T22:10:00.935885+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:10:00.935885+0000","last_undegraded":"2024-04-26T22:10:00.935885+0000","last_fullsized":"2024-04-26T22:10:00.935885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1255,"log_dups_size":0,"ondisk_log_size":1255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":20480,"num_write":1252,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"40'4813","reported_seq":4872,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836289+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:10:05.836289+0000","last_peered":"2024-04-26T22:10:05.836289+0000","last_clean":"2024-04-26T22:10:05.836289+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:10:05.836289+0000","last_undegraded":"2024-04-26T22:10:05.836289+0000","last_fullsized":"2024-04-26T22:10:05.836289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":72672,"num_write_kb":866290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"40'86","reported_seq":166,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953507+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:10:11.953507+0000","last_peered":"2024-04-26T22:10:11.953507+0000","last_clean":"2024-04-26T22:10:11.953507+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:10:11.953507+0000","last_undegraded":"2024-04-26T22:10:11.953507+0000","last_fullsized":"2024-04-26T22:10:11.953507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":62,"num_read_kb":49200,"num_write":70,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"40'3892","reported_seq":3964,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954209+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:10:11.954209+0000","last_peered":"2024-04-26T22:10:11.954209+0000","last_clean":"2024-04-26T22:10:11.954209+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:10:11.954209+0000","last_undegraded":"2024-04-26T22:10:11.954209+0000","last_fullsized":"2024-04-26T22:10:11.954209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":58573,"num_write_kb":703698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"40'6654","reported_seq":6712,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836315+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:10:05.836315+0000","last_peered":"2024-04-26T22:10:05.836315+0000","last_clean":"2024-04-26T22:10:05.836315+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:10:05.836315+0000","last_undegraded":"2024-04-26T22:10:05.836315+0000","last_fullsized":"2024-04-26T22:10:05.836315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":100552,"num_write_kb":1253937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"40'4775","reported_seq":4859,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979203+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:10:11.979203+0000","last_peered":"2024-04-26T22:10:11.979203+0000","last_clean":"2024-04-26T22:10:11.979203+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:10:11.979203+0000","last_undegraded":"2024-04-26T22:10:11.979203+0000","last_fullsized":"2024-04-26T22:10:11.979203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":71741,"num_write_kb":892635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"40'5861","reported_seq":5917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673275+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:10:05.673275+0000","last_peered":"2024-04-26T22:10:05.673275+0000","last_clean":"2024-04-26T22:10:05.673275+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:10:05.673275+0000","last_undegraded":"2024-04-26T22:10:05.673275+0000","last_fullsized":"2024-04-26T22:10:05.673275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88543,"num_write_kb":1086952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"40'3712","reported_seq":3779,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937208+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:10:11.937208+0000","last_peered":"2024-04-26T22:10:11.937208+0000","last_clean":"2024-04-26T22:10:11.937208+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:10:11.937208+0000","last_undegraded":"2024-04-26T22:10:11.937208+0000","last_fullsized":"2024-04-26T22:10:11.937208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":55802,"num_write_kb":677006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"40'6511","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.793662+0000","last_change":"2024-04-26T22:00:39.200941+0000","last_active":"2024-04-26T22:10:05.793662+0000","last_peered":"2024-04-26T22:10:05.793662+0000","last_clean":"2024-04-26T22:10:05.793662+0000","last_became_active":"2024-04-26T22:00:39.200592+0000","last_became_peered":"2024-04-26T22:00:39.200592+0000","last_unstale":"2024-04-26T22:10:05.793662+0000","last_undegraded":"2024-04-26T22:10:05.793662+0000","last_fullsized":"2024-04-26T22:10:05.793662+0000","mapping_epoch":25,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:59.988559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":98396,"num_write_kb":1218831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"40'4288","reported_seq":4364,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.886723+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:10:11.886723+0000","last_peered":"2024-04-26T22:10:11.886723+0000","last_clean":"2024-04-26T22:10:11.886723+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:10:11.886723+0000","last_undegraded":"2024-04-26T22:10:11.886723+0000","last_fullsized":"2024-04-26T22:10:11.886723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:55.523547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":64735,"num_write_kb":800999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"40'4650","reported_seq":4707,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852016+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:10:05.852016+0000","last_peered":"2024-04-26T22:10:05.852016+0000","last_clean":"2024-04-26T22:10:05.852016+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:10:05.852016+0000","last_undegraded":"2024-04-26T22:10:05.852016+0000","last_fullsized":"2024-04-26T22:10:05.852016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":70263,"num_write_kb":869775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"40'5113","reported_seq":5194,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.915535+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:10:11.915535+0000","last_peered":"2024-04-26T22:10:11.915535+0000","last_clean":"2024-04-26T22:10:11.915535+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:10:11.915535+0000","last_undegraded":"2024-04-26T22:10:11.915535+0000","last_fullsized":"2024-04-26T22:10:11.915535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":76970,"num_write_kb":914783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"40'4980","reported_seq":5038,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.818657+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:10:05.818657+0000","last_peered":"2024-04-26T22:10:05.818657+0000","last_clean":"2024-04-26T22:10:05.818657+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:10:05.818657+0000","last_undegraded":"2024-04-26T22:10:05.818657+0000","last_fullsized":"2024-04-26T22:10:05.818657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":75382,"num_write_kb":914602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"40'3556","reported_seq":3628,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.958144+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:10:11.958144+0000","last_peered":"2024-04-26T22:10:11.958144+0000","last_clean":"2024-04-26T22:10:11.958144+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:10:11.958144+0000","last_undegraded":"2024-04-26T22:10:11.958144+0000","last_fullsized":"2024-04-26T22:10:11.958144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":53572,"num_write_kb":653683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"40'5738","reported_seq":5795,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.760118+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:10:05.760118+0000","last_peered":"2024-04-26T22:10:05.760118+0000","last_clean":"2024-04-26T22:10:05.760118+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:10:05.760118+0000","last_undegraded":"2024-04-26T22:10:05.760118+0000","last_fullsized":"2024-04-26T22:10:05.760118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":86675,"num_write_kb":1103247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"40'5159","reported_seq":5238,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951137+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:10:11.951137+0000","last_peered":"2024-04-26T22:10:11.951137+0000","last_clean":"2024-04-26T22:10:11.951137+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:10:11.951137+0000","last_undegraded":"2024-04-26T22:10:11.951137+0000","last_fullsized":"2024-04-26T22:10:11.951137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":77704,"num_write_kb":942965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"40'4895","reported_seq":4967,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842652+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:10:05.842652+0000","last_peered":"2024-04-26T22:10:05.842652+0000","last_clean":"2024-04-26T22:10:05.842652+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:10:05.842652+0000","last_undegraded":"2024-04-26T22:10:05.842652+0000","last_fullsized":"2024-04-26T22:10:05.842652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":363,"num_write":74201,"num_write_kb":908910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"40'4370","reported_seq":4439,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.838189+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:10:11.838189+0000","last_peered":"2024-04-26T22:10:11.838189+0000","last_clean":"2024-04-26T22:10:11.838189+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:10:11.838189+0000","last_undegraded":"2024-04-26T22:10:11.838189+0000","last_fullsized":"2024-04-26T22:10:11.838189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4370,"log_dups_size":0,"ondisk_log_size":4370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":65733,"num_write_kb":804896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"40'6465","reported_seq":6525,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.853311+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:10:05.853311+0000","last_peered":"2024-04-26T22:10:05.853311+0000","last_clean":"2024-04-26T22:10:05.853311+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:10:05.853311+0000","last_undegraded":"2024-04-26T22:10:05.853311+0000","last_fullsized":"2024-04-26T22:10:05.853311+0000","mapping_epoch":18,"log_start":"30'1200","ondisk_log_start":"30'1200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5265,"log_dups_size":0,"ondisk_log_size":5265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":97620,"num_write_kb":1192207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"40'3334","reported_seq":3406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982941+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:10:11.982941+0000","last_peered":"2024-04-26T22:10:11.982941+0000","last_clean":"2024-04-26T22:10:11.982941+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:10:11.982941+0000","last_undegraded":"2024-04-26T22:10:11.982941+0000","last_fullsized":"2024-04-26T22:10:11.982941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":49724,"num_write_kb":588291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"40'4839","reported_seq":4896,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857008+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:10:05.857008+0000","last_peered":"2024-04-26T22:10:05.857008+0000","last_clean":"2024-04-26T22:10:05.857008+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:10:05.857008+0000","last_undegraded":"2024-04-26T22:10:05.857008+0000","last_fullsized":"2024-04-26T22:10:05.857008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":73090,"num_write_kb":888426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"40'5009","reported_seq":5079,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.919327+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:10:11.919327+0000","last_peered":"2024-04-26T22:10:11.919327+0000","last_clean":"2024-04-26T22:10:11.919327+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:10:11.919327+0000","last_undegraded":"2024-04-26T22:10:11.919327+0000","last_fullsized":"2024-04-26T22:10:11.919327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":75285,"num_write_kb":916871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"40'5333","reported_seq":5390,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.697837+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:10:05.697837+0000","last_peered":"2024-04-26T22:10:05.697837+0000","last_clean":"2024-04-26T22:10:05.697837+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:10:05.697837+0000","last_undegraded":"2024-04-26T22:10:05.697837+0000","last_fullsized":"2024-04-26T22:10:05.697837+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80596,"num_write_kb":994453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"40'4157","reported_seq":4231,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959972+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:10:11.959972+0000","last_peered":"2024-04-26T22:10:11.959972+0000","last_clean":"2024-04-26T22:10:11.959972+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:10:11.959972+0000","last_undegraded":"2024-04-26T22:10:11.959972+0000","last_fullsized":"2024-04-26T22:10:11.959972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":62491,"num_write_kb":760147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"40'5861","reported_seq":5904,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835233+0000","last_change":"2024-04-26T22:00:38.947156+0000","last_active":"2024-04-26T22:10:05.835233+0000","last_peered":"2024-04-26T22:10:05.835233+0000","last_clean":"2024-04-26T22:10:05.835233+0000","last_became_active":"2024-04-26T22:00:38.946980+0000","last_became_peered":"2024-04-26T22:00:38.946980+0000","last_unstale":"2024-04-26T22:10:05.835233+0000","last_undegraded":"2024-04-26T22:10:05.835233+0000","last_fullsized":"2024-04-26T22:10:05.835233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:02.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88698,"num_write_kb":1108811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"40'4156","reported_seq":4228,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977006+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:10:11.977006+0000","last_peered":"2024-04-26T22:10:11.977006+0000","last_clean":"2024-04-26T22:10:11.977006+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:10:11.977006+0000","last_undegraded":"2024-04-26T22:10:11.977006+0000","last_fullsized":"2024-04-26T22:10:11.977006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62195,"num_write_kb":734457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"40'5569","reported_seq":5629,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868845+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:10:05.868845+0000","last_peered":"2024-04-26T22:10:05.868845+0000","last_clean":"2024-04-26T22:10:05.868845+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:10:05.868845+0000","last_undegraded":"2024-04-26T22:10:05.868845+0000","last_fullsized":"2024-04-26T22:10:05.868845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5569,"log_dups_size":0,"ondisk_log_size":5569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":83899,"num_write_kb":1024385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"40'4460","reported_seq":4531,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955590+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:10:11.955590+0000","last_peered":"2024-04-26T22:10:11.955590+0000","last_clean":"2024-04-26T22:10:11.955590+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:10:11.955590+0000","last_undegraded":"2024-04-26T22:10:11.955590+0000","last_fullsized":"2024-04-26T22:10:11.955590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":67080,"num_write_kb":827171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"40'5284","reported_seq":5342,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.845154+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:10:05.845154+0000","last_peered":"2024-04-26T22:10:05.845154+0000","last_clean":"2024-04-26T22:10:05.845154+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:10:05.845154+0000","last_undegraded":"2024-04-26T22:10:05.845154+0000","last_fullsized":"2024-04-26T22:10:05.845154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":79924,"num_write_kb":1000899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"40'4361","reported_seq":4441,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972500+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:10:11.972500+0000","last_peered":"2024-04-26T22:10:11.972500+0000","last_clean":"2024-04-26T22:10:11.972500+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:10:11.972500+0000","last_undegraded":"2024-04-26T22:10:11.972500+0000","last_fullsized":"2024-04-26T22:10:11.972500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65464,"num_write_kb":794873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"40'5458","reported_seq":5516,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.860815+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:10:05.860815+0000","last_peered":"2024-04-26T22:10:05.860815+0000","last_clean":"2024-04-26T22:10:05.860815+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:10:05.860815+0000","last_undegraded":"2024-04-26T22:10:05.860815+0000","last_fullsized":"2024-04-26T22:10:05.860815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":82585,"num_write_kb":1000929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"40'3624","reported_seq":3690,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966367+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:10:11.966367+0000","last_peered":"2024-04-26T22:10:11.966367+0000","last_clean":"2024-04-26T22:10:11.966367+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:10:11.966367+0000","last_undegraded":"2024-04-26T22:10:11.966367+0000","last_fullsized":"2024-04-26T22:10:11.966367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":54495,"num_write_kb":662506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"40'4431","reported_seq":4495,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.870020+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:10:05.870020+0000","last_peered":"2024-04-26T22:10:05.870020+0000","last_clean":"2024-04-26T22:10:05.870020+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:10:05.870020+0000","last_undegraded":"2024-04-26T22:10:05.870020+0000","last_fullsized":"2024-04-26T22:10:05.870020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":66582,"num_write_kb":814643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"40'4089","reported_seq":4165,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.832376+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:10:11.832376+0000","last_peered":"2024-04-26T22:10:11.832376+0000","last_clean":"2024-04-26T22:10:11.832376+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:10:11.832376+0000","last_undegraded":"2024-04-26T22:10:11.832376+0000","last_fullsized":"2024-04-26T22:10:11.832376+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4089,"log_dups_size":0,"ondisk_log_size":4089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61370,"num_write_kb":735117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"40'6428","reported_seq":6487,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822613+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:10:05.822613+0000","last_peered":"2024-04-26T22:10:05.822613+0000","last_clean":"2024-04-26T22:10:05.822613+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:10:05.822613+0000","last_undegraded":"2024-04-26T22:10:05.822613+0000","last_fullsized":"2024-04-26T22:10:05.822613+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":97094,"num_write_kb":1219735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"40'4506","reported_seq":4588,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.945454+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:10:11.945454+0000","last_peered":"2024-04-26T22:10:11.945454+0000","last_clean":"2024-04-26T22:10:11.945454+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:10:11.945454+0000","last_undegraded":"2024-04-26T22:10:11.945454+0000","last_fullsized":"2024-04-26T22:10:11.945454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":67596,"num_write_kb":862862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"40'4747","reported_seq":4805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803411+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:10:05.803411+0000","last_peered":"2024-04-26T22:10:05.803411+0000","last_clean":"2024-04-26T22:10:05.803411+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:10:05.803411+0000","last_undegraded":"2024-04-26T22:10:05.803411+0000","last_fullsized":"2024-04-26T22:10:05.803411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71795,"num_write_kb":917779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"40'4901","reported_seq":4976,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.894392+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:10:11.894392+0000","last_peered":"2024-04-26T22:10:11.894392+0000","last_clean":"2024-04-26T22:10:11.894392+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:10:11.894392+0000","last_undegraded":"2024-04-26T22:10:11.894392+0000","last_fullsized":"2024-04-26T22:10:11.894392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4901,"log_dups_size":0,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":73795,"num_write_kb":926135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"40'5581","reported_seq":5639,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.843203+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:10:05.843203+0000","last_peered":"2024-04-26T22:10:05.843203+0000","last_clean":"2024-04-26T22:10:05.843203+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:10:05.843203+0000","last_undegraded":"2024-04-26T22:10:05.843203+0000","last_fullsized":"2024-04-26T22:10:05.843203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5581,"log_dups_size":0,"ondisk_log_size":5581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":84297,"num_write_kb":1026829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"40'3600","reported_seq":3671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.933325+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:10:11.933325+0000","last_peered":"2024-04-26T22:10:11.933325+0000","last_clean":"2024-04-26T22:10:11.933325+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:10:11.933325+0000","last_undegraded":"2024-04-26T22:10:11.933325+0000","last_fullsized":"2024-04-26T22:10:11.933325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":54091,"num_write_kb":649095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"40'5612","reported_seq":5671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.789467+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:10:05.789467+0000","last_peered":"2024-04-26T22:10:05.789467+0000","last_clean":"2024-04-26T22:10:05.789467+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:10:05.789467+0000","last_undegraded":"2024-04-26T22:10:05.789467+0000","last_fullsized":"2024-04-26T22:10:05.789467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":84765,"num_write_kb":1047612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"40'3974","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.882273+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:10:11.882273+0000","last_peered":"2024-04-26T22:10:11.882273+0000","last_clean":"2024-04-26T22:10:11.882273+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:10:11.882273+0000","last_undegraded":"2024-04-26T22:10:11.882273+0000","last_fullsized":"2024-04-26T22:10:11.882273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3974,"log_dups_size":0,"ondisk_log_size":3974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":59852,"num_write_kb":718261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"40'4819","reported_seq":4881,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.858034+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:10:05.858034+0000","last_peered":"2024-04-26T22:10:05.858034+0000","last_clean":"2024-04-26T22:10:05.858034+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:10:05.858034+0000","last_undegraded":"2024-04-26T22:10:05.858034+0000","last_fullsized":"2024-04-26T22:10:05.858034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":72748,"num_write_kb":932622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"40'4140","reported_seq":4220,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.931602+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:10:11.931602+0000","last_peered":"2024-04-26T22:10:11.931602+0000","last_clean":"2024-04-26T22:10:11.931602+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:10:11.931602+0000","last_undegraded":"2024-04-26T22:10:11.931602+0000","last_fullsized":"2024-04-26T22:10:11.931602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":62174,"num_write_kb":775596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"40'4503","reported_seq":4562,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.797708+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:10:05.797708+0000","last_peered":"2024-04-26T22:10:05.797708+0000","last_clean":"2024-04-26T22:10:05.797708+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:10:05.797708+0000","last_undegraded":"2024-04-26T22:10:05.797708+0000","last_fullsized":"2024-04-26T22:10:05.797708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":68209,"num_write_kb":848035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"40'3768","reported_seq":3848,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.978793+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:10:11.978793+0000","last_peered":"2024-04-26T22:10:11.978793+0000","last_clean":"2024-04-26T22:10:11.978793+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:10:11.978793+0000","last_undegraded":"2024-04-26T22:10:11.978793+0000","last_fullsized":"2024-04-26T22:10:11.978793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":56583,"num_write_kb":708038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"40'5303","reported_seq":5359,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.717590+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:10:05.717590+0000","last_peered":"2024-04-26T22:10:05.717590+0000","last_clean":"2024-04-26T22:10:05.717590+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:10:05.717590+0000","last_undegraded":"2024-04-26T22:10:05.717590+0000","last_fullsized":"2024-04-26T22:10:05.717590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80229,"num_write_kb":963753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"40'3667","reported_seq":3745,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.927871+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:10:11.927871+0000","last_peered":"2024-04-26T22:10:11.927871+0000","last_clean":"2024-04-26T22:10:11.927871+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:10:11.927871+0000","last_undegraded":"2024-04-26T22:10:11.927871+0000","last_fullsized":"2024-04-26T22:10:11.927871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3667,"log_dups_size":0,"ondisk_log_size":3667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":55047,"num_write_kb":687611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"40'6069","reported_seq":6128,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810736+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:10:05.810736+0000","last_peered":"2024-04-26T22:10:05.810736+0000","last_clean":"2024-04-26T22:10:05.810736+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:10:05.810736+0000","last_undegraded":"2024-04-26T22:10:05.810736+0000","last_fullsized":"2024-04-26T22:10:05.810736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":91606,"num_write_kb":1169870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"40'4173","reported_seq":4252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953675+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:10:11.953675+0000","last_peered":"2024-04-26T22:10:11.953675+0000","last_clean":"2024-04-26T22:10:11.953675+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:10:11.953675+0000","last_undegraded":"2024-04-26T22:10:11.953675+0000","last_fullsized":"2024-04-26T22:10:11.953675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":62684,"num_write_kb":755589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"40'6512","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.763471+0000","last_change":"2024-04-26T22:00:39.201110+0000","last_active":"2024-04-26T22:10:05.763471+0000","last_peered":"2024-04-26T22:10:05.763471+0000","last_clean":"2024-04-26T22:10:05.763471+0000","last_became_active":"2024-04-26T22:00:39.200691+0000","last_became_peered":"2024-04-26T22:00:39.200691+0000","last_unstale":"2024-04-26T22:10:05.763471+0000","last_undegraded":"2024-04-26T22:10:05.763471+0000","last_fullsized":"2024-04-26T22:10:05.763471+0000","mapping_epoch":25,"log_start":"29'700","ondisk_log_start":"29'700","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:06.181131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":98505,"num_write_kb":1266779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"40'5161","reported_seq":5247,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982946+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:10:11.982946+0000","last_peered":"2024-04-26T22:10:11.982946+0000","last_clean":"2024-04-26T22:10:11.982946+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:10:11.982946+0000","last_undegraded":"2024-04-26T22:10:11.982946+0000","last_fullsized":"2024-04-26T22:10:11.982946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":77820,"num_write_kb":957388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"40'5846","reported_seq":5903,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.706015+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:10:05.706015+0000","last_peered":"2024-04-26T22:10:05.706015+0000","last_clean":"2024-04-26T22:10:05.706015+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:10:05.706015+0000","last_undegraded":"2024-04-26T22:10:05.706015+0000","last_fullsized":"2024-04-26T22:10:05.706015+0000","mapping_epoch":18,"log_start":"29'600","ondisk_log_start":"29'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88593,"num_write_kb":1072922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"40'3484","reported_seq":3552,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972160+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:10:11.972160+0000","last_peered":"2024-04-26T22:10:11.972160+0000","last_clean":"2024-04-26T22:10:11.972160+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:10:11.972160+0000","last_undegraded":"2024-04-26T22:10:11.972160+0000","last_fullsized":"2024-04-26T22:10:11.972160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3484,"log_dups_size":0,"ondisk_log_size":3484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":52218,"num_write_kb":635931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"40'5213","reported_seq":5273,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874711+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:10:05.874711+0000","last_peered":"2024-04-26T22:10:05.874711+0000","last_clean":"2024-04-26T22:10:05.874711+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:10:05.874711+0000","last_undegraded":"2024-04-26T22:10:05.874711+0000","last_fullsized":"2024-04-26T22:10:05.874711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":78898,"num_write_kb":987567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"40'3779","reported_seq":3858,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.698994+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:10:11.698994+0000","last_peered":"2024-04-26T22:10:11.698994+0000","last_clean":"2024-04-26T22:10:11.698994+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:10:11.698994+0000","last_undegraded":"2024-04-26T22:10:11.698994+0000","last_fullsized":"2024-04-26T22:10:11.698994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56904,"num_write_kb":701117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"40'4635","reported_seq":4692,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.724357+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:10:05.724357+0000","last_peered":"2024-04-26T22:10:05.724357+0000","last_clean":"2024-04-26T22:10:05.724357+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:10:05.724357+0000","last_undegraded":"2024-04-26T22:10:05.724357+0000","last_fullsized":"2024-04-26T22:10:05.724357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4635,"log_dups_size":0,"ondisk_log_size":4635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":70184,"num_write_kb":869081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"40'4125","reported_seq":4197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954407+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:10:11.954407+0000","last_peered":"2024-04-26T22:10:11.954407+0000","last_clean":"2024-04-26T22:10:11.954407+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:10:11.954407+0000","last_undegraded":"2024-04-26T22:10:11.954407+0000","last_fullsized":"2024-04-26T22:10:11.954407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62115,"num_write_kb":743778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"40'6598","reported_seq":6661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.878241+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:10:05.878241+0000","last_peered":"2024-04-26T22:10:05.878241+0000","last_clean":"2024-04-26T22:10:05.878241+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:10:05.878241+0000","last_undegraded":"2024-04-26T22:10:05.878241+0000","last_fullsized":"2024-04-26T22:10:05.878241+0000","mapping_epoch":18,"log_start":"29'800","ondisk_log_start":"29'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":99507,"num_write_kb":1174132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"40'3584","reported_seq":3653,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979143+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:10:11.979143+0000","last_peered":"2024-04-26T22:10:11.979143+0000","last_clean":"2024-04-26T22:10:11.979143+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:10:11.979143+0000","last_undegraded":"2024-04-26T22:10:11.979143+0000","last_fullsized":"2024-04-26T22:10:11.979143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3584,"log_dups_size":0,"ondisk_log_size":3584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":54000,"num_write_kb":647349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"40'4854","reported_seq":4911,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.499989+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:10:05.499989+0000","last_peered":"2024-04-26T22:10:05.499989+0000","last_clean":"2024-04-26T22:10:05.499989+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:10:05.499989+0000","last_undegraded":"2024-04-26T22:10:05.499989+0000","last_fullsized":"2024-04-26T22:10:05.499989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73549,"num_write_kb":940613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"40'4744","reported_seq":4833,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.993205+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:10:11.993205+0000","last_peered":"2024-04-26T22:10:11.993205+0000","last_clean":"2024-04-26T22:10:11.993205+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:10:11.993205+0000","last_undegraded":"2024-04-26T22:10:11.993205+0000","last_fullsized":"2024-04-26T22:10:11.993205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":71239,"num_write_kb":843376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"40'5453","reported_seq":5511,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835411+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:10:05.835411+0000","last_peered":"2024-04-26T22:10:05.835411+0000","last_clean":"2024-04-26T22:10:05.835411+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:10:05.835411+0000","last_undegraded":"2024-04-26T22:10:05.835411+0000","last_fullsized":"2024-04-26T22:10:05.835411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5453,"log_dups_size":0,"ondisk_log_size":5453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":82428,"num_write_kb":1036078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"40'4470","reported_seq":4548,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.844612+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:10:11.844612+0000","last_peered":"2024-04-26T22:10:11.844612+0000","last_clean":"2024-04-26T22:10:11.844612+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:10:11.844612+0000","last_undegraded":"2024-04-26T22:10:11.844612+0000","last_fullsized":"2024-04-26T22:10:11.844612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":67360,"num_write_kb":793318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"40'6816","reported_seq":6875,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.873639+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:10:05.873639+0000","last_peered":"2024-04-26T22:10:05.873639+0000","last_clean":"2024-04-26T22:10:05.873639+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:10:05.873639+0000","last_undegraded":"2024-04-26T22:10:05.873639+0000","last_fullsized":"2024-04-26T22:10:05.873639+0000","mapping_epoch":18,"log_start":"29'1100","ondisk_log_start":"29'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":103074,"num_write_kb":1294412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"40'3980","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986399+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:10:11.986399+0000","last_peered":"2024-04-26T22:10:11.986399+0000","last_clean":"2024-04-26T22:10:11.986399+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:10:11.986399+0000","last_undegraded":"2024-04-26T22:10:11.986399+0000","last_fullsized":"2024-04-26T22:10:11.986399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":59871,"num_write_kb":728175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"40'4761","reported_seq":4821,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.861444+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:10:05.861444+0000","last_peered":"2024-04-26T22:10:05.861444+0000","last_clean":"2024-04-26T22:10:05.861444+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:10:05.861444+0000","last_undegraded":"2024-04-26T22:10:05.861444+0000","last_fullsized":"2024-04-26T22:10:05.861444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":72121,"num_write_kb":895283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1800,"num_read_kb":0,"num_write":3978586,"num_write_kb":48381196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264479,"ondisk_log_size":264479,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":109330781,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":691,"num_read_kb":381190,"num_write":3542,"num_write_kb":109337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218726400,"data_stored":218661562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3491,"ondisk_log_size":3491,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":363,"num_write":5266740,"num_write_kb":65270508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338431,"ondisk_log_size":338431,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":143059102,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":553,"num_read_kb":447893,"num_write":4271,"num_write_kb":143037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286179328,"data_stored":286117605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4237,"ondisk_log_size":4237,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525528,"kb_used_data":21200,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":93846312,"statfs":{"total":96636764160,"available":96098623488,"internally_reserved":0,"allocated":21708800,"data_stored":21468517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516422085},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848624,"kb_used_data":66920,"kb_used_omap":1,"kb_used_meta":781694,"kb_avail":93523216,"statfs":{"total":96636764160,"available":95767773184,"internally_reserved":0,"allocated":68526080,"data_stored":68281767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":800455114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574352,"kb_used_data":74184,"kb_used_omap":1,"kb_used_meta":500158,"kb_avail":93797488,"statfs":{"total":96636764160,"available":96048627712,"internally_reserved":0,"allocated":75964416,"data_stored":75718280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":512162246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789860,"kb_used_data":45596,"kb_used_omap":1,"kb_used_meta":744254,"kb_avail":93581980,"statfs":{"total":96636764160,"available":95827947520,"internally_reserved":0,"allocated":46690304,"data_stored":46445504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":762116549},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882668,"kb_used_data":82468,"kb_used_omap":1,"kb_used_meta":800190,"kb_avail":93489172,"statfs":{"total":96636764160,"available":95732912128,"internally_reserved":0,"allocated":84447232,"data_stored":84193910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":819395014},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900648,"kb_used_data":78560,"kb_used_omap":1,"kb_used_meta":822078,"kb_avail":93471192,"statfs":{"total":96636764160,"available":95714500608,"internally_reserved":0,"allocated":80445440,"data_stored":80190170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":841808327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767416,"kb_used_data":86832,"kb_used_omap":1,"kb_used_meta":680574,"kb_avail":93604424,"statfs":{"total":96636764160,"available":95850930176,"internally_reserved":0,"allocated":88915968,"data_stored":88663274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":696908229},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740368,"kb_used_data":41992,"kb_used_omap":1,"kb_used_meta":698366,"kb_avail":93631472,"statfs":{"total":96636764160,"available":95878627328,"internally_reserved":0,"allocated":42999808,"data_stored":42753566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715127237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29372416,"data_stored":29360766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":38112011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12673024,"data_stored":12671093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33652736,"data_stored":33643285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29364224,"data_stored":29360218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":46145536,"data_stored":46138016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50700288,"data_stored":50694746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8585216,"data_stored":8578005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46342144,"data_stored":46327365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37847040,"data_stored":37836920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16875520,"data_stored":16865968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:10:18.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:10:18.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.260+0000 7f0696e62640 1 Processor -- start 2024-04-26T22:10:18.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.260+0000 7f0696e62640 1 -- start start 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0696e62640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f06900c3060 0x7f06900720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0696e62640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06900c46e0 0x7f06900c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0696e62640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0690154ba0 con 0x7f06900c40e0 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0696e62640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0690154e80 con 0x7f06900c3060 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0696e62640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0690155160 con 0x7f06900c46e0 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0694bfe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f06900c3060 0x7f06900720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0687fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f06900c40e0 0x7f0690151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:47670/0 (socket says 172.21.15.135:47670) 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0687fff640 1 -- 172.21.15.135:0/1723967445 learned_addr learned my addr 172.21.15.135:0/1723967445 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:18.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0694bfe640 1 -- 172.21.15.135:0/1723967445 >> v2:172.21.15.139:3300/0 conn(0x7f06900c46e0 msgr2=0x7f06900c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f06953ff640 1 --2- 172.21.15.135:0/1723967445 >> v2:172.21.15.139:3300/0 conn(0x7f06900c46e0 0x7f06900c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0694bfe640 1 --2- 172.21.15.135:0/1723967445 >> v2:172.21.15.139:3300/0 conn(0x7f06900c46e0 0x7f06900c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0694bfe640 1 -- 172.21.15.135:0/1723967445 >> v1:172.21.15.135:6789/0 conn(0x7f06900c40e0 legacy=0x7f0690151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f06877fe640 1 -- 172.21.15.135:0/1723967445 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3096297183 0 0) 0x7f0690154ba0 con 0x7f06900c40e0 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.261+0000 7f0694bfe640 1 -- 172.21.15.135:0/1723967445 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0690155440 con 0x7f06900c3060 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.262+0000 7f0694bfe640 1 --2- 172.21.15.135:0/1723967445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f06900720a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f06900c4570 tx=0x7f068c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ea5e3591f65808 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.262+0000 7f06877fe640 1 -- 172.21.15.135:0/1723967445 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c037070 con 0x7f06900c3060 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.262+0000 7f06877fe640 1 -- 172.21.15.135:0/1723967445 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c032440 con 0x7f06900c3060 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.262+0000 7f06877fe640 1 -- 172.21.15.135:0/1723967445 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c02ae60 con 0x7f06900c3060 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 -- 172.21.15.135:0/1723967445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 msgr2=0x7f06900720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 --2- 172.21.15.135:0/1723967445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f06900720a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f06900c4570 tx=0x7f068c02a110 comp rx=0 tx=0).stop 2024-04-26T22:10:18.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 -- 172.21.15.135:0/1723967445 shutdown_connections 2024-04-26T22:10:18.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 --2- 172.21.15.135:0/1723967445 >> v2:172.21.15.139:3300/0 conn(0x7f06900c46e0 0x7f06900c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 --2- 172.21.15.135:0/1723967445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f06900720a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 -- 172.21.15.135:0/1723967445 >> 172.21.15.135:0/1723967445 conn(0x7f069006c310 msgr2=0x7f069006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:18.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 -- 172.21.15.135:0/1723967445 shutdown_connections 2024-04-26T22:10:18.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.263+0000 7f0696e62640 1 -- 172.21.15.135:0/1723967445 wait complete. 2024-04-26T22:10:18.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.264+0000 7f0696e62640 1 Processor -- start 2024-04-26T22:10:18.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.264+0000 7f0696e62640 1 -- start start 2024-04-26T22:10:18.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f0690147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:18.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06900c40e0 0x7f06901ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:18.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06900c46e0 0x7f06901eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:18.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06900c3440 con 0x7f06900c3060 2024-04-26T22:10:18.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06900c3aa0 con 0x7f06900c46e0 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f06900c35c0 con 0x7f06900c40e0 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f0690147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f0690147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51586/0 (socket says 172.21.15.135:51586) 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 -- 172.21.15.135:0/952915743 learned_addr learned my addr 172.21.15.135:0/952915743 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f06953ff640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06900c46e0 0x7f06901eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0687fff640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06900c40e0 0x7f06901ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 -- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06900c40e0 msgr2=0x7f06901ef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06900c40e0 0x7f06901ef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 -- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06900c46e0 msgr2=0x7f06901eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06900c46e0 0x7f06901eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 -- 172.21.15.135:0/952915743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06900ac9b0 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0694bfe640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f0690147a10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f068c000c00 tx=0x7f068c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c037040 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c042070 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06900acc90 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.265+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c030a50 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.266+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901ec500 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.267+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06901ec010 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.267+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f068c030bf0 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.267+0000 7f06857fa640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f067c06c8f0 0x7f067c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.267+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f068c0acdc0 con 0x7f06900c3060 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.267+0000 7f0687fff640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f067c06c8f0 0x7f067c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:18.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.268+0000 7f0687fff640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f067c06c8f0 0x7f067c06edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0690151aa0 tx=0x7f0680007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:18.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.272+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f068c045360 con 0x7f06900c3060 2024-04-26T22:10:18.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.448+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06900507a0 con 0x7f06900c3060 2024-04-26T22:10:18.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.449+0000 7f06857fa640 1 -- 172.21.15.135:0/952915743 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+22533 (secure 0 0 0) 0x7f068c07aa10 con 0x7f06900c3060 2024-04-26T22:10:18.449 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:18.449 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":40,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:10:00.929237+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6837/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6836/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6834/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6835/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.135:6837/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.139:6832/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6835/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6834/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6835/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.139:6833/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6836/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.139:6834/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6834/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.139:6835/302589348":"2024-04-27T22:05:52.818178+0000","172.21.15.139:6832/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.139:6835/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6833/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.135:6836/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6836/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.135:6837/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6835/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.135:6837/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.135:6835/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6834/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6835/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6834/302589348":"2024-04-27T22:05:52.818178+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.451+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f067c06c8f0 msgr2=0x7f067c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.451+0000 7f0696e62640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f067c06c8f0 0x7f067c06edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0690151aa0 tx=0x7f0680007c00 comp rx=0 tx=0).stop 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.451+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 msgr2=0x7f0690147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.451+0000 7f0696e62640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f0690147a10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f068c000c00 tx=0x7f068c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 shutdown_connections 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f067c06c8f0 0x7f067c06edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06900c46e0 0x7f06901eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06900c40e0 0x7f06901ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 --2- 172.21.15.135:0/952915743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900c3060 0x7f0690147a10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 >> 172.21.15.135:0/952915743 conn(0x7f069006c310 msgr2=0x7f0690074170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 shutdown_connections 2024-04-26T22:10:18.452 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:18.452+0000 7f0696e62640 1 -- 172.21.15.135:0/952915743 wait complete. 2024-04-26T22:10:19.464 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T22:10:19.464 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:19.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.578+0000 7f86f74b0640 1 Processor -- start 2024-04-26T22:10:19.578 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.578+0000 7f86f74b0640 1 -- start start 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f74b0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f86f0075ad0 0x7f86f0158300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f74b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86f0158840 0x7f86f015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f74b0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f0154db0 con 0x7f86f00756f0 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f74b0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f86f0075f50 con 0x7f86f0075ad0 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f74b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f86f0155090 con 0x7f86f0158840 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f4a4b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f86f0075ad0 0x7f86f0158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f524c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f86f00756f0 0x7f86f006d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46826/0 (socket says 172.21.15.135:46826) 2024-04-26T22:10:19.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f524c640 1 -- 172.21.15.135:0/3384019298 learned_addr learned my addr 172.21.15.135:0/3384019298 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:19.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3384019298 >> v2:172.21.15.139:3300/0 conn(0x7f86f0158840 msgr2=0x7f86f015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:19.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f4a4b640 1 --2- 172.21.15.135:0/3384019298 >> v2:172.21.15.139:3300/0 conn(0x7f86f0158840 0x7f86f015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3384019298 >> v1:172.21.15.135:6789/0 conn(0x7f86f00756f0 legacy=0x7f86f006d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3384019298 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0155370 con 0x7f86f0075ad0 2024-04-26T22:10:19.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.579+0000 7f86e7fff640 1 -- 172.21.15.135:0/3384019298 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168317596 0 0) 0x7f86f0154db0 con 0x7f86f00756f0 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.580+0000 7f86f4a4b640 1 --2- 172.21.15.135:0/3384019298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f0158300 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f86f00702c0 tx=0x7f86e002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259da7538c32c285 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.580+0000 7f86e7fff640 1 -- 172.21.15.135:0/3384019298 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0037070 con 0x7f86f0075ad0 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.580+0000 7f86e7fff640 1 -- 172.21.15.135:0/3384019298 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e0032440 con 0x7f86f0075ad0 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.580+0000 7f86e7fff640 1 -- 172.21.15.135:0/3384019298 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e002ae60 con 0x7f86f0075ad0 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 -- 172.21.15.135:0/3384019298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 msgr2=0x7f86f0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 --2- 172.21.15.135:0/3384019298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f0158300 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f86f00702c0 tx=0x7f86e002a110 comp rx=0 tx=0).stop 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 -- 172.21.15.135:0/3384019298 shutdown_connections 2024-04-26T22:10:19.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 --2- 172.21.15.135:0/3384019298 >> v2:172.21.15.139:3300/0 conn(0x7f86f0158840 0x7f86f015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 --2- 172.21.15.135:0/3384019298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f0158300 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 -- 172.21.15.135:0/3384019298 >> 172.21.15.135:0/3384019298 conn(0x7f86f00c5ce0 msgr2=0x7f86f00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:19.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 -- 172.21.15.135:0/3384019298 shutdown_connections 2024-04-26T22:10:19.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.581+0000 7f86f74b0640 1 -- 172.21.15.135:0/3384019298 wait complete. 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.582+0000 7f86f74b0640 1 Processor -- start 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.582+0000 7f86f74b0640 1 -- start start 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f00756f0 0x7f86f00c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f00c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86f0158840 0x7f86f0073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86f006dc50 con 0x7f86f0075ad0 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0155930 con 0x7f86f00756f0 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f86f00705c0 con 0x7f86f0158840 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f00c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f00c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51486/0 (socket says 172.21.15.135:51486) 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3104978378 learned_addr learned my addr 172.21.15.135:0/3104978378 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f524c640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f00756f0 0x7f86f00c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f5a4d640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86f0158840 0x7f86f0073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3104978378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86f0158840 msgr2=0x7f86f0073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86f0158840 0x7f86f0073d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3104978378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f00756f0 msgr2=0x7f86f00c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f00756f0 0x7f86f00c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 -- 172.21.15.135:0/3104978378 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f00c7ce0 con 0x7f86f0075ad0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f4a4b640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f00c2910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0041070 con 0x7f86f0075ad0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e003c950 con 0x7f86f0075ad0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f0072860 con 0x7f86f0075ad0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.583+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0037040 con 0x7f86f0075ad0 2024-04-26T22:10:19.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.584+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f0072e20 con 0x7f86f0075ad0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.584+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86f0073020 con 0x7f86f0075ad0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.585+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86e0044070 con 0x7f86f0075ad0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.585+0000 7f86e5ffb640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f86d406c8f0 0x7f86d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f86e00ad100 con 0x7f86f0075ad0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86f524c640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f86d406c8f0 0x7f86d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86e5ffb640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f86d4077d00 0x7f86d407a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 --> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86e0001d30 con 0x7f86d4077d00 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f86e00b4050 con 0x7f86f0075ad0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86f5a4d640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f86d4077d00 0x7f86d407a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86f524c640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f86d406c8f0 0x7f86d406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f86ec000f60 tx=0x7f86ec007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.586+0000 7f86f5a4d640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f86d4077d00 0x7f86d407a1a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.587+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== osd.0 v2:172.21.15.135:6816/3857357919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86e0001d30 con 0x7f86d4077d00 2024-04-26T22:10:19.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.600+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 --> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f86f00c5a70 con 0x7f86d4077d00 2024-04-26T22:10:19.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.600+0000 7f86e5ffb640 1 -- 172.21.15.135:0/3104978378 <== osd.0 v2:172.21.15.135:6816/3857357919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f86f00c5a70 con 0x7f86d4077d00 2024-04-26T22:10:19.600 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:19.600 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:19.600 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:19.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.600+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f86d4077d00 msgr2=0x7f86d407a1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.600+0000 7f86f74b0640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6816/3857357919,v1:172.21.15.135:6817/3857357919] conn(0x7f86d4077d00 0x7f86d407a1a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f86d406c8f0 msgr2=0x7f86d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f86d406c8f0 0x7f86d406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f86ec000f60 tx=0x7f86ec007c00 comp rx=0 tx=0).stop 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 msgr2=0x7f86f00c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 --2- 172.21.15.135:0/3104978378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86f0075ad0 0x7f86f00c2910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e00046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f5a4d640 1 -- 172.21.15.135:0/3104978378 reap_dead start 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 shutdown_connections 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 >> 172.21.15.135:0/3104978378 conn(0x7f86f00c5ce0 msgr2=0x7f86f006d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 shutdown_connections 2024-04-26T22:10:19.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.601+0000 7f86f74b0640 1 -- 172.21.15.135:0/3104978378 wait complete. 2024-04-26T22:10:19.611 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T22:10:19.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.710+0000 7f85339b4640 1 Processor -- start 2024-04-26T22:10:19.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.710+0000 7f85339b4640 1 -- start start 2024-04-26T22:10:19.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f85339b4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f852c0c3060 0x7f852c0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f85339b4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f852c154ba0 con 0x7f852c0c46e0 2024-04-26T22:10:19.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f85339b4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f852c154e80 con 0x7f852c0c40e0 2024-04-26T22:10:19.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f85339b4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f852c155160 con 0x7f852c0c3060 2024-04-26T22:10:19.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531750640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f852c0c3060 0x7f852c0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531750640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f852c0c3060 0x7f852c0720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51492/0 (socket says 172.21.15.135:51492) 2024-04-26T22:10:19.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531750640 1 -- 172.21.15.135:0/2630798374 learned_addr learned my addr 172.21.15.135:0/2630798374 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:19.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531f51640 1 --1- 172.21.15.135:0/2630798374 >> v1:172.21.15.135:6789/0 conn(0x7f852c0c46e0 0x7f852c0c4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46840/0 (socket says 172.21.15.135:46840) 2024-04-26T22:10:19.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531750640 1 -- 172.21.15.135:0/2630798374 >> v1:172.21.15.139:6789/0 conn(0x7f852c0c40e0 legacy=0x7f852c151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:19.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531750640 1 -- 172.21.15.135:0/2630798374 >> v1:172.21.15.135:6789/0 conn(0x7f852c0c46e0 legacy=0x7f852c0c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.711+0000 7f8531750640 1 -- 172.21.15.135:0/2630798374 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c155440 con 0x7f852c0c3060 2024-04-26T22:10:19.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f8531750640 1 --2- 172.21.15.135:0/2630798374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c3060 0x7f852c0720a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8528009870 tx=0x7f852802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f0e2a4bb9f6f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f8523fff640 1 -- 172.21.15.135:0/2630798374 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528037070 con 0x7f852c0c3060 2024-04-26T22:10:19.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f8523fff640 1 -- 172.21.15.135:0/2630798374 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8528032440 con 0x7f852c0c3060 2024-04-26T22:10:19.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f8523fff640 1 -- 172.21.15.135:0/2630798374 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528032810 con 0x7f852c0c3060 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f85339b4640 1 -- 172.21.15.135:0/2630798374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c3060 msgr2=0x7f852c0720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f85339b4640 1 --2- 172.21.15.135:0/2630798374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c3060 0x7f852c0720a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8528009870 tx=0x7f852802a210 comp rx=0 tx=0).stop 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f85339b4640 1 -- 172.21.15.135:0/2630798374 shutdown_connections 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f85339b4640 1 --2- 172.21.15.135:0/2630798374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c3060 0x7f852c0720a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.712+0000 7f85339b4640 1 -- 172.21.15.135:0/2630798374 >> 172.21.15.135:0/2630798374 conn(0x7f852c06c310 msgr2=0x7f852c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.713+0000 7f85339b4640 1 -- 172.21.15.135:0/2630798374 shutdown_connections 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.713+0000 7f85339b4640 1 -- 172.21.15.135:0/2630798374 wait complete. 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 Processor -- start 2024-04-26T22:10:19.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 -- start start 2024-04-26T22:10:19.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f852c0c3060 0x7f852c14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 0x7f852c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f852c0c46e0 0x7f852c1ea420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0c3aa0 con 0x7f852c0c40e0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0c5280 con 0x7f852c0c46e0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f85339b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f852c0c3440 con 0x7f852c0c3060 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f8530f4f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 0x7f852c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f8530f4f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 0x7f852c153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51504/0 (socket says 172.21.15.135:51504) 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.714+0000 7f8530f4f640 1 -- 172.21.15.135:0/2963808671 learned_addr learned my addr 172.21.15.135:0/2963808671 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8531f51640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f852c0c46e0 0x7f852c1ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8531750640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f852c0c3060 0x7f852c14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8530f4f640 1 -- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f852c0c3060 msgr2=0x7f852c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8530f4f640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f852c0c3060 0x7f852c14fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8530f4f640 1 -- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f852c0c46e0 msgr2=0x7f852c1ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8530f4f640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f852c0c46e0 0x7f852c1ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8530f4f640 1 -- 172.21.15.135:0/2963808671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c1560f0 con 0x7f852c0c40e0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8530f4f640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 0x7f852c153cb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c430 tx=0x7f851c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c011790 con 0x7f852c0c40e0 2024-04-26T22:10:19.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851c017070 con 0x7f852c0c40e0 2024-04-26T22:10:19.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1563d0 con 0x7f852c0c40e0 2024-04-26T22:10:19.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c011bb0 con 0x7f852c0c40e0 2024-04-26T22:10:19.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.715+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1534b0 con 0x7f852c0c40e0 2024-04-26T22:10:19.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.716+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c0507a0 con 0x7f852c0c40e0 2024-04-26T22:10:19.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.717+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f851c01c460 con 0x7f852c0c40e0 2024-04-26T22:10:19.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.717+0000 7f8521ffb640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f851006c8f0 0x7f851006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:19.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.717+0000 7f8531750640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f851006c8f0 0x7f851006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:19.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.717+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f851c08c220 con 0x7f852c0c40e0 2024-04-26T22:10:19.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.717+0000 7f8531750640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f851006c8f0 0x7f851006edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8528034000 tx=0x7f852802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:19.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.721+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f851c059de0 con 0x7f852c0c40e0 2024-04-26T22:10:19.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.898+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f852c153790 con 0x7f851006c8f0 2024-04-26T22:10:19.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.899+0000 7f8521ffb640 1 -- 172.21.15.135:0/2963808671 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f852c153790 con 0x7f851006c8f0 2024-04-26T22:10:19.899 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T22:10:19.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f851006c8f0 msgr2=0x7f851006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f851006c8f0 0x7f851006edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8528034000 tx=0x7f852802e000 comp rx=0 tx=0).stop 2024-04-26T22:10:19.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 msgr2=0x7f852c153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:19.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 0x7f852c153cb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c430 tx=0x7f851c00c8f0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 shutdown_connections 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f851006c8f0 0x7f851006edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f852c0c46e0 0x7f852c1ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f852c0c40e0 0x7f852c153cb0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 --2- 172.21.15.135:0/2963808671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f852c0c3060 0x7f852c14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 >> 172.21.15.135:0/2963808671 conn(0x7f852c06c310 msgr2=0x7f852c149f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 shutdown_connections 2024-04-26T22:10:19.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:19.901+0000 7f85339b4640 1 -- 172.21.15.135:0/2963808671 wait complete. 2024-04-26T22:10:19.912 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T22:10:19.912 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:20.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.013+0000 7f9ad8704640 1 Processor -- start 2024-04-26T22:10:20.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.013+0000 7f9ad8704640 1 -- start start 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.014+0000 7f9ad8704640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ad006d320 0x7f9ad01549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.014+0000 7f9ad8704640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006f720 0x7f9ad006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.014+0000 7f9ad8704640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9ad00c3ce0 0x7f9ad00c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.014+0000 7f9ad8704640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad0071020 con 0x7f9ad006d320 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.014+0000 7f9ad8704640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad0158ae0 con 0x7f9ad00c3ce0 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.014+0000 7f9ad8704640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad0158d80 con 0x7f9ad006f720 2024-04-26T22:10:20.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ad006d320 0x7f9ad01549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ad006d320 0x7f9ad01549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51512/0 (socket says 172.21.15.135:51512) 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 -- 172.21.15.135:0/409930471 learned_addr learned my addr 172.21.15.135:0/409930471 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad6ca1640 1 --2- 172.21.15.135:0/409930471 >> v2:172.21.15.139:3300/0 conn(0x7f9ad00c3ce0 0x7f9ad00c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad5c9f640 1 --2- 172.21.15.135:0/409930471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006f720 0x7f9ad006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 -- 172.21.15.135:0/409930471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006f720 msgr2=0x7f9ad006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 --2- 172.21.15.135:0/409930471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006f720 0x7f9ad006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 -- 172.21.15.135:0/409930471 >> v2:172.21.15.139:3300/0 conn(0x7f9ad00c3ce0 msgr2=0x7f9ad00c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 --2- 172.21.15.135:0/409930471 >> v2:172.21.15.139:3300/0 conn(0x7f9ad00c3ce0 0x7f9ad00c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.015+0000 7f9ad64a0640 1 -- 172.21.15.135:0/409930471 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad006fd60 con 0x7f9ad006d320 2024-04-26T22:10:20.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.016+0000 7f9ad64a0640 1 --2- 172.21.15.135:0/409930471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006d320 0x7f9ad01549d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00707f0 tx=0x7f9acc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef048b946ad49b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.016+0000 7f9ad549e640 1 -- 172.21.15.135:0/409930471 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc036070 con 0x7f9ad006d320 2024-04-26T22:10:20.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.016+0000 7f9ad549e640 1 -- 172.21.15.135:0/409930471 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9acc031600 con 0x7f9ad006d320 2024-04-26T22:10:20.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.016+0000 7f9ad549e640 1 -- 172.21.15.135:0/409930471 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0319d0 con 0x7f9ad006d320 2024-04-26T22:10:20.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 -- 172.21.15.135:0/409930471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006d320 msgr2=0x7f9ad01549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 --2- 172.21.15.135:0/409930471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006d320 0x7f9ad01549d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00707f0 tx=0x7f9acc02a610 comp rx=0 tx=0).stop 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 -- 172.21.15.135:0/409930471 shutdown_connections 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 --2- 172.21.15.135:0/409930471 >> v2:172.21.15.139:3300/0 conn(0x7f9ad00c3ce0 0x7f9ad00c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 --2- 172.21.15.135:0/409930471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006f720 0x7f9ad006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 --2- 172.21.15.135:0/409930471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006d320 0x7f9ad01549d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 -- 172.21.15.135:0/409930471 >> 172.21.15.135:0/409930471 conn(0x7f9ad00c5ce0 msgr2=0x7f9ad00c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 -- 172.21.15.135:0/409930471 shutdown_connections 2024-04-26T22:10:20.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.017+0000 7f9ad8704640 1 -- 172.21.15.135:0/409930471 wait complete. 2024-04-26T22:10:20.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.018+0000 7f9ad8704640 1 Processor -- start 2024-04-26T22:10:20.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.018+0000 7f9ad8704640 1 -- start start 2024-04-26T22:10:20.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006d320 0x7f9ad00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006f720 0x7f9ad0147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad00c3ce0 0x7f9ad0148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0070c60 con 0x7f9ad006f720 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0152360 con 0x7f9ad00c3ce0 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad0070de0 con 0x7f9ad006d320 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006f720 0x7f9ad0147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006f720 0x7f9ad0147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51518/0 (socket says 172.21.15.135:51518) 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 -- 172.21.15.135:0/3902125598 learned_addr learned my addr 172.21.15.135:0/3902125598 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad6ca1640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad00c3ce0 0x7f9ad0148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad64a0640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006d320 0x7f9ad00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 -- 172.21.15.135:0/3902125598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006d320 msgr2=0x7f9ad00c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad006d320 0x7f9ad00c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 -- 172.21.15.135:0/3902125598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad00c3ce0 msgr2=0x7f9ad0148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad00c3ce0 0x7f9ad0148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 -- 172.21.15.135:0/3902125598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad0156c30 con 0x7f9ad006f720 2024-04-26T22:10:20.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad5c9f640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006f720 0x7f9ad0147b00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d800 tx=0x7f9ac000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000b660 con 0x7f9ad006f720 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac0019070 con 0x7f9ad006f720 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000ba40 con 0x7f9ad006f720 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad0156950 con 0x7f9ad006f720 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.019+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad00c88f0 con 0x7f9ad006f720 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.020+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ad0069bc0 con 0x7f9ad006f720 2024-04-26T22:10:20.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ac000bbe0 con 0x7f9ad006f720 2024-04-26T22:10:20.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ac6ffd640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ab406c8f0 0x7f9ab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f9ac008c670 con 0x7f9ad006f720 2024-04-26T22:10:20.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ad64a0640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ab406c8f0 0x7f9ab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ac6ffd640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f9ab4077d10 0x7f9ab407a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 --> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ac0016e70 con 0x7f9ab4077d10 2024-04-26T22:10:20.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ac0093050 con 0x7f9ad006f720 2024-04-26T22:10:20.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.021+0000 7f9ad6ca1640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f9ab4077d10 0x7f9ab407a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.022+0000 7f9ad64a0640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ab406c8f0 0x7f9ab406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9acc033000 tx=0x7f9acc009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.022+0000 7f9ad6ca1640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f9ab4077d10 0x7f9ab407a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.022+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== osd.1 v2:172.21.15.135:6808/3325535972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ac0016e70 con 0x7f9ab4077d10 2024-04-26T22:10:20.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.035+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 --> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9ad00c5ac0 con 0x7f9ab4077d10 2024-04-26T22:10:20.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.035+0000 7f9ac6ffd640 1 -- 172.21.15.135:0/3902125598 <== osd.1 v2:172.21.15.135:6808/3325535972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9ad00c5ac0 con 0x7f9ab4077d10 2024-04-26T22:10:20.036 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:20.036 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:20.036 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f9ab4077d10 msgr2=0x7f9ab407a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6808/3325535972,v1:172.21.15.135:6809/3325535972] conn(0x7f9ab4077d10 0x7f9ab407a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ab406c8f0 msgr2=0x7f9ab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9ab406c8f0 0x7f9ab406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9acc033000 tx=0x7f9acc009000 comp rx=0 tx=0).stop 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006f720 msgr2=0x7f9ad0147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 --2- 172.21.15.135:0/3902125598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad006f720 0x7f9ad0147b00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d800 tx=0x7f9ac000dcc0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad6ca1640 1 -- 172.21.15.135:0/3902125598 reap_dead start 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 shutdown_connections 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 >> 172.21.15.135:0/3902125598 conn(0x7f9ad00c5ce0 msgr2=0x7f9ad0151640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 shutdown_connections 2024-04-26T22:10:20.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.036+0000 7f9ad8704640 1 -- 172.21.15.135:0/3902125598 wait complete. 2024-04-26T22:10:20.046 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T22:10:20.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.146+0000 7febe6e83640 1 Processor -- start 2024-04-26T22:10:20.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.146+0000 7febe6e83640 1 -- start start 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.146+0000 7febe6e83640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.146+0000 7febe6e83640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7febe006f790 0x7febe00c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.146+0000 7febe6e83640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe00c3d10 con 0x7febe0157680 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.146+0000 7febe6e83640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7febe00c4290 con 0x7febe006f790 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febe6e83640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7febe00c3fb0 con 0x7febe0150d20 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7febe006f790 0x7febe00c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7febe006f790 0x7febe00c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51528/0 (socket says 172.21.15.135:51528) 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 -- 172.21.15.135:0/1607024293 learned_addr learned my addr 172.21.15.135:0/1607024293 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febe4c1f640 1 --2- 172.21.15.135:0/1607024293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 -- 172.21.15.135:0/1607024293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 msgr2=0x7febe015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 --2- 172.21.15.135:0/1607024293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 -- 172.21.15.135:0/1607024293 >> v1:172.21.15.139:6789/0 conn(0x7febe0157680 legacy=0x7febe006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.147+0000 7febdffff640 1 -- 172.21.15.135:0/1607024293 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe0156e80 con 0x7febe006f790 2024-04-26T22:10:20.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febdffff640 1 --2- 172.21.15.135:0/1607024293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe006f790 0x7febe00c32e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7febcc009870 tx=0x7febcc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ded03c00f834a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febdf7fe640 1 -- 172.21.15.135:0/1607024293 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc037070 con 0x7febe006f790 2024-04-26T22:10:20.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febdf7fe640 1 -- 172.21.15.135:0/1607024293 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc032440 con 0x7febe006f790 2024-04-26T22:10:20.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febdf7fe640 1 -- 172.21.15.135:0/1607024293 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc032810 con 0x7febe006f790 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febe6e83640 1 -- 172.21.15.135:0/1607024293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe006f790 msgr2=0x7febe00c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febe6e83640 1 --2- 172.21.15.135:0/1607024293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe006f790 0x7febe00c32e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7febcc009870 tx=0x7febcc02a1c0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febe6e83640 1 -- 172.21.15.135:0/1607024293 shutdown_connections 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febe6e83640 1 --2- 172.21.15.135:0/1607024293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe006f790 0x7febe00c32e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febe6e83640 1 --2- 172.21.15.135:0/1607024293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.148+0000 7febe6e83640 1 -- 172.21.15.135:0/1607024293 >> 172.21.15.135:0/1607024293 conn(0x7febe00c5ce0 msgr2=0x7febe00c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.149+0000 7febe6e83640 1 -- 172.21.15.135:0/1607024293 shutdown_connections 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.149+0000 7febe6e83640 1 -- 172.21.15.135:0/1607024293 wait complete. 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.150+0000 7febe6e83640 1 Processor -- start 2024-04-26T22:10:20.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.150+0000 7febe6e83640 1 -- start start 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.150+0000 7febe6e83640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febe006f790 0x7febe0074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.150+0000 7febe6e83640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe01418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.150+0000 7febe6e83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 0x7febe0141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe6e83640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febe006da30 con 0x7febe0157680 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe6e83640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7febe00c3820 con 0x7febe006f790 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe6e83640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7febe006dbb0 con 0x7febe0150d20 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 0x7febe0141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 0x7febe0141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51538/0 (socket says 172.21.15.135:51538) 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 -- 172.21.15.135:0/378601024 learned_addr learned my addr 172.21.15.135:0/378601024 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe4c1f640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febe006f790 0x7febe0074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febdffff640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe01418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 -- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 msgr2=0x7febe01418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe01418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 -- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febe006f790 msgr2=0x7febe0074590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febe006f790 0x7febe0074590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 -- 172.21.15.135:0/378601024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01504c0 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe5420640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 0x7febe0141de0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7febd800c6e0 tx=0x7febd800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8003b70 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd8003d30 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01ece30 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.151+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8005310 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.152+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01f0b10 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.153+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe006d7b0 con 0x7febe0157680 2024-04-26T22:10:20.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.153+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febd80054b0 con 0x7febe0157680 2024-04-26T22:10:20.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.153+0000 7febdd7fa640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7febc406c8f0 0x7febc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.153+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7febd808c020 con 0x7febe0157680 2024-04-26T22:10:20.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.153+0000 7febe4c1f640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7febc406c8f0 0x7febc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.154+0000 7febe4c1f640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7febc406c8f0 0x7febc406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7febd4009b40 tx=0x7febd4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.158+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febe006d7b0 con 0x7febe0157680 2024-04-26T22:10:20.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.334+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7febe01ed000 con 0x7febc406c8f0 2024-04-26T22:10:20.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.335+0000 7febdd7fa640 1 -- 172.21.15.135:0/378601024 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7febe01ed000 con 0x7febc406c8f0 2024-04-26T22:10:20.335 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T22:10:20.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7febc406c8f0 msgr2=0x7febc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7febc406c8f0 0x7febc406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7febd4009b40 tx=0x7febd4009190 comp rx=0 tx=0).stop 2024-04-26T22:10:20.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 msgr2=0x7febe0141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 0x7febe0141de0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7febd800c6e0 tx=0x7febd800cba0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 shutdown_connections 2024-04-26T22:10:20.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7febc406c8f0 0x7febc406edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe0157680 0x7febe0141de0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febe0150d20 0x7febe01418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 --2- 172.21.15.135:0/378601024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febe006f790 0x7febe0074590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 >> 172.21.15.135:0/378601024 conn(0x7febe00c5ce0 msgr2=0x7febe0155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 shutdown_connections 2024-04-26T22:10:20.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.337+0000 7febe6e83640 1 -- 172.21.15.135:0/378601024 wait complete. 2024-04-26T22:10:20.348 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T22:10:20.348 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:20.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.450+0000 7f68234ea640 1 Processor -- start 2024-04-26T22:10:20.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.450+0000 7f68234ea640 1 -- start start 2024-04-26T22:10:20.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f68234ea640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f681c0ab470 0x7f681c073960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f68234ea640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f681c156ef0 con 0x7f681c0ac560 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f68234ea640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f681c070660 con 0x7f681c156ad0 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f68234ea640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f681c0ab9a0 con 0x7f681c0ab470 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6821286640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f681c0ac560 0x7f681c070b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46844/0 (socket says 172.21.15.135:46844) 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6821286640 1 -- 172.21.15.135:0/2892652444 learned_addr learned my addr 172.21.15.135:0/2892652444 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6820a85640 1 --2- 172.21.15.135:0/2892652444 >> v2:172.21.15.139:3300/0 conn(0x7f681c0ab470 0x7f681c073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215478019 0 0) 0x7f681c156ef0 con 0x7f681c0ac560 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6808003520 con 0x7f681c0ac560 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424016059 0 0) 0x7f681c070660 con 0x7f681c156ad0 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f681c156ef0 con 0x7f681c156ad0 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 539308196 0 0) 0x7f6808003520 con 0x7f681c0ac560 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 >> v2:172.21.15.139:3300/0 conn(0x7f681c0ab470 msgr2=0x7f681c073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 --2- 172.21.15.135:0/2892652444 >> v2:172.21.15.139:3300/0 conn(0x7f681c0ab470 0x7f681c073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 >> v1:172.21.15.139:6789/0 conn(0x7f681c156ad0 legacy=0x7f681c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c071270 con 0x7f681c0ac560 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f6818001670 con 0x7f681c0ac560 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6820a85640 1 --2- 172.21.15.135:0/2892652444 >> v2:172.21.15.139:3300/0 conn(0x7f681c0ab470 0x7f681c073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 >> v1:172.21.15.135:6789/0 conn(0x7f681c0ac560 legacy=0x7f681c070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6808003b90 0x7f6808003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6808004800 0x7f6808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f680800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f681c0ab9a0 con 0x7f6808005280 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.451+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f680800c1c0 con 0x7f6808003b90 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f680800c3c0 con 0x7f6808004800 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f680800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821286640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6808003b90 0x7f6808003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6820a85640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6808004800 0x7f6808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 -- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6808004800 msgr2=0x7f6808004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6808004800 0x7f6808004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 -- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6808003b90 msgr2=0x7f6808003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6808003b90 0x7f6808003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 -- 172.21.15.135:0/2892652444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6808003520 con 0x7f6808005280 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.452+0000 7f6821a87640 1 -- 172.21.15.135:0/2892652444 reap_dead start 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f6821a87640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f680800b9c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6814001fd0 tx=0x7f6808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f988d72b2943fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814001bf0 con 0x7f6808005280 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6814003b70 con 0x7f6808005280 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f6813fff640 1 -- 172.21.15.135:0/2892652444 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68140050d0 con 0x7f6808005280 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 -- 172.21.15.135:0/2892652444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 msgr2=0x7f680800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f680800b9c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6814001fd0 tx=0x7f6808005830 comp rx=0 tx=0).stop 2024-04-26T22:10:20.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 -- 172.21.15.135:0/2892652444 shutdown_connections 2024-04-26T22:10:20.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 --2- 172.21.15.135:0/2892652444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f680800b9c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 -- 172.21.15.135:0/2892652444 >> 172.21.15.135:0/2892652444 conn(0x7f681c0ca2f0 msgr2=0x7f681c0ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 -- 172.21.15.135:0/2892652444 shutdown_connections 2024-04-26T22:10:20.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.453+0000 7f68234ea640 1 -- 172.21.15.135:0/2892652444 wait complete. 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.454+0000 7f68234ea640 1 Processor -- start 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.454+0000 7f68234ea640 1 -- start start 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f68234ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f681c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f68234ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c145d00 0x7f681c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f68234ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06ca50 0x7f681c1ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f68234ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f681c0c6b90 con 0x7f6808005280 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f68234ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f681c0abed0 con 0x7f681c06ca50 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f68234ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f681c0ac940 con 0x7f681c145d00 2024-04-26T22:10:20.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f681c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f681c152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51554/0 (socket says 172.21.15.135:51554) 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 -- 172.21.15.135:0/1315957541 learned_addr learned my addr 172.21.15.135:0/1315957541 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821a87640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06ca50 0x7f681c1ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6820a85640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c145d00 0x7f681c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 -- 172.21.15.135:0/1315957541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c145d00 msgr2=0x7f681c06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f681c145d00 0x7f681c06c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 -- 172.21.15.135:0/1315957541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06ca50 msgr2=0x7f681c1ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f681c06ca50 0x7f681c1ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.455+0000 7f6821286640 1 -- 172.21.15.135:0/1315957541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c1ec1b0 con 0x7f6808005280 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.456+0000 7f6821286640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f681c152920 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6818001d70 tx=0x7f6818007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.456+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681800f6b0 con 0x7f6808005280 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.456+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6818018070 con 0x7f6808005280 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.456+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c1ec470 con 0x7f6808005280 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.456+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681800fab0 con 0x7f6808005280 2024-04-26T22:10:20.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.456+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c1eca30 con 0x7f6808005280 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.457+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f681c0694b0 con 0x7f6808005280 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.457+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f681800fc50 con 0x7f6808005280 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.457+0000 7f6811ffb640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680006c8f0 0x7f680006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f681808c210 con 0x7f6808005280 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6820a85640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680006c8f0 0x7f680006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6811ffb640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6800077d10 0x7f680007a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 --> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6818020e80 con 0x7f6800077d10 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6818093050 con 0x7f6808005280 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6821a87640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6800077d10 0x7f680007a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6820a85640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680006c8f0 0x7f680006edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f681c149b80 tx=0x7f680c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.458+0000 7f6821a87640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6800077d10 0x7f680007a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.459+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== osd.2 v2:172.21.15.135:6824/626924944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6818020e80 con 0x7f6800077d10 2024-04-26T22:10:20.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 --> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f681c151380 con 0x7f6800077d10 2024-04-26T22:10:20.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f6811ffb640 1 -- 172.21.15.135:0/1315957541 <== osd.2 v2:172.21.15.135:6824/626924944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f681c151380 con 0x7f6800077d10 2024-04-26T22:10:20.472 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:20.472 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:20.472 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6800077d10 msgr2=0x7f680007a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6824/626924944,v1:172.21.15.135:6825/626924944] conn(0x7f6800077d10 0x7f680007a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680006c8f0 msgr2=0x7f680006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f680006c8f0 0x7f680006edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f681c149b80 tx=0x7f680c008040 comp rx=0 tx=0).stop 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 msgr2=0x7f681c152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 --2- 172.21.15.135:0/1315957541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808005280 0x7f681c152920 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6818001d70 tx=0x7f6818007430 comp rx=0 tx=0).stop 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f6821a87640 1 -- 172.21.15.135:0/1315957541 reap_dead start 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 shutdown_connections 2024-04-26T22:10:20.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 >> 172.21.15.135:0/1315957541 conn(0x7f681c0ca2f0 msgr2=0x7f681c155a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 shutdown_connections 2024-04-26T22:10:20.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.472+0000 7f68234ea640 1 -- 172.21.15.135:0/1315957541 wait complete. 2024-04-26T22:10:20.484 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T22:10:20.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.581+0000 7fc4fcd51640 1 Processor -- start 2024-04-26T22:10:20.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.581+0000 7fc4fcd51640 1 -- start start 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4fcd51640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4f813feb0 0x7fc4f81576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4fcd51640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4f80c5050 0x7fc4f815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4fcd51640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f8156c10 con 0x7fc4f80c2b80 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4fcd51640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f8157240 con 0x7fc4f813feb0 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4fcd51640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f80c5570 con 0x7fc4f80c5050 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4f813feb0 0x7fc4f81576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4f813feb0 0x7fc4f81576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51558/0 (socket says 172.21.15.135:51558) 2024-04-26T22:10:20.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 -- 172.21.15.135:0/2833354171 learned_addr learned my addr 172.21.15.135:0/2833354171 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 -- 172.21.15.135:0/2833354171 >> v2:172.21.15.139:3300/0 conn(0x7fc4f80c5050 msgr2=0x7fc4f815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 --2- 172.21.15.135:0/2833354171 >> v2:172.21.15.139:3300/0 conn(0x7fc4f80c5050 0x7fc4f815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 -- 172.21.15.135:0/2833354171 >> v1:172.21.15.139:6789/0 conn(0x7fc4f80c2b80 legacy=0x7fc4f806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.582+0000 7fc4f659c640 1 -- 172.21.15.135:0/2833354171 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f80c5850 con 0x7fc4f813feb0 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.583+0000 7fc4f659c640 1 --2- 172.21.15.135:0/2833354171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f81576d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8070760 tx=0x7fc4e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a90333ad819d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.583+0000 7fc4f5d9b640 1 -- 172.21.15.135:0/2833354171 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8037070 con 0x7fc4f813feb0 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.583+0000 7fc4f5d9b640 1 -- 172.21.15.135:0/2833354171 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e802ae20 con 0x7fc4f813feb0 2024-04-26T22:10:20.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.583+0000 7fc4f5d9b640 1 -- 172.21.15.135:0/2833354171 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8032660 con 0x7fc4f813feb0 2024-04-26T22:10:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 -- 172.21.15.135:0/2833354171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 msgr2=0x7fc4f81576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/2833354171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f81576d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8070760 tx=0x7fc4e802a110 comp rx=0 tx=0).stop 2024-04-26T22:10:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 -- 172.21.15.135:0/2833354171 shutdown_connections 2024-04-26T22:10:20.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/2833354171 >> v2:172.21.15.139:3300/0 conn(0x7fc4f80c5050 0x7fc4f815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/2833354171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f81576d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 -- 172.21.15.135:0/2833354171 >> 172.21.15.135:0/2833354171 conn(0x7fc4f80c5ce0 msgr2=0x7fc4f80c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 -- 172.21.15.135:0/2833354171 shutdown_connections 2024-04-26T22:10:20.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.584+0000 7fc4fcd51640 1 -- 172.21.15.135:0/2833354171 wait complete. 2024-04-26T22:10:20.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.585+0000 7fc4fcd51640 1 Processor -- start 2024-04-26T22:10:20.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.585+0000 7fc4fcd51640 1 -- start start 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4fcd51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4f80c2b80 0x7fc4f8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4fcd51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4f80c5050 0x7fc4f8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4fcd51640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f8073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4fcd51640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f806dbb0 con 0x7fc4f813feb0 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4fcd51640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f806da30 con 0x7fc4f80c2b80 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4fcd51640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f8140290 con 0x7fc4f80c5050 2024-04-26T22:10:20.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f8073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f8073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51574/0 (socket says 172.21.15.135:51574) 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 -- 172.21.15.135:0/3683026907 learned_addr learned my addr 172.21.15.135:0/3683026907 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f659c640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4f80c2b80 0x7fc4f8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4edd9b640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4f80c5050 0x7fc4f8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 -- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4f80c5050 msgr2=0x7fc4f8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4f80c5050 0x7fc4f8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 -- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4f80c2b80 msgr2=0x7fc4f8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4f80c2b80 0x7fc4f8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f6d9d640 1 -- 172.21.15.135:0/3683026907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f8158800 con 0x7fc4f813feb0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.586+0000 7fc4f659c640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4f80c2b80 0x7fc4f8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.587+0000 7fc4f6d9d640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f8073010 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b990 tx=0x7fc4f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.587+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0019070 con 0x7fc4f813feb0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.587+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f000cba0 con 0x7fc4f813feb0 2024-04-26T22:10:20.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.587+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f8158ae0 con 0x7fc4f813feb0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.587+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0014440 con 0x7fc4f813feb0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.587+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f80adfe0 con 0x7fc4f813feb0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.588+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f8158cb0 con 0x7fc4f813feb0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.588+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4f001c070 con 0x7fc4f813feb0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.589+0000 7fc4ef7fe640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc4d806c920 0x7fc4d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.589+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fc4f008bf80 con 0x7fc4f813feb0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.589+0000 7fc4f659c640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc4d806c920 0x7fc4d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.589+0000 7fc4f659c640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc4d806c920 0x7fc4d806ede0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8034000 tx=0x7fc4e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.593+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4f8158cb0 con 0x7fc4f813feb0 2024-04-26T22:10:20.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.770+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc4f81f0910 con 0x7fc4d806c920 2024-04-26T22:10:20.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.771+0000 7fc4ef7fe640 1 -- 172.21.15.135:0/3683026907 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc4e80328f0 con 0x7fc4d806c920 2024-04-26T22:10:20.771 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc4d806c920 msgr2=0x7fc4d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc4d806c920 0x7fc4d806ede0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8034000 tx=0x7fc4e802e000 comp rx=0 tx=0).stop 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 msgr2=0x7fc4f8073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f8073010 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b990 tx=0x7fc4f000be50 comp rx=0 tx=0).stop 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 shutdown_connections 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fc4d806c920 0x7fc4d806ede0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f813feb0 0x7fc4f8073010 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4f80c5050 0x7fc4f8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 --2- 172.21.15.135:0/3683026907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4f80c2b80 0x7fc4f8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 >> 172.21.15.135:0/3683026907 conn(0x7fc4f80c5ce0 msgr2=0x7fc4f813ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 shutdown_connections 2024-04-26T22:10:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.773+0000 7fc4fcd51640 1 -- 172.21.15.135:0/3683026907 wait complete. 2024-04-26T22:10:20.784 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T22:10:20.784 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:20.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 Processor -- start 2024-04-26T22:10:20.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 -- start start 2024-04-26T22:10:20.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f16741543b0 0x7f16740ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16740c8a10 0x7f16740c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1674154790 con 0x7f16740cbd80 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f16740cc210 con 0x7f16741543b0 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f1678b6a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1674157ca0 con 0x7f16740c8a10 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f167259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f16741543b0 0x7f16740ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.885+0000 7f167259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f16741543b0 0x7f16740ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51588/0 (socket says 172.21.15.135:51588) 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f167259c640 1 -- 172.21.15.135:0/398482599 learned_addr learned my addr 172.21.15.135:0/398482599 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f1672d9d640 1 --2- 172.21.15.135:0/398482599 >> v2:172.21.15.139:3300/0 conn(0x7f16740c8a10 0x7f16740c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f167259c640 1 -- 172.21.15.135:0/398482599 >> v2:172.21.15.139:3300/0 conn(0x7f16740c8a10 msgr2=0x7f16740c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f167259c640 1 --2- 172.21.15.135:0/398482599 >> v2:172.21.15.139:3300/0 conn(0x7f16740c8a10 0x7f16740c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f167259c640 1 -- 172.21.15.135:0/398482599 >> v1:172.21.15.139:6789/0 conn(0x7f16740cbd80 legacy=0x7f1674157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f167259c640 1 -- 172.21.15.135:0/398482599 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167406c4b0 con 0x7f16741543b0 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.886+0000 7f167259c640 1 --2- 172.21.15.135:0/398482599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16741543b0 0x7f16740ca2b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f16740bfe10 tx=0x7f166402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c09cae49daa40cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1671d9b640 1 -- 172.21.15.135:0/398482599 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664037070 con 0x7f16741543b0 2024-04-26T22:10:20.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1671d9b640 1 -- 172.21.15.135:0/398482599 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1664032440 con 0x7f16741543b0 2024-04-26T22:10:20.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1671d9b640 1 -- 172.21.15.135:0/398482599 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664032810 con 0x7f16741543b0 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 -- 172.21.15.135:0/398482599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16741543b0 msgr2=0x7f16740ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 --2- 172.21.15.135:0/398482599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16741543b0 0x7f16740ca2b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f16740bfe10 tx=0x7f166402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 -- 172.21.15.135:0/398482599 shutdown_connections 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 --2- 172.21.15.135:0/398482599 >> v2:172.21.15.139:3300/0 conn(0x7f16740c8a10 0x7f16740c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 --2- 172.21.15.135:0/398482599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16741543b0 0x7f16740ca2b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 -- 172.21.15.135:0/398482599 >> 172.21.15.135:0/398482599 conn(0x7f16740c8e30 msgr2=0x7f16740c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 -- 172.21.15.135:0/398482599 shutdown_connections 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.887+0000 7f1678b6a640 1 -- 172.21.15.135:0/398482599 wait complete. 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 Processor -- start 2024-04-26T22:10:20.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 -- start start 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16740c8a10 0x7f16740aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16741543b0 0x7f167414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16740b1580 0x7f16741ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1674156d20 con 0x7f16740c8a10 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1674157070 con 0x7f16740b1580 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f1678b6a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f16740c7490 con 0x7f16741543b0 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f167259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16740c8a10 0x7f16740aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f167259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16740c8a10 0x7f16740aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51596/0 (socket says 172.21.15.135:51596) 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.889+0000 7f167259c640 1 -- 172.21.15.135:0/2255837095 learned_addr learned my addr 172.21.15.135:0/2255837095 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f167259c640 1 -- 172.21.15.135:0/2255837095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16741543b0 msgr2=0x7f167414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f166bfff640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16741543b0 0x7f167414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f1672d9d640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16740b1580 0x7f16741ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f167259c640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16741543b0 0x7f167414f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f167259c640 1 -- 172.21.15.135:0/2255837095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16740b1580 msgr2=0x7f16741ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f167259c640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16740b1580 0x7f16741ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f167259c640 1 -- 172.21.15.135:0/2255837095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16740b1fd0 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f167259c640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16740c8a10 0x7f16740aef60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1664000c00 tx=0x7f16640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664037040 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1664042070 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1674151290 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664030bb0 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.890+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16741ec760 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.891+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16741ecb40 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1664047460 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166affd640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f16640ada10 con 0x7f16740c8a10 2024-04-26T22:10:20.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166bfff640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166affd640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7f1654077d10 0x7f165407a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 --> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1664032440 con 0x7f1654077d10 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f16640adea0 con 0x7f16740c8a10 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f1672d9d640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7f1654077d10 0x7f165407a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.892+0000 7f166bfff640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f165406c8f0 0x7f165406edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16740c30c0 tx=0x7f165c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.893+0000 7f1672d9d640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7f1654077d10 0x7f165407a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:20.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.893+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== osd.3 v2:172.21.15.135:6800/3564981873 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1664032440 con 0x7f1654077d10 2024-04-26T22:10:20.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.906+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 --> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f16741519f0 con 0x7f1654077d10 2024-04-26T22:10:20.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.906+0000 7f166affd640 1 -- 172.21.15.135:0/2255837095 <== osd.3 v2:172.21.15.135:6800/3564981873 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f16741519f0 con 0x7f1654077d10 2024-04-26T22:10:20.907 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:20.907 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:20.907 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7f1654077d10 msgr2=0x7f165407a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6800/3564981873,v1:172.21.15.135:6801/3564981873] conn(0x7f1654077d10 0x7f165407a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f165406c8f0 msgr2=0x7f165406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f165406c8f0 0x7f165406edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16740c30c0 tx=0x7f165c009040 comp rx=0 tx=0).stop 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16740c8a10 msgr2=0x7f16740aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 --2- 172.21.15.135:0/2255837095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16740c8a10 0x7f16740aef60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1664000c00 tx=0x7f16640046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1672d9d640 1 -- 172.21.15.135:0/2255837095 reap_dead start 2024-04-26T22:10:20.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 shutdown_connections 2024-04-26T22:10:20.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 >> 172.21.15.135:0/2255837095 conn(0x7f16740c8e30 msgr2=0x7f16740cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:20.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 shutdown_connections 2024-04-26T22:10:20.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:20.907+0000 7f1678b6a640 1 -- 172.21.15.135:0/2255837095 wait complete. 2024-04-26T22:10:20.917 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T22:10:21.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.015+0000 7fe490efc640 1 Processor -- start 2024-04-26T22:10:21.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.016+0000 7fe490efc640 1 -- start start 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.016+0000 7fe490efc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 0x7fe48c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.016+0000 7fe490efc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c15a1a0 0x7fe48c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.016+0000 7fe490efc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe48c0c8df0 con 0x7fe48c154160 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.016+0000 7fe490efc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe48c0c9100 con 0x7fe48c0ca5a0 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.016+0000 7fe490efc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe48c154540 con 0x7fe48c15a1a0 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe483fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 0x7fe48c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe483fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 0x7fe48c156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:50348/0 (socket says 172.21.15.135:50348) 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe48ad9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c15a1a0 0x7fe48c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe48a59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe48c154160 0x7fe48c0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:56066/0 (socket says 172.21.15.135:56066) 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe483fff640 1 -- 172.21.15.135:0/1626596124 learned_addr learned my addr 172.21.15.135:0/1626596124 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe48ad9d640 1 -- 172.21.15.135:0/1626596124 >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 msgr2=0x7fe48c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe48ad9d640 1 --2- 172.21.15.135:0/1626596124 >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 0x7fe48c156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe48ad9d640 1 -- 172.21.15.135:0/1626596124 >> v1:172.21.15.139:6789/0 conn(0x7fe48c154160 legacy=0x7fe48c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe48ad9d640 1 -- 172.21.15.135:0/1626596124 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c15a910 con 0x7fe48c15a1a0 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.017+0000 7fe483fff640 1 --2- 172.21.15.135:0/1626596124 >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 0x7fe48c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:21.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.018+0000 7fe48ad9d640 1 --2- 172.21.15.135:0/1626596124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c15a1a0 0x7fe48c0c9590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe484009d90 tx=0x7fe48402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a942e2b628ec78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.020 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe489d9b640 1 -- 172.21.15.135:0/1626596124 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484007740 con 0x7fe48c15a1a0 2024-04-26T22:10:21.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe489d9b640 1 -- 172.21.15.135:0/1626596124 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4840078e0 con 0x7fe48c15a1a0 2024-04-26T22:10:21.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe489d9b640 1 -- 172.21.15.135:0/1626596124 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484007cb0 con 0x7fe48c15a1a0 2024-04-26T22:10:21.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 -- 172.21.15.135:0/1626596124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c15a1a0 msgr2=0x7fe48c0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 --2- 172.21.15.135:0/1626596124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c15a1a0 0x7fe48c0c9590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe484009d90 tx=0x7fe48402a660 comp rx=0 tx=0).stop 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 -- 172.21.15.135:0/1626596124 shutdown_connections 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 --2- 172.21.15.135:0/1626596124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c15a1a0 0x7fe48c0c9590 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 --2- 172.21.15.135:0/1626596124 >> v2:172.21.15.139:3300/0 conn(0x7fe48c0ca5a0 0x7fe48c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 -- 172.21.15.135:0/1626596124 >> 172.21.15.135:0/1626596124 conn(0x7fe48c0ae010 msgr2=0x7fe48c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 -- 172.21.15.135:0/1626596124 shutdown_connections 2024-04-26T22:10:21.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.021+0000 7fe490efc640 1 -- 172.21.15.135:0/1626596124 wait complete. 2024-04-26T22:10:21.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 Processor -- start 2024-04-26T22:10:21.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 -- start start 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c0ca5a0 0x7fe48c0c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48c154160 0x7fe48c1ec860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 0x7fe48c1ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c15a5f0 con 0x7fe48c15a1a0 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c15aed0 con 0x7fe48c154160 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.023+0000 7fe490efc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe48c0c7510 con 0x7fe48c0ca5a0 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 0x7fe48c1ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 0x7fe48c1ecda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51598/0 (socket says 172.21.15.135:51598) 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 -- 172.21.15.135:0/3865812012 learned_addr learned my addr 172.21.15.135:0/3865812012 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48a59c640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c0ca5a0 0x7fe48c0c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe483fff640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48c154160 0x7fe48c1ec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 -- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c0ca5a0 msgr2=0x7fe48c0c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c0ca5a0 0x7fe48c0c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 -- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48c154160 msgr2=0x7fe48c1ec860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48c154160 0x7fe48c1ec860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 -- 172.21.15.135:0/3865812012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c159c80 con 0x7fe48c15a1a0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe48ad9d640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 0x7fe48c1ecda0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe484009eb0 tx=0x7fe484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48403f070 con 0x7fe48c15a1a0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe484031c60 con 0x7fe48c15a1a0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48c151eb0 con 0x7fe48c15a1a0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484009450 con 0x7fe48c15a1a0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.024+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48c0c2b10 con 0x7fe48c15a1a0 2024-04-26T22:10:21.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.025+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48c076650 con 0x7fe48c15a1a0 2024-04-26T22:10:21.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.026+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe48403b410 con 0x7fe48c15a1a0 2024-04-26T22:10:21.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.026+0000 7fe482ffd640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe46c06c8f0 0x7fe46c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.026+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fe4840ad630 con 0x7fe48c15a1a0 2024-04-26T22:10:21.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.026+0000 7fe48a59c640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe46c06c8f0 0x7fe46c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.026+0000 7fe48a59c640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe46c06c8f0 0x7fe46c06edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe47c007fd0 tx=0x7fe47c007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.030+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe48407ac30 con 0x7fe48c15a1a0 2024-04-26T22:10:21.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.214+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe48c0507a0 con 0x7fe46c06c8f0 2024-04-26T22:10:21.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.215+0000 7fe482ffd640 1 -- 172.21.15.135:0/3865812012 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe48c0507a0 con 0x7fe46c06c8f0 2024-04-26T22:10:21.215 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe46c06c8f0 msgr2=0x7fe46c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe46c06c8f0 0x7fe46c06edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe47c007fd0 tx=0x7fe47c007e80 comp rx=0 tx=0).stop 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 msgr2=0x7fe48c1ecda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 0x7fe48c1ecda0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe484009eb0 tx=0x7fe484004a40 comp rx=0 tx=0).stop 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 shutdown_connections 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fe46c06c8f0 0x7fe46c06edb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe48c15a1a0 0x7fe48c1ecda0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48c154160 0x7fe48c1ec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 --2- 172.21.15.135:0/3865812012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe48c0ca5a0 0x7fe48c0c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 >> 172.21.15.135:0/3865812012 conn(0x7fe48c0ae010 msgr2=0x7fe48c1507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 shutdown_connections 2024-04-26T22:10:21.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.217+0000 7fe490efc640 1 -- 172.21.15.135:0/3865812012 wait complete. 2024-04-26T22:10:21.228 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T22:10:21.228 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:21.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.330+0000 7f41847da640 1 Processor -- start 2024-04-26T22:10:21.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.330+0000 7f41847da640 1 -- start start 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f41847da640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f417c0ac560 0x7f417c073960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f41847da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f417c0ab470 0x7f417c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f41847da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c070b60 0x7f417c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f41847da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f417c0ab9a0 con 0x7f417c0ab470 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f41847da640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f417c070660 con 0x7f417c0ac560 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f41847da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f417c0abbf0 con 0x7f417c070b60 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f417c0ab470 0x7f417c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f417c0ab470 0x7f417c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51612/0 (socket says 172.21.15.135:51612) 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 -- 172.21.15.135:0/1145649751 learned_addr learned my addr 172.21.15.135:0/1145649751 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4182d77640 1 --2- 172.21.15.135:0/1145649751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c070b60 0x7f417c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4182576640 1 --2- 172.21.15.135:0/1145649751 >> v2:172.21.15.139:3300/0 conn(0x7f417c0ac560 0x7f417c073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 -- 172.21.15.135:0/1145649751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c070b60 msgr2=0x7f417c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 --2- 172.21.15.135:0/1145649751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c070b60 0x7f417c153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 -- 172.21.15.135:0/1145649751 >> v2:172.21.15.139:3300/0 conn(0x7f417c0ac560 msgr2=0x7f417c073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 --2- 172.21.15.135:0/1145649751 >> v2:172.21.15.139:3300/0 conn(0x7f417c0ac560 0x7f417c073960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.331+0000 7f4181d75640 1 -- 172.21.15.135:0/1145649751 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c0abed0 con 0x7f417c0ab470 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.332+0000 7f4181d75640 1 --2- 172.21.15.135:0/1145649751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c0ab470 0x7f417c156ad0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ca250 tx=0x7f416c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589fb216d64ca127 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.332+0000 7f4181574640 1 -- 172.21.15.135:0/1145649751 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c037070 con 0x7f417c0ab470 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.332+0000 7f4181574640 1 -- 172.21.15.135:0/1145649751 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c02ae00 con 0x7f417c0ab470 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.332+0000 7f4181574640 1 -- 172.21.15.135:0/1145649751 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c032660 con 0x7f417c0ab470 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.332+0000 7f41847da640 1 -- 172.21.15.135:0/1145649751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c0ab470 msgr2=0x7f417c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.332+0000 7f41847da640 1 --2- 172.21.15.135:0/1145649751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c0ab470 0x7f417c156ad0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ca250 tx=0x7f416c02a0b0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 -- 172.21.15.135:0/1145649751 shutdown_connections 2024-04-26T22:10:21.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 --2- 172.21.15.135:0/1145649751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c0ab470 0x7f417c156ad0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 --2- 172.21.15.135:0/1145649751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c070b60 0x7f417c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 --2- 172.21.15.135:0/1145649751 >> v2:172.21.15.139:3300/0 conn(0x7f417c0ac560 0x7f417c073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 -- 172.21.15.135:0/1145649751 >> 172.21.15.135:0/1145649751 conn(0x7f417c0ca2f0 msgr2=0x7f417c0ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 -- 172.21.15.135:0/1145649751 shutdown_connections 2024-04-26T22:10:21.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.333+0000 7f41847da640 1 -- 172.21.15.135:0/1145649751 wait complete. 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 Processor -- start 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 -- start start 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c070b60 0x7f417c0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f417c0ab470 0x7f417c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c0ac560 0x7f417c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f417c0acac0 con 0x7f417c070b60 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f417c0ac940 con 0x7f417c0ab470 2024-04-26T22:10:21.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f41847da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f417c070fd0 con 0x7f417c0ac560 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f4182576640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c070b60 0x7f417c0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f4182576640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c070b60 0x7f417c0adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51622/0 (socket says 172.21.15.135:51622) 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.334+0000 7f4182576640 1 -- 172.21.15.135:0/1537177269 learned_addr learned my addr 172.21.15.135:0/1537177269 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4182576640 1 -- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c0ac560 msgr2=0x7f417c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4182576640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417c0ac560 0x7f417c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4182576640 1 -- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f417c0ab470 msgr2=0x7f417c1efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4182576640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f417c0ab470 0x7f417c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4182576640 1 -- 172.21.15.135:0/1537177269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c06f5e0 con 0x7f417c070b60 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4182576640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c070b60 0x7f417c0adb00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f417800c7f0 tx=0x7f417800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178015070 con 0x7f417c070b60 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4178004220 con 0x7f417c070b60 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c1f0340 con 0x7f417c070b60 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178003610 con 0x7f417c070b60 2024-04-26T22:10:21.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.335+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c15aa30 con 0x7f417c070b60 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.336+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f417c06ee40 con 0x7f417c070b60 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.336+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41780039c0 con 0x7f417c070b60 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4172ffd640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f417808c680 con 0x7f417c070b60 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4181d75640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4172ffd640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f4160077d10 0x7f416007a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 --> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f417801be50 con 0x7f4160077d10 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4178093050 con 0x7f417c070b60 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4181d75640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416006c8f0 0x7f416006edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f416c034000 tx=0x7f416c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.337+0000 7f4182d77640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f4160077d10 0x7f416007a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.338+0000 7f4182d77640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f4160077d10 0x7f416007a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.339+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== osd.4 v2:172.21.15.139:6808/3666291794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f417801be50 con 0x7f4160077d10 2024-04-26T22:10:21.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.351+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 --> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f417c0694b0 con 0x7f4160077d10 2024-04-26T22:10:21.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f4172ffd640 1 -- 172.21.15.135:0/1537177269 <== osd.4 v2:172.21.15.139:6808/3666291794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f417c0694b0 con 0x7f4160077d10 2024-04-26T22:10:21.352 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:21.352 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:21.352 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:21.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f4160077d10 msgr2=0x7f416007a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.139:6808/3666291794,v1:172.21.15.139:6809/3666291794] conn(0x7f4160077d10 0x7f416007a1b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416006c8f0 msgr2=0x7f416006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f416006c8f0 0x7f416006edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f416c034000 tx=0x7f416c02c000 comp rx=0 tx=0).stop 2024-04-26T22:10:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c070b60 msgr2=0x7f417c0adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 --2- 172.21.15.135:0/1537177269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f417c070b60 0x7f417c0adb00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f417800c7f0 tx=0x7f417800ccb0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f4182d77640 1 -- 172.21.15.135:0/1537177269 reap_dead start 2024-04-26T22:10:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 shutdown_connections 2024-04-26T22:10:21.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 >> 172.21.15.135:0/1537177269 conn(0x7f417c0ca2f0 msgr2=0x7f417c154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 shutdown_connections 2024-04-26T22:10:21.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.352+0000 7f41847da640 1 -- 172.21.15.135:0/1537177269 wait complete. 2024-04-26T22:10:21.362 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T22:10:21.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.461+0000 7fb0e2a64640 1 Processor -- start 2024-04-26T22:10:21.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.461+0000 7fb0e2a64640 1 -- start start 2024-04-26T22:10:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e2a64640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb0dc13fe20 0x7fb0dc0c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e2a64640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0dc15a230 con 0x7fb0dc157010 2024-04-26T22:10:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e2a64640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0dc15a510 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e2a64640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb0dc0c39b0 con 0x7fb0dc13fe20 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e1001640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb0dc157010 0x7fb0dc157440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46850/0 (socket says 172.21.15.135:46850) 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e1001640 1 -- 172.21.15.135:0/1921979761 learned_addr learned my addr 172.21.15.135:0/1921979761 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.462+0000 7fb0e0800640 1 --2- 172.21.15.135:0/1921979761 >> v2:172.21.15.139:3300/0 conn(0x7fb0dc13fe20 0x7fb0dc0c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2057654516 0 0) 0x7fb0dc15a230 con 0x7fb0dc157010 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0cc003520 con 0x7fb0dc157010 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558597765 0 0) 0x7fb0dc15a510 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0dc15a230 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766867305 0 0) 0x7fb0cc003520 con 0x7fb0dc157010 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0e0800640 1 -- 172.21.15.135:0/1921979761 >> v1:172.21.15.139:6789/0 conn(0x7fb0dc0c3c70 legacy=0x7fb0dc06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0e0800640 1 -- 172.21.15.135:0/1921979761 >> v1:172.21.15.135:6789/0 conn(0x7fb0dc157010 legacy=0x7fb0dc157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.463+0000 7fb0e0800640 1 -- 172.21.15.135:0/1921979761 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc15a7f0 con 0x7fb0dc13fe20 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.464+0000 7fb0e0800640 1 --2- 172.21.15.135:0/1921979761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc13fe20 0x7fb0dc0c5430 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00097a0 tx=0x7fb0d002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67049cb023850cca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.464+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0037070 con 0x7fb0dc13fe20 2024-04-26T22:10:21.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.464+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d0032440 con 0x7fb0dc13fe20 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.464+0000 7fb0db7fe640 1 -- 172.21.15.135:0/1921979761 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0032810 con 0x7fb0dc13fe20 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 -- 172.21.15.135:0/1921979761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc13fe20 msgr2=0x7fb0dc0c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/1921979761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc13fe20 0x7fb0dc0c5430 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00097a0 tx=0x7fb0d002a230 comp rx=0 tx=0).stop 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 -- 172.21.15.135:0/1921979761 shutdown_connections 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/1921979761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc13fe20 0x7fb0dc0c5430 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 -- 172.21.15.135:0/1921979761 >> 172.21.15.135:0/1921979761 conn(0x7fb0dc0c5d70 msgr2=0x7fb0dc0c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 -- 172.21.15.135:0/1921979761 shutdown_connections 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.465+0000 7fb0e2a64640 1 -- 172.21.15.135:0/1921979761 wait complete. 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.466+0000 7fb0e2a64640 1 Processor -- start 2024-04-26T22:10:21.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.466+0000 7fb0e2a64640 1 -- start start 2024-04-26T22:10:21.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e2a64640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 0x7fb0dc0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e2a64640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0dc13fe20 0x7fb0dc13ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e2a64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc157010 0x7fb0dc0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e2a64640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc15adb0 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e2a64640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc15b120 con 0x7fb0dc157010 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e2a64640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc157b50 con 0x7fb0dc13fe20 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 0x7fb0dc0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 0x7fb0dc0ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51632/0 (socket says 172.21.15.135:51632) 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 -- 172.21.15.135:0/367928235 learned_addr learned my addr 172.21.15.135:0/367928235 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e1001640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc157010 0x7fb0dc0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0dbfff640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0dc13fe20 0x7fb0dc13ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 -- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0dc13fe20 msgr2=0x7fb0dc13ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0dc13fe20 0x7fb0dc13ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 -- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc157010 msgr2=0x7fb0dc0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc157010 0x7fb0dc0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.467+0000 7fb0e0800640 1 -- 172.21.15.135:0/367928235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc0747d0 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.468+0000 7fb0e0800640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 0x7fb0dc0ac160 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0035040 tx=0x7fb0d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.468+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0041030 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.468+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d003d4e0 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.468+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc1ec760 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.468+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0037040 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.468+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc13aa30 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.469+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0dc069da0 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.469+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0d0044070 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.470+0000 7fb0d97fa640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.470+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fb0d00ac810 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.470+0000 7fb0dbfff640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.471+0000 7fb0dbfff640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0c006c8f0 0x7fb0c006edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0708a0 tx=0x7fb0c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.474+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0dc069da0 con 0x7fb0dc0c3c70 2024-04-26T22:10:21.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.649+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fb0dc154910 con 0x7fb0c006c8f0 2024-04-26T22:10:21.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.650+0000 7fb0d97fa640 1 -- 172.21.15.135:0/367928235 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb0dc154910 con 0x7fb0c006c8f0 2024-04-26T22:10:21.650 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T22:10:21.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.651+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0c006c8f0 msgr2=0x7fb0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.651+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0c006c8f0 0x7fb0c006edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0708a0 tx=0x7fb0c8009040 comp rx=0 tx=0).stop 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.651+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 msgr2=0x7fb0dc0ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.651+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 0x7fb0dc0ac160 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0035040 tx=0x7fb0d00046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 shutdown_connections 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0dc157010 0x7fb0dc0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0dc13fe20 0x7fb0dc13ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 --2- 172.21.15.135:0/367928235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0dc0c3c70 0x7fb0dc0ac160 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 >> 172.21.15.135:0/367928235 conn(0x7fb0dc0c5d70 msgr2=0x7fb0dc158890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 shutdown_connections 2024-04-26T22:10:21.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.652+0000 7fb0e2a64640 1 -- 172.21.15.135:0/367928235 wait complete. 2024-04-26T22:10:21.663 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T22:10:21.663 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:21.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.764+0000 7f55eec85640 1 Processor -- start 2024-04-26T22:10:21.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.764+0000 7f55eec85640 1 -- start start 2024-04-26T22:10:21.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55eec85640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f55e8070490 0x7f55e806c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55eec85640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55e80c3f90 0x7f55e8075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55eec85640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e806d920 con 0x7f55e80756f0 2024-04-26T22:10:21.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55eec85640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f55e815ace0 con 0x7f55e80c3f90 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55eec85640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f55e815b260 con 0x7f55e8070490 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55e80c3f90 0x7f55e8075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55e80c3f90 0x7f55e8075ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51644/0 (socket says 172.21.15.135:51644) 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55eca21640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f55e80756f0 0x7f55e8156d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46852/0 (socket says 172.21.15.135:46852) 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 -- 172.21.15.135:0/3631842292 learned_addr learned my addr 172.21.15.135:0/3631842292 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 -- 172.21.15.135:0/3631842292 >> v2:172.21.15.139:3300/0 conn(0x7f55e8070490 msgr2=0x7f55e806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55e7fff640 1 --2- 172.21.15.135:0/3631842292 >> v2:172.21.15.139:3300/0 conn(0x7f55e8070490 0x7f55e806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 --2- 172.21.15.135:0/3631842292 >> v2:172.21.15.139:3300/0 conn(0x7f55e8070490 0x7f55e806c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55e77fe640 1 -- 172.21.15.135:0/3631842292 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077014723 0 0) 0x7f55e806d920 con 0x7f55e80756f0 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 -- 172.21.15.135:0/3631842292 >> v1:172.21.15.135:6789/0 conn(0x7f55e80756f0 legacy=0x7f55e8156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55ed222640 1 -- 172.21.15.135:0/3631842292 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e806db90 con 0x7f55e80c3f90 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.765+0000 7f55e7fff640 1 --2- 172.21.15.135:0/3631842292 >> v2:172.21.15.139:3300/0 conn(0x7f55e8070490 0x7f55e806c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:10:21.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55ed222640 1 --2- 172.21.15.135:0/3631842292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80c3f90 0x7f55e8075ad0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f55e80c4490 tx=0x7f55e002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3149d0d305330bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55e77fe640 1 -- 172.21.15.135:0/3631842292 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0036070 con 0x7f55e80c3f90 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55e77fe640 1 -- 172.21.15.135:0/3631842292 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e0031600 con 0x7f55e80c3f90 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55e77fe640 1 -- 172.21.15.135:0/3631842292 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00319d0 con 0x7f55e80c3f90 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 -- 172.21.15.135:0/3631842292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80c3f90 msgr2=0x7f55e8075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 --2- 172.21.15.135:0/3631842292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80c3f90 0x7f55e8075ad0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f55e80c4490 tx=0x7f55e002a730 comp rx=0 tx=0).stop 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 -- 172.21.15.135:0/3631842292 shutdown_connections 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 --2- 172.21.15.135:0/3631842292 >> v2:172.21.15.139:3300/0 conn(0x7f55e8070490 0x7f55e806c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 --2- 172.21.15.135:0/3631842292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80c3f90 0x7f55e8075ad0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 -- 172.21.15.135:0/3631842292 >> 172.21.15.135:0/3631842292 conn(0x7f55e80c5ce0 msgr2=0x7f55e80c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 -- 172.21.15.135:0/3631842292 shutdown_connections 2024-04-26T22:10:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.766+0000 7f55eec85640 1 -- 172.21.15.135:0/3631842292 wait complete. 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 Processor -- start 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 -- start start 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55e8070490 0x7f55e80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80756f0 0x7f55e814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55e80c3f90 0x7f55e8149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8154640 con 0x7f55e80756f0 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80c4680 con 0x7f55e80c3f90 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55eec85640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f55e81547c0 con 0x7f55e8070490 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55e7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80756f0 0x7f55e814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55e7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80756f0 0x7f55e814eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51650/0 (socket says 172.21.15.135:51650) 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.768+0000 7f55e7fff640 1 -- 172.21.15.135:0/2546457464 learned_addr learned my addr 172.21.15.135:0/2546457464 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55eca21640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55e8070490 0x7f55e80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55ed222640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55e80c3f90 0x7f55e8149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e7fff640 1 -- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55e8070490 msgr2=0x7f55e80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e7fff640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55e8070490 0x7f55e80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e7fff640 1 -- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55e80c3f90 msgr2=0x7f55e8149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e7fff640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55e80c3f90 0x7f55e8149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e7fff640 1 -- 172.21.15.135:0/2546457464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e8073b30 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55ed222640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55e80c3f90 0x7f55e8149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e7fff640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80756f0 0x7f55e814eb50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c7f0 tx=0x7f55d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4015070 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d4004220 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007830 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e8073e00 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.769+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e80ad9b0 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.770+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55e8068dc0 con 0x7f55e80756f0 2024-04-26T22:10:21.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.770+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55d40079d0 con 0x7f55e80756f0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55e57fa640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f55cc06c8f0 0x7f55cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55eca21640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f55cc06c8f0 0x7f55cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f55d408c270 con 0x7f55e80756f0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55e57fa640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f55cc077d10 0x7f55cc07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 --> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55cc07a880 con 0x7f55cc077d10 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f55d4093050 con 0x7f55e80756f0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55eca21640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f55cc06c8f0 0x7f55cc06edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000f60 tx=0x7f55dc007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.771+0000 7f55ed222640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f55cc077d10 0x7f55cc07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.772+0000 7f55ed222640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f55cc077d10 0x7f55cc07a1b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.772+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== osd.5 v2:172.21.15.139:6800/2023089101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55cc07a880 con 0x7f55cc077d10 2024-04-26T22:10:21.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.785+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 --> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f55e80694b0 con 0x7f55cc077d10 2024-04-26T22:10:21.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55e57fa640 1 -- 172.21.15.135:0/2546457464 <== osd.5 v2:172.21.15.139:6800/2023089101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f55e80694b0 con 0x7f55cc077d10 2024-04-26T22:10:21.786 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:21.786 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:21.786 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:21.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f55cc077d10 msgr2=0x7f55cc07a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.139:6800/2023089101,v1:172.21.15.139:6801/2023089101] conn(0x7f55cc077d10 0x7f55cc07a1b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f55cc06c8f0 msgr2=0x7f55cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f55cc06c8f0 0x7f55cc06edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000f60 tx=0x7f55dc007b50 comp rx=0 tx=0).stop 2024-04-26T22:10:21.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80756f0 msgr2=0x7f55e814eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 --2- 172.21.15.135:0/2546457464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55e80756f0 0x7f55e814eb50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c7f0 tx=0x7f55d400ccb0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55ed222640 1 -- 172.21.15.135:0/2546457464 reap_dead start 2024-04-26T22:10:21.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 shutdown_connections 2024-04-26T22:10:21.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 >> 172.21.15.135:0/2546457464 conn(0x7f55e80c5ce0 msgr2=0x7f55e806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 shutdown_connections 2024-04-26T22:10:21.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.786+0000 7f55eec85640 1 -- 172.21.15.135:0/2546457464 wait complete. 2024-04-26T22:10:21.797 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T22:10:21.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.895+0000 7f56607af640 1 Processor -- start 2024-04-26T22:10:21.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.895+0000 7f56607af640 1 -- start start 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f56607af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56581523e0 0x7f56581554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f56607af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5658070670 con 0x7f565806f920 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f56607af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56580700b0 con 0x7f5658073aa0 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f56607af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f56580ab8f0 con 0x7f56581523e0 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565dd4a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56581523e0 0x7f56581554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565ed4c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f565806f920 0x7f56580abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46854/0 (socket says 172.21.15.135:46854) 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565ed4c640 1 -- 172.21.15.135:0/3327623711 learned_addr learned my addr 172.21.15.135:0/3327623711 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565dd4a640 1 -- 172.21.15.135:0/3327623711 >> v1:172.21.15.139:6789/0 conn(0x7f5658073aa0 legacy=0x7f5658151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565dd4a640 1 -- 172.21.15.135:0/3327623711 >> v1:172.21.15.135:6789/0 conn(0x7f565806f920 legacy=0x7f56580abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565d549640 1 -- 172.21.15.135:0/3327623711 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030952868 0 0) 0x7f5658070670 con 0x7f565806f920 2024-04-26T22:10:21.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.896+0000 7f565dd4a640 1 -- 172.21.15.135:0/3327623711 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5658070390 con 0x7f56581523e0 2024-04-26T22:10:21.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f565dd4a640 1 --2- 172.21.15.135:0/3327623711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56581523e0 0x7f56581554c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f56480097a0 tx=0x7f564802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc47b12d1c70d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f565d549640 1 -- 172.21.15.135:0/3327623711 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648037070 con 0x7f56581523e0 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f565d549640 1 -- 172.21.15.135:0/3327623711 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648032440 con 0x7f56581523e0 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f565d549640 1 -- 172.21.15.135:0/3327623711 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648032810 con 0x7f56581523e0 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 -- 172.21.15.135:0/3327623711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56581523e0 msgr2=0x7f56581554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 --2- 172.21.15.135:0/3327623711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56581523e0 0x7f56581554c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f56480097a0 tx=0x7f564802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 -- 172.21.15.135:0/3327623711 shutdown_connections 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 --2- 172.21.15.135:0/3327623711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56581523e0 0x7f56581554c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 -- 172.21.15.135:0/3327623711 >> 172.21.15.135:0/3327623711 conn(0x7f56580c6ed0 msgr2=0x7f56580c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 -- 172.21.15.135:0/3327623711 shutdown_connections 2024-04-26T22:10:21.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.897+0000 7f56607af640 1 -- 172.21.15.135:0/3327623711 wait complete. 2024-04-26T22:10:21.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.898+0000 7f56607af640 1 Processor -- start 2024-04-26T22:10:21.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 -- start start 2024-04-26T22:10:21.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 0x7f56580c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5658073aa0 0x7f56580c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56581523e0 0x7f56581ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5658073e80 con 0x7f565806f920 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f565806fdb0 con 0x7f5658073aa0 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f56607af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5658074000 con 0x7f56581523e0 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 0x7f56580c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 0x7f56580c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51674/0 (socket says 172.21.15.135:51674) 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 -- 172.21.15.135:0/1632850372 learned_addr learned my addr 172.21.15.135:0/1632850372 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 -- 172.21.15.135:0/1632850372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56581523e0 msgr2=0x7f56581ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56581523e0 0x7f56581ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 -- 172.21.15.135:0/1632850372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5658073aa0 msgr2=0x7f56580c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.899+0000 7f565e54b640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5658073aa0 0x7f56580c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f565e54b640 1 -- 172.21.15.135:0/1632850372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565814a990 con 0x7f565806f920 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f565e54b640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 0x7f56580c9dc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f565400c460 tx=0x7f565400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654016020 con 0x7f565806f920 2024-04-26T22:10:21.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5654012870 con 0x7f565806f920 2024-04-26T22:10:21.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654012c60 con 0x7f565806f920 2024-04-26T22:10:21.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56581ec350 con 0x7f565806f920 2024-04-26T22:10:21.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.900+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56580c3fb0 con 0x7f565806f920 2024-04-26T22:10:21.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.901+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56581ec160 con 0x7f565806f920 2024-04-26T22:10:21.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.901+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5654019020 con 0x7f565806f920 2024-04-26T22:10:21.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.902+0000 7f564effd640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f563c06c8f0 0x7f563c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:21.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.902+0000 7f565dd4a640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f563c06c8f0 0x7f563c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:21.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.902+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f565408c010 con 0x7f565806f920 2024-04-26T22:10:21.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.902+0000 7f565dd4a640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f563c06c8f0 0x7f563c06edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5648034000 tx=0x7f564802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:21.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:21.906+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5654020030 con 0x7f565806f920 2024-04-26T22:10:22.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.087+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f565814a740 con 0x7f563c06c8f0 2024-04-26T22:10:22.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.087+0000 7f564effd640 1 -- 172.21.15.135:0/1632850372 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f565814a740 con 0x7f563c06c8f0 2024-04-26T22:10:22.087 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f563c06c8f0 msgr2=0x7f563c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f563c06c8f0 0x7f563c06edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5648034000 tx=0x7f564802c000 comp rx=0 tx=0).stop 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 msgr2=0x7f56580c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 0x7f56580c9dc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f565400c460 tx=0x7f565400c920 comp rx=0 tx=0).stop 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 shutdown_connections 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f563c06c8f0 0x7f563c06edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56581523e0 0x7f56581ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5658073aa0 0x7f56580c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 --2- 172.21.15.135:0/1632850372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f565806f920 0x7f56580c9dc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 >> 172.21.15.135:0/1632850372 conn(0x7f56580c6ed0 msgr2=0x7f5658154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 shutdown_connections 2024-04-26T22:10:22.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.090+0000 7f56607af640 1 -- 172.21.15.135:0/1632850372 wait complete. 2024-04-26T22:10:22.100 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T22:10:22.101 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:22.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.203+0000 7fcdbd16a640 1 Processor -- start 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.203+0000 7fcdbd16a640 1 -- start start 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdbd16a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb813ffc0 0x7fcdb815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdbd16a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb815a320 con 0x7fcdb8070cf0 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdbd16a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb80c6b80 con 0x7fcdb813fa70 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdbd16a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb815a930 con 0x7fcdb813ffc0 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdb6d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fcdb8070cf0 0x7fcdb8156d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46858/0 (socket says 172.21.15.135:46858) 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdb6d9d640 1 -- 172.21.15.135:0/1806107833 learned_addr learned my addr 172.21.15.135:0/1806107833 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:22.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdaffff640 1 --2- 172.21.15.135:0/1806107833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb813ffc0 0x7fcdb815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889131243 0 0) 0x7fcdb815a320 con 0x7fcdb8070cf0 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.204+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcda4003520 con 0x7fcdb8070cf0 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290524597 0 0) 0x7fcdb80c6b80 con 0x7fcdb813fa70 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdb815a320 con 0x7fcdb813fa70 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1974900828 0 0) 0x7fcda4003520 con 0x7fcdb8070cf0 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdaffff640 1 -- 172.21.15.135:0/1806107833 >> v1:172.21.15.139:6789/0 conn(0x7fcdb813fa70 legacy=0x7fcdb806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdaffff640 1 -- 172.21.15.135:0/1806107833 >> v1:172.21.15.135:6789/0 conn(0x7fcdb8070cf0 legacy=0x7fcdb8156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdaffff640 1 -- 172.21.15.135:0/1806107833 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb80c2ba0 con 0x7fcdb813ffc0 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.205+0000 7fcdaffff640 1 --2- 172.21.15.135:0/1806107833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb813ffc0 0x7fcdb815adc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097d0 tx=0x7fcda002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beeb17277cd0b8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0037070 con 0x7fcdb813ffc0 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda002acc0 con 0x7fcdb813ffc0 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdb659c640 1 -- 172.21.15.135:0/1806107833 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8001670 con 0x7fcdb813ffc0 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1806107833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb813ffc0 msgr2=0x7fcdb815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 --2- 172.21.15.135:0/1806107833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb813ffc0 0x7fcdb815adc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097d0 tx=0x7fcda002a230 comp rx=0 tx=0).stop 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1806107833 shutdown_connections 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 --2- 172.21.15.135:0/1806107833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb813ffc0 0x7fcdb815adc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1806107833 >> 172.21.15.135:0/1806107833 conn(0x7fcdb80c5ce0 msgr2=0x7fcdb80c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1806107833 shutdown_connections 2024-04-26T22:10:22.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.206+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1806107833 wait complete. 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 Processor -- start 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 -- start start 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb8070cf0 0x7fcdb813e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb813fa70 0x7fcdb80ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdb813ffc0 0x7fcdb80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8157420 con 0x7fcdb813fa70 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb80710d0 con 0x7fcdb813ffc0 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdbd16a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb80c2e80 con 0x7fcdb8070cf0 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb813fa70 0x7fcdb80ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb813fa70 0x7fcdb80ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51688/0 (socket says 172.21.15.135:51688) 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.208+0000 7fcdaffff640 1 -- 172.21.15.135:0/1586686411 learned_addr learned my addr 172.21.15.135:0/1586686411 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaffff640 1 -- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb8070cf0 msgr2=0x7fcdb813e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaffff640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdb8070cf0 0x7fcdb813e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaffff640 1 -- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdb813ffc0 msgr2=0x7fcdb80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaffff640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdb813ffc0 0x7fcdb80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaffff640 1 -- 172.21.15.135:0/1586686411 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb813c220 con 0x7fcdb813fa70 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaffff640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb813fa70 0x7fcdb80ac390 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097d0 tx=0x7fcda00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0037050 con 0x7fcdb813fa70 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda0038040 con 0x7fcdb813fa70 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb813c500 con 0x7fcdb813fa70 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0006e10 con 0x7fcdb813fa70 2024-04-26T22:10:22.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.209+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb80ad200 con 0x7fcdb813fa70 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.210+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcdb80ad5e0 con 0x7fcdb813fa70 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.210+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcda0045020 con 0x7fcdb813fa70 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdaf7fe640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd9806c920 0x7fcd9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fcda00ac2e0 con 0x7fcdb813fa70 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdb6d9d640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd9806c920 0x7fcd9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdaf7fe640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7fcd98077d40 0x7fcd9807a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 --> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcda0034e90 con 0x7fcd98077d40 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcda00b3050 con 0x7fcdb813fa70 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdb6d9d640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd9806c920 0x7fcd9806ede0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcda8001d70 tx=0x7fcda8000f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.211+0000 7fcdb759e640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7fcd98077d40 0x7fcd9807a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.212+0000 7fcdb759e640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7fcd98077d40 0x7fcd9807a1e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.212+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== osd.6 v2:172.21.15.139:6816/3816888463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcda0034e90 con 0x7fcd98077d40 2024-04-26T22:10:22.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.225+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 --> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcdb8069bc0 con 0x7fcd98077d40 2024-04-26T22:10:22.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdaf7fe640 1 -- 172.21.15.135:0/1586686411 <== osd.6 v2:172.21.15.139:6816/3816888463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcdb8069bc0 con 0x7fcd98077d40 2024-04-26T22:10:22.226 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:22.226 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:22.226 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7fcd98077d40 msgr2=0x7fcd9807a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.139:6816/3816888463,v1:172.21.15.139:6817/3816888463] conn(0x7fcd98077d40 0x7fcd9807a1e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd9806c920 msgr2=0x7fcd9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd9806c920 0x7fcd9806ede0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcda8001d70 tx=0x7fcda8000f90 comp rx=0 tx=0).stop 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb813fa70 msgr2=0x7fcdb80ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 --2- 172.21.15.135:0/1586686411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb813fa70 0x7fcdb80ac390 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097d0 tx=0x7fcda00046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdb759e640 1 -- 172.21.15.135:0/1586686411 reap_dead start 2024-04-26T22:10:22.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 shutdown_connections 2024-04-26T22:10:22.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 >> 172.21.15.135:0/1586686411 conn(0x7fcdb80c5ce0 msgr2=0x7fcdb8159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 shutdown_connections 2024-04-26T22:10:22.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.226+0000 7fcdbd16a640 1 -- 172.21.15.135:0/1586686411 wait complete. 2024-04-26T22:10:22.237 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T22:10:22.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 Processor -- start 2024-04-26T22:10:22.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 -- start start 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 0x7ffb701576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 0x7ffb7015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb70156c10 con 0x7ffb700c2b80 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffb70157240 con 0x7ffb7013feb0 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.335+0000 7ffb76551640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffb700c5570 con 0x7ffb700c5050 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6f7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 0x7ffb7015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 0x7ffb701576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb74aee640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffb700c2b80 0x7ffb7006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:56108/0 (socket says 172.21.15.135:56108) 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 0x7ffb701576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:50434/0 (socket says 172.21.15.135:50434) 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb74aee640 1 -- 172.21.15.135:0/3920431697 learned_addr learned my addr 172.21.15.135:0/3920431697 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6f7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 0x7ffb7015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:39964/0 (socket says 172.21.15.135:39964) 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6effd640 1 -- 172.21.15.135:0/3920431697 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4070771436 0 0) 0x7ffb70156c10 con 0x7ffb700c2b80 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6effd640 1 -- 172.21.15.135:0/3920431697 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb58003520 con 0x7ffb700c2b80 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/3920431697 >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 msgr2=0x7ffb701576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6f7fe640 1 --2- 172.21.15.135:0/3920431697 >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 0x7ffb701576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/3920431697 >> v1:172.21.15.139:6789/0 conn(0x7ffb700c2b80 legacy=0x7ffb7006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/3920431697 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb700c5850 con 0x7ffb700c5050 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.336+0000 7ffb6ffff640 1 --2- 172.21.15.135:0/3920431697 >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 0x7ffb701576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.337+0000 7ffb6f7fe640 1 --2- 172.21.15.135:0/3920431697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 0x7ffb7015a1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb640097a0 tx=0x7ffb6402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec44ff0f9aa2f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.337+0000 7ffb6effd640 1 -- 172.21.15.135:0/3920431697 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64037070 con 0x7ffb700c5050 2024-04-26T22:10:22.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.337+0000 7ffb6effd640 1 -- 172.21.15.135:0/3920431697 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6402ae50 con 0x7ffb700c5050 2024-04-26T22:10:22.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.337+0000 7ffb6effd640 1 -- 172.21.15.135:0/3920431697 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64032640 con 0x7ffb700c5050 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 -- 172.21.15.135:0/3920431697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 msgr2=0x7ffb7015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 --2- 172.21.15.135:0/3920431697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 0x7ffb7015a1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb640097a0 tx=0x7ffb6402a100 comp rx=0 tx=0).stop 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 -- 172.21.15.135:0/3920431697 shutdown_connections 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 --2- 172.21.15.135:0/3920431697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c5050 0x7ffb7015a1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 --2- 172.21.15.135:0/3920431697 >> v2:172.21.15.139:3300/0 conn(0x7ffb7013feb0 0x7ffb701576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 -- 172.21.15.135:0/3920431697 >> 172.21.15.135:0/3920431697 conn(0x7ffb700c5ce0 msgr2=0x7ffb700c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 -- 172.21.15.135:0/3920431697 shutdown_connections 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.338+0000 7ffb76551640 1 -- 172.21.15.135:0/3920431697 wait complete. 2024-04-26T22:10:22.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.339+0000 7ffb76551640 1 Processor -- start 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.339+0000 7ffb76551640 1 -- start start 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c2b80 0x7ffb701e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 0x7ffb701ef810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb7013feb0 0x7ffb701efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7006dbb0 con 0x7ffb700c5050 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7006da30 con 0x7ffb7013feb0 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffb70140290 con 0x7ffb700c2b80 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 0x7ffb701ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 0x7ffb701ef810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51694/0 (socket says 172.21.15.135:51694) 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/999912906 learned_addr learned my addr 172.21.15.135:0/999912906 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6ffff640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c2b80 0x7ffb701e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb74aee640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb7013feb0 0x7ffb701efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c2b80 msgr2=0x7ffb701e9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c2b80 0x7ffb701e9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb7013feb0 msgr2=0x7ffb701efd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb7013feb0 0x7ffb701efd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 -- 172.21.15.135:0/999912906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb701e9f40 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6f7fe640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 0x7ffb701ef810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffb64000c00 tx=0x7ffb640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64041070 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6403c460 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb70074880 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.340+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64037040 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.341+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7013a770 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.342+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb700507a0 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.342+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb64044070 con 0x7ffb700c5050 2024-04-26T22:10:22.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.342+0000 7ffb6cff9640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffb5c06c920 0x7ffb5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.343+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7ffb640ac8a0 con 0x7ffb700c5050 2024-04-26T22:10:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.343+0000 7ffb6ffff640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffb5c06c920 0x7ffb5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.343+0000 7ffb6ffff640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffb5c06c920 0x7ffb5c06ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009560 tx=0x7ffb60009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.347+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb64079fc0 con 0x7ffb700c5050 2024-04-26T22:10:22.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.526+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ffb70074b60 con 0x7ffb5c06c920 2024-04-26T22:10:22.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.526+0000 7ffb6cff9640 1 -- 172.21.15.135:0/999912906 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb70074b60 con 0x7ffb5c06c920 2024-04-26T22:10:22.526 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T22:10:22.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffb5c06c920 msgr2=0x7ffb5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffb5c06c920 0x7ffb5c06ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009560 tx=0x7ffb60009190 comp rx=0 tx=0).stop 2024-04-26T22:10:22.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 msgr2=0x7ffb701ef810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 0x7ffb701ef810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffb64000c00 tx=0x7ffb640046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 shutdown_connections 2024-04-26T22:10:22.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7ffb5c06c920 0x7ffb5c06ede0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb7013feb0 0x7ffb701efd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb700c5050 0x7ffb701ef810 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 --2- 172.21.15.135:0/999912906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb700c2b80 0x7ffb701e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 >> 172.21.15.135:0/999912906 conn(0x7ffb700c5ce0 msgr2=0x7ffb7013e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 shutdown_connections 2024-04-26T22:10:22.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.528+0000 7ffb76551640 1 -- 172.21.15.135:0/999912906 wait complete. 2024-04-26T22:10:22.539 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T22:10:22.539 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T22:10:22.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.641+0000 7f6b7df51640 1 Processor -- start 2024-04-26T22:10:22.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.641+0000 7f6b7df51640 1 -- start start 2024-04-26T22:10:22.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b7df51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 0x7f6b780c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b7df51640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 0x7f6b780c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b7df51640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b780c47f0 con 0x7f6b78157470 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b7df51640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6b780c5120 con 0x7f6b78070560 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b7df51640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6b781579a0 con 0x7f6b78154110 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b77fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 0x7f6b780c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 0x7f6b780c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 0x7f6b780c1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:39968/0 (socket says 172.21.15.135:39968) 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b77fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 0x7f6b780c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.135:50462/0 (socket says 172.21.15.135:50462) 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b777fe640 1 -- 172.21.15.135:0/70724988 learned_addr learned my addr 172.21.15.135:0/70724988 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.642+0000 7f6b76ffd640 1 --1- 172.21.15.135:0/70724988 >> v1:172.21.15.139:6789/0 conn(0x7f6b78157470 0x7f6b780c3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:56120/0 (socket says 172.21.15.135:56120) 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b777fe640 1 -- 172.21.15.135:0/70724988 >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 msgr2=0x7f6b780c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b777fe640 1 --2- 172.21.15.135:0/70724988 >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 0x7f6b780c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b777fe640 1 -- 172.21.15.135:0/70724988 >> v1:172.21.15.139:6789/0 conn(0x7f6b78157470 legacy=0x7f6b780c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b777fe640 1 -- 172.21.15.135:0/70724988 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b780c4ad0 con 0x7f6b78154110 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b767fc640 1 -- 172.21.15.135:0/70724988 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145318910 0 0) 0x7f6b780c47f0 con 0x7f6b78157470 2024-04-26T22:10:22.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b77fff640 1 --2- 172.21.15.135:0/70724988 >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 0x7f6b780c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:22.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.643+0000 7f6b777fe640 1 --2- 172.21.15.135:0/70724988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 0x7f6b780c1d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009870 tx=0x7f6b6802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8399c1a2b88a7301 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b767fc640 1 -- 172.21.15.135:0/70724988 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68037070 con 0x7f6b78154110 2024-04-26T22:10:22.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b767fc640 1 -- 172.21.15.135:0/70724988 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b68032440 con 0x7f6b78154110 2024-04-26T22:10:22.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b767fc640 1 -- 172.21.15.135:0/70724988 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68032810 con 0x7f6b78154110 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 -- 172.21.15.135:0/70724988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 msgr2=0x7f6b780c1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 --2- 172.21.15.135:0/70724988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 0x7f6b780c1d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009870 tx=0x7f6b6802a130 comp rx=0 tx=0).stop 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 -- 172.21.15.135:0/70724988 shutdown_connections 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 --2- 172.21.15.135:0/70724988 >> v2:172.21.15.139:3300/0 conn(0x7f6b78070560 0x7f6b780c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 --2- 172.21.15.135:0/70724988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78154110 0x7f6b780c1d20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 -- 172.21.15.135:0/70724988 >> 172.21.15.135:0/70724988 conn(0x7f6b780aa6a0 msgr2=0x7f6b780aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 -- 172.21.15.135:0/70724988 shutdown_connections 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.644+0000 7f6b7df51640 1 -- 172.21.15.135:0/70724988 wait complete. 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.645+0000 7f6b7df51640 1 Processor -- start 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.645+0000 7f6b7df51640 1 -- start start 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b7df51640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b78070560 0x7f6b78072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b7df51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b78154110 0x7f6b7814c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b7df51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78157470 0x7f6b78073520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b7df51640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b781544f0 con 0x7f6b78070560 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b7df51640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b780c42e0 con 0x7f6b78154110 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b7df51640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6b78154670 con 0x7f6b78157470 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b78070560 0x7f6b78072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b78070560 0x7f6b78072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51704/0 (socket says 172.21.15.135:51704) 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b777fe640 1 -- 172.21.15.135:0/3889840061 learned_addr learned my addr 172.21.15.135:0/3889840061 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:22.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b77fff640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78157470 0x7f6b78073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.646+0000 7f6b76ffd640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b78154110 0x7f6b7814c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b777fe640 1 -- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78157470 msgr2=0x7f6b78073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b777fe640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b78157470 0x7f6b78073520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b777fe640 1 -- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b78154110 msgr2=0x7f6b7814c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b777fe640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b78154110 0x7f6b7814c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b777fe640 1 -- 172.21.15.135:0/3889840061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7814b120 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b777fe640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b78070560 0x7f6b78072b40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009870 tx=0x7f6b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68037040 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b68004230 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7814b400 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6803c670 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.647+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7814b9c0 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.648+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b780694b0 con 0x7f6b78070560 2024-04-26T22:10:22.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.648+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b6803a070 con 0x7f6b78070560 2024-04-26T22:10:22.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b5ffff640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b6406c920 0x7f6b6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f6b680aca50 con 0x7f6b78070560 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b76ffd640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b6406c920 0x7f6b6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b5ffff640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f6b64077d40 0x7f6b6407a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 --> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b6407a8b0 con 0x7f6b64077d40 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b680b12c0 con 0x7f6b78070560 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b76ffd640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b6406c920 0x7f6b6406ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b7814ae30 tx=0x7f6b6c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.649+0000 7f6b77fff640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f6b64077d40 0x7f6b6407a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.650+0000 7f6b77fff640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f6b64077d40 0x7f6b6407a1e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.650+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== osd.7 v2:172.21.15.139:6824/3012659507 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b6407a8b0 con 0x7f6b64077d40 2024-04-26T22:10:22.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.663+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 --> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b7806d320 con 0x7f6b64077d40 2024-04-26T22:10:22.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b5ffff640 1 -- 172.21.15.135:0/3889840061 <== osd.7 v2:172.21.15.139:6824/3012659507 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6b7806d320 con 0x7f6b64077d40 2024-04-26T22:10:22.664 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-26T22:10:22.664 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T22:10:22.664 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-26T22:10:22.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f6b64077d40 msgr2=0x7f6b6407a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.139:6824/3012659507,v1:172.21.15.139:6825/3012659507] conn(0x7f6b64077d40 0x7f6b6407a1e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b6406c920 msgr2=0x7f6b6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f6b6406c920 0x7f6b6406ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b7814ae30 tx=0x7f6b6c007de0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b78070560 msgr2=0x7f6b78072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 --2- 172.21.15.135:0/3889840061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b78070560 0x7f6b78072b40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009870 tx=0x7f6b680046c0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b77fff640 1 -- 172.21.15.135:0/3889840061 reap_dead start 2024-04-26T22:10:22.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 shutdown_connections 2024-04-26T22:10:22.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 >> 172.21.15.135:0/3889840061 conn(0x7f6b780aa6a0 msgr2=0x7f6b780aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 shutdown_connections 2024-04-26T22:10:22.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.664+0000 7f6b7df51640 1 -- 172.21.15.135:0/3889840061 wait complete. 2024-04-26T22:10:22.674 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T22:10:22.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.773+0000 7fcd2530a640 1 Processor -- start 2024-04-26T22:10:22.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.773+0000 7fcd2530a640 1 -- start start 2024-04-26T22:10:22.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.773+0000 7fcd2530a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcd20070f70 0x7fcd200713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.773+0000 7fcd2530a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd200707e0 con 0x7fcd2013ddd0 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.773+0000 7fcd2530a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd20155450 con 0x7fcd20071cc0 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.773+0000 7fcd2530a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcd20155210 con 0x7fcd20070f70 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcd20070f70 0x7fcd200713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1effd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fcd2013ddd0 0x7fcd20154b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46874/0 (socket says 172.21.15.135:46874) 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1effd640 1 -- 172.21.15.135:0/332532260 learned_addr learned my addr 172.21.15.135:0/332532260 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/332532260 >> v1:172.21.15.139:6789/0 conn(0x7fcd20071cc0 legacy=0x7fcd20073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/332532260 >> v1:172.21.15.135:6789/0 conn(0x7fcd2013ddd0 legacy=0x7fcd20154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1e7fc640 1 -- 172.21.15.135:0/332532260 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080397824 0 0) 0x7fcd200707e0 con 0x7fcd2013ddd0 2024-04-26T22:10:22.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.774+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/332532260 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd20155730 con 0x7fcd20070f70 2024-04-26T22:10:22.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd1f7fe640 1 --2- 172.21.15.135:0/332532260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd20070f70 0x7fcd200713c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009d90 tx=0x7fcd1802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b807325ee34f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd1e7fc640 1 -- 172.21.15.135:0/332532260 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18007740 con 0x7fcd20070f70 2024-04-26T22:10:22.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd1e7fc640 1 -- 172.21.15.135:0/332532260 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd180078e0 con 0x7fcd20070f70 2024-04-26T22:10:22.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd1e7fc640 1 -- 172.21.15.135:0/332532260 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18007cb0 con 0x7fcd20070f70 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 -- 172.21.15.135:0/332532260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd20070f70 msgr2=0x7fcd200713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 --2- 172.21.15.135:0/332532260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd20070f70 0x7fcd200713c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009d90 tx=0x7fcd1802a660 comp rx=0 tx=0).stop 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 -- 172.21.15.135:0/332532260 shutdown_connections 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 --2- 172.21.15.135:0/332532260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd20070f70 0x7fcd200713c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 -- 172.21.15.135:0/332532260 >> 172.21.15.135:0/332532260 conn(0x7fcd200ca2f0 msgr2=0x7fcd200ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 -- 172.21.15.135:0/332532260 shutdown_connections 2024-04-26T22:10:22.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.775+0000 7fcd2530a640 1 -- 172.21.15.135:0/332532260 wait complete. 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.776+0000 7fcd2530a640 1 Processor -- start 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.776+0000 7fcd2530a640 1 -- start start 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd2530a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd20070f70 0x7fcd200c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd2530a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd20071cc0 0x7fcd201570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd2530a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 0x7fcd20157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd2530a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20073e10 con 0x7fcd2013ddd0 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd2530a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20071900 con 0x7fcd20070f70 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd2530a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd20073f90 con 0x7fcd20071cc0 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 0x7fcd20157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 0x7fcd20157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51720/0 (socket says 172.21.15.135:51720) 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/1255528682 learned_addr learned my addr 172.21.15.135:0/1255528682 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd17fff640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd20071cc0 0x7fcd201570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1effd640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd20070f70 0x7fcd200c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd20071cc0 msgr2=0x7fcd201570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd20071cc0 0x7fcd201570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd20070f70 msgr2=0x7fcd200c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd20070f70 0x7fcd200c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.777+0000 7fcd1f7fe640 1 -- 172.21.15.135:0/1255528682 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd200c4670 con 0x7fcd2013ddd0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.778+0000 7fcd1f7fe640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 0x7fcd20157630 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009d90 tx=0x7fcd18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.778+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18040070 con 0x7fcd2013ddd0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.778+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd18031c90 con 0x7fcd2013ddd0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.778+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd201ed2f0 con 0x7fcd2013ddd0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.778+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1803c640 con 0x7fcd2013ddd0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.778+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2013b190 con 0x7fcd2013ddd0 2024-04-26T22:10:22.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.779+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd200507a0 con 0x7fcd2013ddd0 2024-04-26T22:10:22.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.779+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd18039040 con 0x7fcd2013ddd0 2024-04-26T22:10:22.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.780+0000 7fcd177fe640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0006c8f0 0x7fcd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:22.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.780+0000 7fcd1effd640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0006c8f0 0x7fcd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:22.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.780+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fcd180ad9f0 con 0x7fcd2013ddd0 2024-04-26T22:10:22.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.780+0000 7fcd1effd640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0006c8f0 0x7fcd0006edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000f60 tx=0x7fcd10007d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:22.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.784+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd1807b700 con 0x7fcd2013ddd0 2024-04-26T22:10:22.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.962+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fcd20074110 con 0x7fcd0006c8f0 2024-04-26T22:10:22.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.963+0000 7fcd177fe640 1 -- 172.21.15.135:0/1255528682 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcd20074110 con 0x7fcd0006c8f0 2024-04-26T22:10:22.963 INFO:teuthology.orchestra.run.smithi135.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0006c8f0 msgr2=0x7fcd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0006c8f0 0x7fcd0006edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000f60 tx=0x7fcd10007d00 comp rx=0 tx=0).stop 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 msgr2=0x7fcd20157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 0x7fcd20157630 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009d90 tx=0x7fcd18004a40 comp rx=0 tx=0).stop 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 shutdown_connections 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fcd0006c8f0 0x7fcd0006edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd2013ddd0 0x7fcd20157630 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd20071cc0 0x7fcd201570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 --2- 172.21.15.135:0/1255528682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd20070f70 0x7fcd200c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:22.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 >> 172.21.15.135:0/1255528682 conn(0x7fcd200ca2f0 msgr2=0x7fcd200af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:22.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 shutdown_connections 2024-04-26T22:10:22.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:22.965+0000 7fcd2530a640 1 -- 172.21.15.135:0/1255528682 wait complete. 2024-04-26T22:10:22.976 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:10:23.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.074+0000 7fdb621e9640 1 Processor -- start 2024-04-26T22:10:23.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.074+0000 7fdb621e9640 1 -- start start 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb621e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 0x7fdb5c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb621e9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb5c154220 0x7fdb5c072930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb621e9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5c156de0 con 0x7fdb5c0c7000 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb621e9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdb5c157b60 con 0x7fdb5c154220 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb621e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c0cad40 con 0x7fdb5c0c9830 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 0x7fdb5c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:23.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb5c154220 0x7fdb5c072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 0x7fdb5c153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:39996/0 (socket says 172.21.15.135:39996) 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5affd640 1 -- 172.21.15.135:0/619118850 learned_addr learned my addr 172.21.15.135:0/619118850 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5bfff640 1 -- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 msgr2=0x7fdb5c153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5bfff640 1 --2- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 0x7fdb5c153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5bfff640 1 -- 172.21.15.135:0/619118850 >> v1:172.21.15.139:6789/0 conn(0x7fdb5c0c7000 legacy=0x7fdb5c0ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5a7fc640 1 -- 172.21.15.135:0/619118850 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2575807627 0 0) 0x7fdb5c156de0 con 0x7fdb5c0c7000 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5bfff640 1 -- 172.21.15.135:0/619118850 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c157080 con 0x7fdb5c154220 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.075+0000 7fdb5affd640 1 --2- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 0x7fdb5c153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.076+0000 7fdb5bfff640 1 --2- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c154220 0x7fdb5c072930 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009d90 tx=0x7fdb5402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c80167c68b022b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.076+0000 7fdb5a7fc640 1 -- 172.21.15.135:0/619118850 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54007740 con 0x7fdb5c154220 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.076+0000 7fdb5a7fc640 1 -- 172.21.15.135:0/619118850 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb540078e0 con 0x7fdb5c154220 2024-04-26T22:10:23.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.076+0000 7fdb5a7fc640 1 -- 172.21.15.135:0/619118850 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54007cb0 con 0x7fdb5c154220 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 -- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c154220 msgr2=0x7fdb5c072930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 --2- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c154220 0x7fdb5c072930 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009d90 tx=0x7fdb5402a660 comp rx=0 tx=0).stop 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 -- 172.21.15.135:0/619118850 shutdown_connections 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 --2- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c154220 0x7fdb5c072930 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 --2- 172.21.15.135:0/619118850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c9830 0x7fdb5c153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 -- 172.21.15.135:0/619118850 >> 172.21.15.135:0/619118850 conn(0x7fdb5c0adf20 msgr2=0x7fdb5c0ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 -- 172.21.15.135:0/619118850 shutdown_connections 2024-04-26T22:10:23.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.077+0000 7fdb621e9640 1 -- 172.21.15.135:0/619118850 wait complete. 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.078+0000 7fdb621e9640 1 Processor -- start 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.078+0000 7fdb621e9640 1 -- start start 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb621e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c7000 0x7fdb5c155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb621e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c0c9830 0x7fdb5c0c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb621e9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 0x7fdb5c0c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb621e9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c0c73e0 con 0x7fdb5c154220 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb621e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c157640 con 0x7fdb5c0c9830 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb621e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c0c7560 con 0x7fdb5c0c7000 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 0x7fdb5c0c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 0x7fdb5c0c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51724/0 (socket says 172.21.15.135:51724) 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 -- 172.21.15.135:0/806730419 learned_addr learned my addr 172.21.15.135:0/806730419 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5b7fe640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c7000 0x7fdb5c155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:23.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5affd640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c0c9830 0x7fdb5c0c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 -- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c7000 msgr2=0x7fdb5c155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c7000 0x7fdb5c155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 -- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c0c9830 msgr2=0x7fdb5c0c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c0c9830 0x7fdb5c0c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.079+0000 7fdb5bfff640 1 -- 172.21.15.135:0/806730419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c14ef50 con 0x7fdb5c154220 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.080+0000 7fdb5bfff640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 0x7fdb5c0c2eb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009eb0 tx=0x7fdb54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.080+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54040070 con 0x7fdb5c154220 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.080+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb54031c40 con 0x7fdb5c154220 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.080+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c14f230 con 0x7fdb5c154220 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.080+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54009450 con 0x7fdb5c154220 2024-04-26T22:10:23.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.080+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c0b0230 con 0x7fdb5c154220 2024-04-26T22:10:23.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.081+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c0766c0 con 0x7fdb5c154220 2024-04-26T22:10:23.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.081+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb54039040 con 0x7fdb5c154220 2024-04-26T22:10:23.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.082+0000 7fdb43fff640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb4806c920 0x7fdb4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:23.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.082+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fdb540ad0f0 con 0x7fdb5c154220 2024-04-26T22:10:23.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.082+0000 7fdb5b7fe640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb4806c920 0x7fdb4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:23.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.082+0000 7fdb5b7fe640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb4806c920 0x7fdb4806ede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000f60 tx=0x7fdb4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:23.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.086+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb5407a670 con 0x7fdb5c154220 2024-04-26T22:10:23.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.262+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb5c0507a0 con 0x7fdb4806c920 2024-04-26T22:10:23.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.274+0000 7fdb43fff640 1 -- 172.21.15.135:0/806730419 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493645 (secure 0 0 0) 0x7fdb5c0507a0 con 0x7fdb4806c920 2024-04-26T22:10:23.279 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:23.282 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb4806c920 msgr2=0x7fdb4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb4806c920 0x7fdb4806ede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000f60 tx=0x7fdb4c009040 comp rx=0 tx=0).stop 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 msgr2=0x7fdb5c0c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 0x7fdb5c0c2eb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009eb0 tx=0x7fdb54004a40 comp rx=0 tx=0).stop 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 shutdown_connections 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fdb4806c920 0x7fdb4806ede0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb5c154220 0x7fdb5c0c2eb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c0c9830 0x7fdb5c0c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 --2- 172.21.15.135:0/806730419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0c7000 0x7fdb5c155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:23.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 >> 172.21.15.135:0/806730419 conn(0x7fdb5c0adf20 msgr2=0x7fdb5c1531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:23.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 shutdown_connections 2024-04-26T22:10:23.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:23.278+0000 7fdb621e9640 1 -- 172.21.15.135:0/806730419 wait complete. 2024-04-26T22:10:23.289 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-04-26T22:10:21.309774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252981638,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":3350,"num_read_kb":829546,"num_write":9253300,"num_write_kb":113906497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610757,"ondisk_log_size":610757,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6029464,"kb_used_data":497752,"kb_used_omap":12,"kb_used_meta":5531635,"kb_avail":748945256,"statfs":{"total":773094113280,"available":766919942144,"internally_reserved":0,"allocated":509698048,"data_stored":507719436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":5664394801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367561287,"num_objects":-3740,"num_object_clones":0,"num_object_copies":-7480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3740,"num_whiteouts":0,"num_read":514,"num_read_kb":106,"num_write":3931,"num_write_kb":2031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"5.18","version":"40'4026","reported_seq":4100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.983679+0000","last_change":"2024-04-26T22:00:06.813225+0000","last_active":"2024-04-26T22:10:11.983679+0000","last_peered":"2024-04-26T22:10:11.983679+0000","last_clean":"2024-04-26T22:10:11.983679+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:10:11.983679+0000","last_undegraded":"2024-04-26T22:10:11.983679+0000","last_fullsized":"2024-04-26T22:10:11.983679+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":60598,"num_write_kb":740487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"39'27","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.941276+0000","last_change":"2024-04-26T21:59:56.644064+0000","last_active":"2024-04-26T22:10:00.941276+0000","last_peered":"2024-04-26T22:10:00.941276+0000","last_clean":"2024-04-26T22:10:00.941276+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:10:00.941276+0000","last_undegraded":"2024-04-26T22:10:00.941276+0000","last_fullsized":"2024-04-26T22:10:00.941276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:07.837473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":16408,"num_write":20,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'5701","reported_seq":5758,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.764348+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:10:05.764348+0000","last_peered":"2024-04-26T22:10:05.764348+0000","last_clean":"2024-04-26T22:10:05.764348+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:10:05.764348+0000","last_undegraded":"2024-04-26T22:10:05.764348+0000","last_fullsized":"2024-04-26T22:10:05.764348+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86528,"num_write_kb":1106988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"40'21","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:15.703604+0000","last_change":"2024-04-26T22:00:05.813359+0000","last_active":"2024-04-26T22:10:15.703604+0000","last_peered":"2024-04-26T22:10:15.703604+0000","last_clean":"2024-04-26T22:10:15.703604+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:10:15.703604+0000","last_undegraded":"2024-04-26T22:10:15.703604+0000","last_fullsized":"2024-04-26T22:10:15.703604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:57.584465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190714,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"40'4869","reported_seq":4958,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988315+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:10:11.988315+0000","last_peered":"2024-04-26T22:10:11.988315+0000","last_clean":"2024-04-26T22:10:11.988315+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:10:11.988315+0000","last_undegraded":"2024-04-26T22:10:11.988315+0000","last_fullsized":"2024-04-26T22:10:11.988315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":73297,"num_write_kb":899566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"40'20","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857789+0000","last_change":"2024-04-26T21:59:56.646039+0000","last_active":"2024-04-26T22:10:05.857789+0000","last_peered":"2024-04-26T22:10:05.857789+0000","last_clean":"2024-04-26T22:10:05.857789+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:10:05.857789+0000","last_undegraded":"2024-04-26T22:10:05.857789+0000","last_fullsized":"2024-04-26T22:10:05.857789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:57.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"40'6511","reported_seq":6573,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848823+0000","last_change":"2024-04-26T21:59:57.646419+0000","last_active":"2024-04-26T22:10:05.848823+0000","last_peered":"2024-04-26T22:10:05.848823+0000","last_clean":"2024-04-26T22:10:05.848823+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:10:05.848823+0000","last_undegraded":"2024-04-26T22:10:05.848823+0000","last_fullsized":"2024-04-26T22:10:05.848823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6511,"log_dups_size":0,"ondisk_log_size":6511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":98412,"num_write_kb":1227645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"40'132","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.904919+0000","last_change":"2024-04-26T22:00:05.820118+0000","last_active":"2024-04-26T22:10:11.904919+0000","last_peered":"2024-04-26T22:10:11.904919+0000","last_clean":"2024-04-26T22:10:11.904919+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:10:11.904919+0000","last_undegraded":"2024-04-26T22:10:11.904919+0000","last_fullsized":"2024-04-26T22:10:11.904919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:02.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":127,"num_write_kb":8276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"40'3508","reported_seq":3574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.949165+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:10:11.949165+0000","last_peered":"2024-04-26T22:10:11.949165+0000","last_clean":"2024-04-26T22:10:11.949165+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:10:11.949165+0000","last_undegraded":"2024-04-26T22:10:11.949165+0000","last_fullsized":"2024-04-26T22:10:11.949165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3508,"log_dups_size":0,"ondisk_log_size":3508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":52619,"num_write_kb":632470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"40'116","reported_seq":197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.110609+0000","last_change":"2024-04-26T21:59:56.647456+0000","last_active":"2024-04-26T22:10:10.110609+0000","last_peered":"2024-04-26T22:10:10.110609+0000","last_clean":"2024-04-26T22:10:10.110609+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:10:10.110609+0000","last_undegraded":"2024-04-26T22:10:10.110609+0000","last_fullsized":"2024-04-26T22:10:10.110609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:18.886629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":45092,"num_write":102,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"40'3918","reported_seq":3975,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.754455+0000","last_change":"2024-04-26T21:59:57.646791+0000","last_active":"2024-04-26T22:10:05.754455+0000","last_peered":"2024-04-26T22:10:05.754455+0000","last_clean":"2024-04-26T22:10:05.754455+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:10:05.754455+0000","last_undegraded":"2024-04-26T22:10:05.754455+0000","last_fullsized":"2024-04-26T22:10:05.754455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":59308,"num_write_kb":714963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"40'1268","reported_seq":1325,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.959833+0000","last_change":"2024-04-26T22:00:05.817773+0000","last_active":"2024-04-26T22:10:00.959833+0000","last_peered":"2024-04-26T22:10:00.959833+0000","last_clean":"2024-04-26T22:10:00.959833+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:10:00.959833+0000","last_undegraded":"2024-04-26T22:10:00.959833+0000","last_fullsized":"2024-04-26T22:10:00.959833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1268,"log_dups_size":0,"ondisk_log_size":1268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:35.067789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":1265,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'3315","reported_seq":3384,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.932922+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:10:11.932922+0000","last_peered":"2024-04-26T22:10:11.932922+0000","last_clean":"2024-04-26T22:10:11.932922+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:10:11.932922+0000","last_undegraded":"2024-04-26T22:10:11.932922+0000","last_fullsized":"2024-04-26T22:10:11.932922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3315,"log_dups_size":0,"ondisk_log_size":3315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":49749,"num_write_kb":591259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"40'20","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.954736+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:10:01.954736+0000","last_peered":"2024-04-26T22:10:01.954736+0000","last_clean":"2024-04-26T22:10:01.954736+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:10:01.954736+0000","last_undegraded":"2024-04-26T22:10:01.954736+0000","last_fullsized":"2024-04-26T22:10:01.954736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:05.385006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":18,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"40'6620","reported_seq":6664,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.806952+0000","last_change":"2024-04-26T22:00:39.151257+0000","last_active":"2024-04-26T22:10:05.806952+0000","last_peered":"2024-04-26T22:10:05.806952+0000","last_clean":"2024-04-26T22:10:05.806952+0000","last_became_active":"2024-04-26T22:00:39.151025+0000","last_became_peered":"2024-04-26T22:00:39.151025+0000","last_unstale":"2024-04-26T22:10:05.806952+0000","last_undegraded":"2024-04-26T22:10:05.806952+0000","last_fullsized":"2024-04-26T22:10:05.806952+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:06.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":100009,"num_write_kb":1224131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"40'41","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955354+0000","last_change":"2024-04-26T22:00:05.816817+0000","last_active":"2024-04-26T22:10:11.955354+0000","last_peered":"2024-04-26T22:10:11.955354+0000","last_clean":"2024-04-26T22:10:11.955354+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:10:11.955354+0000","last_undegraded":"2024-04-26T22:10:11.955354+0000","last_fullsized":"2024-04-26T22:10:11.955354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:05.171423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":30,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"40'4367","reported_seq":4448,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.971730+0000","last_change":"2024-04-26T22:00:06.814961+0000","last_active":"2024-04-26T22:10:11.971730+0000","last_peered":"2024-04-26T22:10:11.971730+0000","last_clean":"2024-04-26T22:10:11.971730+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:10:11.971730+0000","last_undegraded":"2024-04-26T22:10:11.971730+0000","last_fullsized":"2024-04-26T22:10:11.971730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65356,"num_write_kb":790948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"40'4740","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.661582+0000","last_change":"2024-04-26T21:59:57.638345+0000","last_active":"2024-04-26T22:10:05.661582+0000","last_peered":"2024-04-26T22:10:05.661582+0000","last_clean":"2024-04-26T22:10:05.661582+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:10:05.661582+0000","last_undegraded":"2024-04-26T22:10:05.661582+0000","last_fullsized":"2024-04-26T22:10:05.661582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":71515,"num_write_kb":926848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"40'206","reported_seq":280,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.865453+0000","last_change":"2024-04-26T21:59:56.645199+0000","last_active":"2024-04-26T22:10:05.865453+0000","last_peered":"2024-04-26T22:10:05.865453+0000","last_clean":"2024-04-26T22:10:05.865453+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:10:05.865453+0000","last_undegraded":"2024-04-26T22:10:05.865453+0000","last_fullsized":"2024-04-26T22:10:05.865453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:07.948517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":29169,"num_write":192,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"40'39","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.495965+0000","last_change":"2024-04-26T22:00:05.816037+0000","last_active":"2024-04-26T22:10:01.495965+0000","last_peered":"2024-04-26T22:10:01.495965+0000","last_clean":"2024-04-26T22:10:01.495965+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:10:01.495965+0000","last_undegraded":"2024-04-26T22:10:01.495965+0000","last_fullsized":"2024-04-26T22:10:01.495965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:31.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":13991,"num_write":36,"num_write_kb":4124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'3984","reported_seq":4061,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986082+0000","last_change":"2024-04-26T22:00:06.815105+0000","last_active":"2024-04-26T22:10:11.986082+0000","last_peered":"2024-04-26T22:10:11.986082+0000","last_clean":"2024-04-26T22:10:11.986082+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:10:11.986082+0000","last_undegraded":"2024-04-26T22:10:11.986082+0000","last_fullsized":"2024-04-26T22:10:11.986082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59797,"num_write_kb":746756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"40'4602","reported_seq":4661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852754+0000","last_change":"2024-04-26T21:59:57.659038+0000","last_active":"2024-04-26T22:10:05.852754+0000","last_peered":"2024-04-26T22:10:05.852754+0000","last_clean":"2024-04-26T22:10:05.852754+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:10:05.852754+0000","last_undegraded":"2024-04-26T22:10:05.852754+0000","last_fullsized":"2024-04-26T22:10:05.852754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69468,"num_write_kb":843448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"40'24","reported_seq":86,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850262+0000","last_change":"2024-04-26T21:59:56.649073+0000","last_active":"2024-04-26T22:10:05.850262+0000","last_peered":"2024-04-26T22:10:05.850262+0000","last_clean":"2024-04-26T22:10:05.850262+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:10:05.850262+0000","last_undegraded":"2024-04-26T22:10:05.850262+0000","last_fullsized":"2024-04-26T22:10:05.850262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":20,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937503+0000","last_change":"2024-04-26T22:00:05.817324+0000","last_active":"2024-04-26T22:10:11.937503+0000","last_peered":"2024-04-26T22:10:11.937503+0000","last_clean":"2024-04-26T22:10:11.937503+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:10:11.937503+0000","last_undegraded":"2024-04-26T22:10:11.937503+0000","last_fullsized":"2024-04-26T22:10:11.937503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:50.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"40'3976","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.916627+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:10:11.916627+0000","last_peered":"2024-04-26T22:10:11.916627+0000","last_clean":"2024-04-26T22:10:11.916627+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:10:11.916627+0000","last_undegraded":"2024-04-26T22:10:11.916627+0000","last_fullsized":"2024-04-26T22:10:11.916627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3976,"log_dups_size":0,"ondisk_log_size":3976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:42.491295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59917,"num_write_kb":706723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"40'5423","reported_seq":5480,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.761465+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:10:05.761465+0000","last_peered":"2024-04-26T22:10:05.761465+0000","last_clean":"2024-04-26T22:10:05.761465+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:10:05.761465+0000","last_undegraded":"2024-04-26T22:10:05.761465+0000","last_fullsized":"2024-04-26T22:10:05.761465+0000","mapping_epoch":18,"log_start":"28'200","ondisk_log_start":"28'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":81931,"num_write_kb":1035269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'44","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:15.678035+0000","last_change":"2024-04-26T21:59:56.645338+0000","last_active":"2024-04-26T22:10:15.678035+0000","last_peered":"2024-04-26T22:10:15.678035+0000","last_clean":"2024-04-26T22:10:15.678035+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:10:15.678035+0000","last_undegraded":"2024-04-26T22:10:15.678035+0000","last_fullsized":"2024-04-26T22:10:15.678035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363115,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":324,"num_write":38,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"40'85","reported_seq":147,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.980211+0000","last_change":"2024-04-26T22:00:05.825395+0000","last_active":"2024-04-26T22:10:11.980211+0000","last_peered":"2024-04-26T22:10:11.980211+0000","last_clean":"2024-04-26T22:10:11.980211+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:10:11.980211+0000","last_undegraded":"2024-04-26T22:10:11.980211+0000","last_fullsized":"2024-04-26T22:10:11.980211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":6841,"num_write":81,"num_write_kb":4168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"40'4751","reported_seq":4829,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937603+0000","last_change":"2024-04-26T22:00:06.823296+0000","last_active":"2024-04-26T22:10:11.937603+0000","last_peered":"2024-04-26T22:10:11.937603+0000","last_clean":"2024-04-26T22:10:11.937603+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:10:11.937603+0000","last_undegraded":"2024-04-26T22:10:11.937603+0000","last_fullsized":"2024-04-26T22:10:11.937603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":71494,"num_write_kb":858502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"40'5174","reported_seq":5233,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.821492+0000","last_change":"2024-04-26T21:59:57.648677+0000","last_active":"2024-04-26T22:10:05.821492+0000","last_peered":"2024-04-26T22:10:05.821492+0000","last_clean":"2024-04-26T22:10:05.821492+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:10:05.821492+0000","last_undegraded":"2024-04-26T22:10:05.821492+0000","last_fullsized":"2024-04-26T22:10:05.821492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:23:06.389123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":78246,"num_write_kb":941647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"40'8","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823374+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:10:05.823374+0000","last_peered":"2024-04-26T22:10:05.823374+0000","last_clean":"2024-04-26T22:10:05.823374+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:10:05.823374+0000","last_undegraded":"2024-04-26T22:10:05.823374+0000","last_fullsized":"2024-04-26T22:10:05.823374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":4,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"40'37","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962744+0000","last_change":"2024-04-26T22:00:05.820508+0000","last_active":"2024-04-26T22:10:11.962744+0000","last_peered":"2024-04-26T22:10:11.962744+0000","last_clean":"2024-04-26T22:10:11.962744+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:10:11.962744+0000","last_undegraded":"2024-04-26T22:10:11.962744+0000","last_fullsized":"2024-04-26T22:10:11.962744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:20.666138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":31,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"40'3913","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.962293+0000","last_change":"2024-04-26T22:00:06.813865+0000","last_active":"2024-04-26T22:10:11.962293+0000","last_peered":"2024-04-26T22:10:11.962293+0000","last_clean":"2024-04-26T22:10:11.962293+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:10:11.962293+0000","last_undegraded":"2024-04-26T22:10:11.962293+0000","last_fullsized":"2024-04-26T22:10:11.962293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":58791,"num_write_kb":740377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'4599","reported_seq":4657,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.774768+0000","last_change":"2024-04-26T21:59:57.643292+0000","last_active":"2024-04-26T22:10:05.774768+0000","last_peered":"2024-04-26T22:10:05.774768+0000","last_clean":"2024-04-26T22:10:05.774768+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:10:05.774768+0000","last_undegraded":"2024-04-26T22:10:05.774768+0000","last_fullsized":"2024-04-26T22:10:05.774768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69140,"num_write_kb":847801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"40'8","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836838+0000","last_change":"2024-04-26T21:59:56.647956+0000","last_active":"2024-04-26T22:10:05.836838+0000","last_peered":"2024-04-26T22:10:05.836838+0000","last_clean":"2024-04-26T22:10:05.836838+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:10:05.836838+0000","last_undegraded":"2024-04-26T22:10:05.836838+0000","last_fullsized":"2024-04-26T22:10:05.836838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"40'23","reported_seq":100,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.979062+0000","last_change":"2024-04-26T22:00:05.815099+0000","last_active":"2024-04-26T22:10:14.979062+0000","last_peered":"2024-04-26T22:10:14.979062+0000","last_clean":"2024-04-26T22:10:14.979062+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:10:14.979062+0000","last_undegraded":"2024-04-26T22:10:14.979062+0000","last_fullsized":"2024-04-26T22:10:14.979062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":98,"num_write":20,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"40'3929","reported_seq":3998,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.938066+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:10:11.938066+0000","last_peered":"2024-04-26T22:10:11.938066+0000","last_clean":"2024-04-26T22:10:11.938066+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:10:11.938066+0000","last_undegraded":"2024-04-26T22:10:11.938066+0000","last_fullsized":"2024-04-26T22:10:11.938066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":59023,"num_write_kb":718389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'5311","reported_seq":5368,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.831429+0000","last_change":"2024-04-26T21:59:57.656299+0000","last_active":"2024-04-26T22:10:05.831429+0000","last_peered":"2024-04-26T22:10:05.831429+0000","last_clean":"2024-04-26T22:10:05.831429+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:10:05.831429+0000","last_undegraded":"2024-04-26T22:10:05.831429+0000","last_fullsized":"2024-04-26T22:10:05.831429+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80379,"num_write_kb":1004221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'218","reported_seq":283,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.887657+0000","last_change":"2024-04-26T21:59:56.648315+0000","last_active":"2024-04-26T22:10:14.887657+0000","last_peered":"2024-04-26T22:10:14.887657+0000","last_clean":"2024-04-26T22:10:14.887657+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:10:14.887657+0000","last_undegraded":"2024-04-26T22:10:14.887657+0000","last_fullsized":"2024-04-26T22:10:14.887657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":61,"num_write":416,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"40'22","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979441+0000","last_change":"2024-04-26T22:00:05.816446+0000","last_active":"2024-04-26T22:10:11.979441+0000","last_peered":"2024-04-26T22:10:11.979441+0000","last_clean":"2024-04-26T22:10:11.979441+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:10:11.979441+0000","last_undegraded":"2024-04-26T22:10:11.979441+0000","last_fullsized":"2024-04-26T22:10:11.979441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:47.479915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":15,"num_write_kb":8199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"40'4334","reported_seq":4409,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.892208+0000","last_change":"2024-04-26T22:00:06.814925+0000","last_active":"2024-04-26T22:10:11.892208+0000","last_peered":"2024-04-26T22:10:11.892208+0000","last_clean":"2024-04-26T22:10:11.892208+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:10:11.892208+0000","last_undegraded":"2024-04-26T22:10:11.892208+0000","last_fullsized":"2024-04-26T22:10:11.892208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":65365,"num_write_kb":782662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"40'5496","reported_seq":5557,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857732+0000","last_change":"2024-04-26T21:59:57.650934+0000","last_active":"2024-04-26T22:10:05.857732+0000","last_peered":"2024-04-26T22:10:05.857732+0000","last_clean":"2024-04-26T22:10:05.857732+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:10:05.857732+0000","last_undegraded":"2024-04-26T22:10:05.857732+0000","last_fullsized":"2024-04-26T22:10:05.857732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":82892,"num_write_kb":1050745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'35","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.834069+0000","last_change":"2024-04-26T21:59:56.649770+0000","last_active":"2024-04-26T22:10:05.834069+0000","last_peered":"2024-04-26T22:10:05.834069+0000","last_clean":"2024-04-26T22:10:05.834069+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:10:05.834069+0000","last_undegraded":"2024-04-26T22:10:05.834069+0000","last_fullsized":"2024-04-26T22:10:05.834069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:39.952093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":29,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"28'1","reported_seq":50,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940808+0000","last_change":"2024-04-26T22:00:05.817195+0000","last_active":"2024-04-26T22:10:00.940808+0000","last_peered":"2024-04-26T22:10:00.940808+0000","last_clean":"2024-04-26T22:10:00.940808+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:10:00.940808+0000","last_undegraded":"2024-04-26T22:10:00.940808+0000","last_fullsized":"2024-04-26T22:10:00.940808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:11:45.494436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"40'3789","reported_seq":3862,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937218+0000","last_change":"2024-04-26T22:00:06.813137+0000","last_active":"2024-04-26T22:10:11.937218+0000","last_peered":"2024-04-26T22:10:11.937218+0000","last_clean":"2024-04-26T22:10:11.937218+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:10:11.937218+0000","last_undegraded":"2024-04-26T22:10:11.937218+0000","last_fullsized":"2024-04-26T22:10:11.937218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":56913,"num_write_kb":707433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"40'4759","reported_seq":4817,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803693+0000","last_change":"2024-04-26T21:59:57.645373+0000","last_active":"2024-04-26T22:10:05.803693+0000","last_peered":"2024-04-26T22:10:05.803693+0000","last_clean":"2024-04-26T22:10:05.803693+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:10:05.803693+0000","last_undegraded":"2024-04-26T22:10:05.803693+0000","last_fullsized":"2024-04-26T22:10:05.803693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71808,"num_write_kb":921006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"40'7","reported_seq":66,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.839915+0000","last_change":"2024-04-26T21:59:56.646131+0000","last_active":"2024-04-26T22:10:05.839915+0000","last_peered":"2024-04-26T22:10:05.839915+0000","last_clean":"2024-04-26T22:10:05.839915+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:10:05.839915+0000","last_undegraded":"2024-04-26T22:10:05.839915+0000","last_fullsized":"2024-04-26T22:10:05.839915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"40'183","reported_seq":242,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966795+0000","last_change":"2024-04-26T22:00:05.818789+0000","last_active":"2024-04-26T22:10:11.966795+0000","last_peered":"2024-04-26T22:10:11.966795+0000","last_clean":"2024-04-26T22:10:11.966795+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:10:11.966795+0000","last_undegraded":"2024-04-26T22:10:11.966795+0000","last_fullsized":"2024-04-26T22:10:11.966795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":27847,"num_write":173,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"40'3541","reported_seq":3610,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.940587+0000","last_change":"2024-04-26T22:00:06.818108+0000","last_active":"2024-04-26T22:10:11.940587+0000","last_peered":"2024-04-26T22:10:11.940587+0000","last_clean":"2024-04-26T22:10:11.940587+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:10:11.940587+0000","last_undegraded":"2024-04-26T22:10:11.940587+0000","last_fullsized":"2024-04-26T22:10:11.940587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":53073,"num_write_kb":661602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'5106","reported_seq":5164,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.813565+0000","last_change":"2024-04-26T21:59:57.639571+0000","last_active":"2024-04-26T22:10:05.813565+0000","last_peered":"2024-04-26T22:10:05.813565+0000","last_clean":"2024-04-26T22:10:05.813565+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:10:05.813565+0000","last_undegraded":"2024-04-26T22:10:05.813565+0000","last_fullsized":"2024-04-26T22:10:05.813565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:54.250576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":77293,"num_write_kb":982493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"40'221","reported_seq":286,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850755+0000","last_change":"2024-04-26T21:59:56.650254+0000","last_active":"2024-04-26T22:10:05.850755+0000","last_peered":"2024-04-26T22:10:05.850755+0000","last_clean":"2024-04-26T22:10:05.850755+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:10:05.850755+0000","last_undegraded":"2024-04-26T22:10:05.850755+0000","last_fullsized":"2024-04-26T22:10:05.850755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":213,"num_write_kb":8367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"40'5","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:20.475567+0000","last_change":"2024-04-26T22:10:20.475608+0000","last_active":"2024-04-26T22:10:20.475567+0000","last_peered":"2024-04-26T22:10:20.475567+0000","last_clean":"2024-04-26T22:10:20.475567+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:10:20.475567+0000","last_undegraded":"2024-04-26T22:10:20.475567+0000","last_fullsized":"2024-04-26T22:10:20.475567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:20.475457+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:20.475457+0000","last_clean_scrub_stamp":"2024-04-26T22:10:20.475457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:02.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"40'3541","reported_seq":3614,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.910214+0000","last_change":"2024-04-26T22:00:06.824327+0000","last_active":"2024-04-26T22:10:11.910214+0000","last_peered":"2024-04-26T22:10:11.910214+0000","last_clean":"2024-04-26T22:10:11.910214+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:10:11.910214+0000","last_undegraded":"2024-04-26T22:10:11.910214+0000","last_fullsized":"2024-04-26T22:10:11.910214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:21.673373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":52805,"num_write_kb":629973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'5114","reported_seq":5175,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.018575+0000","last_change":"2024-04-26T21:59:57.650020+0000","last_active":"2024-04-26T22:10:11.018575+0000","last_peered":"2024-04-26T22:10:11.018575+0000","last_clean":"2024-04-26T22:10:11.018575+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:10:11.018575+0000","last_undegraded":"2024-04-26T22:10:11.018575+0000","last_fullsized":"2024-04-26T22:10:11.018575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":77095,"num_write_kb":967489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"40'19","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.889177+0000","last_change":"2024-04-26T21:59:56.641689+0000","last_active":"2024-04-26T22:10:14.889177+0000","last_peered":"2024-04-26T22:10:14.889177+0000","last_clean":"2024-04-26T22:10:14.889177+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:10:14.889177+0000","last_undegraded":"2024-04-26T22:10:14.889177+0000","last_fullsized":"2024-04-26T22:10:14.889177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88181,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":134,"num_write":15,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"40'5","reported_seq":72,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.980329+0000","last_change":"2024-04-26T22:00:05.822516+0000","last_active":"2024-04-26T22:10:14.980329+0000","last_peered":"2024-04-26T22:10:14.980329+0000","last_clean":"2024-04-26T22:10:14.980329+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:10:14.980329+0000","last_undegraded":"2024-04-26T22:10:14.980329+0000","last_fullsized":"2024-04-26T22:10:14.980329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'4139","reported_seq":4216,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.947123+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:10:11.947123+0000","last_peered":"2024-04-26T22:10:11.947123+0000","last_clean":"2024-04-26T22:10:11.947123+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:10:11.947123+0000","last_undegraded":"2024-04-26T22:10:11.947123+0000","last_fullsized":"2024-04-26T22:10:11.947123+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":62199,"num_write_kb":728313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"40'5239","reported_seq":5297,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810597+0000","last_change":"2024-04-26T21:59:57.642509+0000","last_active":"2024-04-26T22:10:05.810597+0000","last_peered":"2024-04-26T22:10:05.810597+0000","last_clean":"2024-04-26T22:10:05.810597+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:10:05.810597+0000","last_undegraded":"2024-04-26T22:10:05.810597+0000","last_fullsized":"2024-04-26T22:10:05.810597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:40.956210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":79195,"num_write_kb":994121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"40'30","reported_seq":97,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.863131+0000","last_change":"2024-04-26T21:59:56.648084+0000","last_active":"2024-04-26T22:10:05.863131+0000","last_peered":"2024-04-26T22:10:05.863131+0000","last_clean":"2024-04-26T22:10:05.863131+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:10:05.863131+0000","last_undegraded":"2024-04-26T22:10:05.863131+0000","last_fullsized":"2024-04-26T22:10:05.863131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:48.280231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":8201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"40'211","reported_seq":268,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.956625+0000","last_change":"2024-04-26T22:00:05.815597+0000","last_active":"2024-04-26T22:10:11.956625+0000","last_peered":"2024-04-26T22:10:11.956625+0000","last_clean":"2024-04-26T22:10:11.956625+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:10:11.956625+0000","last_undegraded":"2024-04-26T22:10:11.956625+0000","last_fullsized":"2024-04-26T22:10:11.956625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":74,"num_write":424,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"40'4145","reported_seq":4221,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.957211+0000","last_change":"2024-04-26T22:00:06.817274+0000","last_active":"2024-04-26T22:10:11.957211+0000","last_peered":"2024-04-26T22:10:11.957211+0000","last_clean":"2024-04-26T22:10:11.957211+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:10:11.957211+0000","last_undegraded":"2024-04-26T22:10:11.957211+0000","last_fullsized":"2024-04-26T22:10:11.957211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":62372,"num_write_kb":775201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"40'5748","reported_seq":5806,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.779720+0000","last_change":"2024-04-26T21:59:57.657139+0000","last_active":"2024-04-26T22:10:05.779720+0000","last_peered":"2024-04-26T22:10:05.779720+0000","last_clean":"2024-04-26T22:10:05.779720+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:10:05.779720+0000","last_undegraded":"2024-04-26T22:10:05.779720+0000","last_fullsized":"2024-04-26T22:10:05.779720+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":87080,"num_write_kb":1065746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"39'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.940251+0000","last_change":"2024-04-26T21:59:56.641500+0000","last_active":"2024-04-26T22:10:00.940251+0000","last_peered":"2024-04-26T22:10:00.940251+0000","last_clean":"2024-04-26T22:10:00.940251+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:10:00.940251+0000","last_undegraded":"2024-04-26T22:10:00.940251+0000","last_fullsized":"2024-04-26T22:10:00.940251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:01.221589+0000","last_change":"2024-04-26T21:59:46.373113+0000","last_active":"2024-04-26T22:10:01.221589+0000","last_peered":"2024-04-26T22:10:01.221589+0000","last_clean":"2024-04-26T22:10:01.221589+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:10:01.221589+0000","last_undegraded":"2024-04-26T22:10:01.221589+0000","last_fullsized":"2024-04-26T22:10:01.221589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:45.086527+0000","last_clean_scrub_stamp":"2024-04-26T21:59:45.086527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:35.160126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.944318+0000","last_change":"2024-04-26T22:00:05.823565+0000","last_active":"2024-04-26T22:10:11.944318+0000","last_peered":"2024-04-26T22:10:11.944318+0000","last_clean":"2024-04-26T22:10:11.944318+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:10:11.944318+0000","last_undegraded":"2024-04-26T22:10:11.944318+0000","last_fullsized":"2024-04-26T22:10:11.944318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:27.319441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"40'3379","reported_seq":3460,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959450+0000","last_change":"2024-04-26T22:00:06.814390+0000","last_active":"2024-04-26T22:10:11.959450+0000","last_peered":"2024-04-26T22:10:11.959450+0000","last_clean":"2024-04-26T22:10:11.959450+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:10:11.959450+0000","last_undegraded":"2024-04-26T22:10:11.959450+0000","last_fullsized":"2024-04-26T22:10:11.959450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:36.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":50792,"num_write_kb":610820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"40'5433","reported_seq":5494,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.869142+0000","last_change":"2024-04-26T21:59:57.655942+0000","last_active":"2024-04-26T22:10:05.869142+0000","last_peered":"2024-04-26T22:10:05.869142+0000","last_clean":"2024-04-26T22:10:05.869142+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:10:05.869142+0000","last_undegraded":"2024-04-26T22:10:05.869142+0000","last_fullsized":"2024-04-26T22:10:05.869142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:27.620525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":81848,"num_write_kb":1014508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"40'3","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810386+0000","last_change":"2024-04-26T21:59:56.646732+0000","last_active":"2024-04-26T22:10:05.810386+0000","last_peered":"2024-04-26T22:10:05.810386+0000","last_clean":"2024-04-26T22:10:05.810386+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:10:05.810386+0000","last_undegraded":"2024-04-26T22:10:05.810386+0000","last_fullsized":"2024-04-26T22:10:05.810386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"40'201","reported_seq":252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.948751+0000","last_change":"2024-04-26T22:00:05.822933+0000","last_active":"2024-04-26T22:10:11.948751+0000","last_peered":"2024-04-26T22:10:11.948751+0000","last_clean":"2024-04-26T22:10:11.948751+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:10:11.948751+0000","last_undegraded":"2024-04-26T22:10:11.948751+0000","last_fullsized":"2024-04-26T22:10:11.948751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":196,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"40'3780","reported_seq":3857,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.815165+0000","last_change":"2024-04-26T22:00:06.823887+0000","last_active":"2024-04-26T22:10:11.815165+0000","last_peered":"2024-04-26T22:10:11.815165+0000","last_clean":"2024-04-26T22:10:11.815165+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:10:11.815165+0000","last_undegraded":"2024-04-26T22:10:11.815165+0000","last_fullsized":"2024-04-26T22:10:11.815165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:03.103588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":56837,"num_write_kb":699720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"40'5733","reported_seq":5794,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849434+0000","last_change":"2024-04-26T21:59:57.649287+0000","last_active":"2024-04-26T22:10:05.849434+0000","last_peered":"2024-04-26T22:10:05.849434+0000","last_clean":"2024-04-26T22:10:05.849434+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:10:05.849434+0000","last_undegraded":"2024-04-26T22:10:05.849434+0000","last_fullsized":"2024-04-26T22:10:05.849434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5733,"log_dups_size":0,"ondisk_log_size":5733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":86585,"num_write_kb":1068440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"40'19","reported_seq":89,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.804869+0000","last_change":"2024-04-26T21:59:56.644283+0000","last_active":"2024-04-26T22:10:05.804869+0000","last_peered":"2024-04-26T22:10:05.804869+0000","last_clean":"2024-04-26T22:10:05.804869+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:10:05.804869+0000","last_undegraded":"2024-04-26T22:10:05.804869+0000","last_fullsized":"2024-04-26T22:10:05.804869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16390,"num_write":17,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"40'379","reported_seq":446,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955307+0000","last_change":"2024-04-26T22:00:05.816337+0000","last_active":"2024-04-26T22:10:11.955307+0000","last_peered":"2024-04-26T22:10:11.955307+0000","last_clean":"2024-04-26T22:10:11.955307+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:10:11.955307+0000","last_undegraded":"2024-04-26T22:10:11.955307+0000","last_fullsized":"2024-04-26T22:10:11.955307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":25852,"num_write":364,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"40'3909","reported_seq":3987,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946493+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:10:11.946493+0000","last_peered":"2024-04-26T22:10:11.946493+0000","last_clean":"2024-04-26T22:10:11.946493+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:10:11.946493+0000","last_undegraded":"2024-04-26T22:10:11.946493+0000","last_fullsized":"2024-04-26T22:10:11.946493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":58819,"num_write_kb":699051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"40'6248","reported_seq":6306,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.823315+0000","last_change":"2024-04-26T21:59:57.658492+0000","last_active":"2024-04-26T22:10:05.823315+0000","last_peered":"2024-04-26T22:10:05.823315+0000","last_clean":"2024-04-26T22:10:05.823315+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:10:05.823315+0000","last_undegraded":"2024-04-26T22:10:05.823315+0000","last_fullsized":"2024-04-26T22:10:05.823315+0000","mapping_epoch":18,"log_start":"28'400","ondisk_log_start":"28'400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:06.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":94569,"num_write_kb":1160134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"40'111","reported_seq":183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850721+0000","last_change":"2024-04-26T21:59:56.642307+0000","last_active":"2024-04-26T22:10:05.850721+0000","last_peered":"2024-04-26T22:10:05.850721+0000","last_clean":"2024-04-26T22:10:05.850721+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:10:05.850721+0000","last_undegraded":"2024-04-26T22:10:05.850721+0000","last_fullsized":"2024-04-26T22:10:05.850721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":106,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"40'13","reported_seq":68,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.935130+0000","last_change":"2024-04-26T22:00:05.821010+0000","last_active":"2024-04-26T22:10:11.935130+0000","last_peered":"2024-04-26T22:10:11.935130+0000","last_clean":"2024-04-26T22:10:11.935130+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:10:11.935130+0000","last_undegraded":"2024-04-26T22:10:11.935130+0000","last_fullsized":"2024-04-26T22:10:11.935130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"40'4399","reported_seq":4473,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.897004+0000","last_change":"2024-04-26T22:00:06.815588+0000","last_active":"2024-04-26T22:10:11.897004+0000","last_peered":"2024-04-26T22:10:11.897004+0000","last_clean":"2024-04-26T22:10:11.897004+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:10:11.897004+0000","last_undegraded":"2024-04-26T22:10:11.897004+0000","last_fullsized":"2024-04-26T22:10:11.897004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:42.266643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":66147,"num_write_kb":822005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"40'5224","reported_seq":5282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822320+0000","last_change":"2024-04-26T21:59:57.658684+0000","last_active":"2024-04-26T22:10:05.822320+0000","last_peered":"2024-04-26T22:10:05.822320+0000","last_clean":"2024-04-26T22:10:05.822320+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:10:05.822320+0000","last_undegraded":"2024-04-26T22:10:05.822320+0000","last_fullsized":"2024-04-26T22:10:05.822320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":78957,"num_write_kb":961695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"40'16","reported_seq":83,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.849672+0000","last_change":"2024-04-26T21:59:56.646576+0000","last_active":"2024-04-26T22:10:05.849672+0000","last_peered":"2024-04-26T22:10:05.849672+0000","last_clean":"2024-04-26T22:10:05.849672+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:10:05.849672+0000","last_undegraded":"2024-04-26T22:10:05.849672+0000","last_fullsized":"2024-04-26T22:10:05.849672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:26.229655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":12294,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951413+0000","last_change":"2024-04-26T22:00:05.822839+0000","last_active":"2024-04-26T22:10:11.951413+0000","last_peered":"2024-04-26T22:10:11.951413+0000","last_clean":"2024-04-26T22:10:11.951413+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:10:11.951413+0000","last_undegraded":"2024-04-26T22:10:11.951413+0000","last_fullsized":"2024-04-26T22:10:11.951413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:35.782914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"40'3562","reported_seq":3638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982357+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:10:11.982357+0000","last_peered":"2024-04-26T22:10:11.982357+0000","last_clean":"2024-04-26T22:10:11.982357+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:10:11.982357+0000","last_undegraded":"2024-04-26T22:10:11.982357+0000","last_fullsized":"2024-04-26T22:10:11.982357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3562,"log_dups_size":0,"ondisk_log_size":3562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":53350,"num_write_kb":651982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"40'5336","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848168+0000","last_change":"2024-04-26T22:00:39.151213+0000","last_active":"2024-04-26T22:10:05.848168+0000","last_peered":"2024-04-26T22:10:05.848168+0000","last_clean":"2024-04-26T22:10:05.848168+0000","last_became_active":"2024-04-26T22:00:39.150992+0000","last_became_peered":"2024-04-26T22:00:39.150992+0000","last_unstale":"2024-04-26T22:10:05.848168+0000","last_undegraded":"2024-04-26T22:10:05.848168+0000","last_fullsized":"2024-04-26T22:10:05.848168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:26.086210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":80461,"num_write_kb":1019956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"40'287","reported_seq":383,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.839145+0000","last_change":"2024-04-26T21:59:56.639728+0000","last_active":"2024-04-26T22:10:05.839145+0000","last_peered":"2024-04-26T22:10:05.839145+0000","last_clean":"2024-04-26T22:10:05.839145+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:10:05.839145+0000","last_undegraded":"2024-04-26T22:10:05.839145+0000","last_fullsized":"2024-04-26T22:10:05.839145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":12387,"num_write":284,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"40'4","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954588+0000","last_change":"2024-04-26T22:00:05.810788+0000","last_active":"2024-04-26T22:10:11.954588+0000","last_peered":"2024-04-26T22:10:11.954588+0000","last_clean":"2024-04-26T22:10:11.954588+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:10:11.954588+0000","last_undegraded":"2024-04-26T22:10:11.954588+0000","last_fullsized":"2024-04-26T22:10:11.954588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"40'4185","reported_seq":4270,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.946111+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:10:11.946111+0000","last_peered":"2024-04-26T22:10:11.946111+0000","last_clean":"2024-04-26T22:10:11.946111+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:10:11.946111+0000","last_undegraded":"2024-04-26T22:10:11.946111+0000","last_fullsized":"2024-04-26T22:10:11.946111+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":62914,"num_write_kb":762533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'5750","reported_seq":5810,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.854474+0000","last_change":"2024-04-26T21:59:57.637231+0000","last_active":"2024-04-26T22:10:05.854474+0000","last_peered":"2024-04-26T22:10:05.854474+0000","last_clean":"2024-04-26T22:10:05.854474+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:10:05.854474+0000","last_undegraded":"2024-04-26T22:10:05.854474+0000","last_fullsized":"2024-04-26T22:10:05.854474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":86794,"num_write_kb":1073964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"40'6","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836648+0000","last_change":"2024-04-26T21:59:56.644913+0000","last_active":"2024-04-26T22:10:05.836648+0000","last_peered":"2024-04-26T22:10:05.836648+0000","last_clean":"2024-04-26T22:10:05.836648+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:10:05.836648+0000","last_undegraded":"2024-04-26T22:10:05.836648+0000","last_fullsized":"2024-04-26T22:10:05.836648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"40'3","reported_seq":53,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.943365+0000","last_change":"2024-04-26T22:00:05.815315+0000","last_active":"2024-04-26T22:10:07.943365+0000","last_peered":"2024-04-26T22:10:07.943365+0000","last_clean":"2024-04-26T22:10:07.943365+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:10:07.943365+0000","last_undegraded":"2024-04-26T22:10:07.943365+0000","last_fullsized":"2024-04-26T22:10:07.943365+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:03.410488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"40'3717","reported_seq":3786,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.778102+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:10:11.778102+0000","last_peered":"2024-04-26T22:10:11.778102+0000","last_clean":"2024-04-26T22:10:11.778102+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:10:11.778102+0000","last_undegraded":"2024-04-26T22:10:11.778102+0000","last_fullsized":"2024-04-26T22:10:11.778102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3717,"log_dups_size":0,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":55831,"num_write_kb":673382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"40'6101","reported_seq":6161,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868996+0000","last_change":"2024-04-26T21:59:57.659327+0000","last_active":"2024-04-26T22:10:05.868996+0000","last_peered":"2024-04-26T22:10:05.868996+0000","last_clean":"2024-04-26T22:10:05.868996+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:10:05.868996+0000","last_undegraded":"2024-04-26T22:10:05.868996+0000","last_fullsized":"2024-04-26T22:10:05.868996+0000","mapping_epoch":18,"log_start":"28'300","ondisk_log_start":"28'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:12.503750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":92157,"num_write_kb":1133252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"40'5","reported_seq":63,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867571+0000","last_change":"2024-04-26T21:59:56.651448+0000","last_active":"2024-04-26T22:10:05.867571+0000","last_peered":"2024-04-26T22:10:05.867571+0000","last_clean":"2024-04-26T22:10:05.867571+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:10:05.867571+0000","last_undegraded":"2024-04-26T22:10:05.867571+0000","last_fullsized":"2024-04-26T22:10:05.867571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:53.246460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"40'374","reported_seq":467,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953183+0000","last_change":"2024-04-26T22:00:05.820413+0000","last_active":"2024-04-26T22:10:11.953183+0000","last_peered":"2024-04-26T22:10:11.953183+0000","last_clean":"2024-04-26T22:10:11.953183+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:10:11.953183+0000","last_undegraded":"2024-04-26T22:10:11.953183+0000","last_fullsized":"2024-04-26T22:10:11.953183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":31157,"num_write":363,"num_write_kb":8442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"40'5464","reported_seq":5545,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.917366+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:10:11.917366+0000","last_peered":"2024-04-26T22:10:11.917366+0000","last_clean":"2024-04-26T22:10:11.917366+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:10:11.917366+0000","last_undegraded":"2024-04-26T22:10:11.917366+0000","last_fullsized":"2024-04-26T22:10:11.917366+0000","mapping_epoch":22,"log_start":"31'200","ondisk_log_start":"31'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":82133,"num_write_kb":980051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'5077","reported_seq":5133,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673410+0000","last_change":"2024-04-26T21:59:57.651860+0000","last_active":"2024-04-26T22:10:05.673410+0000","last_peered":"2024-04-26T22:10:05.673410+0000","last_clean":"2024-04-26T22:10:05.673410+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:10:05.673410+0000","last_undegraded":"2024-04-26T22:10:05.673410+0000","last_fullsized":"2024-04-26T22:10:05.673410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:27.460392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":76618,"num_write_kb":958363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"40'19","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852434+0000","last_change":"2024-04-26T21:59:56.641696+0000","last_active":"2024-04-26T22:10:05.852434+0000","last_peered":"2024-04-26T22:10:05.852434+0000","last_clean":"2024-04-26T22:10:05.852434+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:10:05.852434+0000","last_undegraded":"2024-04-26T22:10:05.852434+0000","last_fullsized":"2024-04-26T22:10:05.852434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":32816,"num_write":13,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"40'15","reported_seq":89,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.994445+0000","last_change":"2024-04-26T22:00:05.821413+0000","last_active":"2024-04-26T22:10:11.994445+0000","last_peered":"2024-04-26T22:10:11.994445+0000","last_clean":"2024-04-26T22:10:11.994445+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:10:11.994445+0000","last_undegraded":"2024-04-26T22:10:11.994445+0000","last_fullsized":"2024-04-26T22:10:11.994445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:52.573362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"40'4714","reported_seq":4797,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982824+0000","last_change":"2024-04-26T22:00:06.813143+0000","last_active":"2024-04-26T22:10:11.982824+0000","last_peered":"2024-04-26T22:10:11.982824+0000","last_clean":"2024-04-26T22:10:11.982824+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:10:11.982824+0000","last_undegraded":"2024-04-26T22:10:11.982824+0000","last_fullsized":"2024-04-26T22:10:11.982824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":70866,"num_write_kb":843772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"40'5114","reported_seq":5170,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.579883+0000","last_change":"2024-04-26T21:59:57.656528+0000","last_active":"2024-04-26T22:10:05.579883+0000","last_peered":"2024-04-26T22:10:05.579883+0000","last_clean":"2024-04-26T22:10:05.579883+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:10:05.579883+0000","last_undegraded":"2024-04-26T22:10:05.579883+0000","last_fullsized":"2024-04-26T22:10:05.579883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:35.389948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77180,"num_write_kb":958350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"40'824","reported_seq":917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.848033+0000","last_change":"2024-04-26T21:59:56.644097+0000","last_active":"2024-04-26T22:10:05.848033+0000","last_peered":"2024-04-26T22:10:05.848033+0000","last_clean":"2024-04-26T22:10:05.848033+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:10:05.848033+0000","last_undegraded":"2024-04-26T22:10:05.848033+0000","last_fullsized":"2024-04-26T22:10:05.848033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:20.307361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":36914,"num_write":813,"num_write_kb":13003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"40'5","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.980423+0000","last_change":"2024-04-26T22:00:05.820793+0000","last_active":"2024-04-26T22:10:14.980423+0000","last_peered":"2024-04-26T22:10:14.980423+0000","last_clean":"2024-04-26T22:10:14.980423+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:10:14.980423+0000","last_undegraded":"2024-04-26T22:10:14.980423+0000","last_fullsized":"2024-04-26T22:10:14.980423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"40'3745","reported_seq":3824,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.928243+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:10:11.928243+0000","last_peered":"2024-04-26T22:10:11.928243+0000","last_clean":"2024-04-26T22:10:11.928243+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:10:11.928243+0000","last_undegraded":"2024-04-26T22:10:11.928243+0000","last_fullsized":"2024-04-26T22:10:11.928243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56184,"num_write_kb":680067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"40'7267","reported_seq":7326,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824603+0000","last_change":"2024-04-26T21:59:57.649929+0000","last_active":"2024-04-26T22:10:05.824603+0000","last_peered":"2024-04-26T22:10:05.824603+0000","last_clean":"2024-04-26T22:10:05.824603+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:10:05.824603+0000","last_undegraded":"2024-04-26T22:10:05.824603+0000","last_fullsized":"2024-04-26T22:10:05.824603+0000","mapping_epoch":18,"log_start":"30'1800","ondisk_log_start":"30'1800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:49.284348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":109924,"num_write_kb":1323267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"40'370","reported_seq":448,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:14.892108+0000","last_change":"2024-04-26T21:59:56.649307+0000","last_active":"2024-04-26T22:10:14.892108+0000","last_peered":"2024-04-26T22:10:14.892108+0000","last_clean":"2024-04-26T22:10:14.892108+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:10:14.892108+0000","last_undegraded":"2024-04-26T22:10:14.892108+0000","last_fullsized":"2024-04-26T22:10:14.892108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":367,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"40'231","reported_seq":307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966124+0000","last_change":"2024-04-26T22:00:05.821589+0000","last_active":"2024-04-26T22:10:11.966124+0000","last_peered":"2024-04-26T22:10:11.966124+0000","last_clean":"2024-04-26T22:10:11.966124+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:10:11.966124+0000","last_undegraded":"2024-04-26T22:10:11.966124+0000","last_fullsized":"2024-04-26T22:10:11.966124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:41.259409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":33257,"num_write":223,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"40'4111","reported_seq":4188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967266+0000","last_change":"2024-04-26T22:00:06.816410+0000","last_active":"2024-04-26T22:10:11.967266+0000","last_peered":"2024-04-26T22:10:11.967266+0000","last_clean":"2024-04-26T22:10:11.967266+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:10:11.967266+0000","last_undegraded":"2024-04-26T22:10:11.967266+0000","last_fullsized":"2024-04-26T22:10:11.967266+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4111,"log_dups_size":0,"ondisk_log_size":4111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:41.149237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61736,"num_write_kb":766472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"40'4628","reported_seq":4688,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.867072+0000","last_change":"2024-04-26T21:59:57.644861+0000","last_active":"2024-04-26T22:10:05.867072+0000","last_peered":"2024-04-26T22:10:05.867072+0000","last_clean":"2024-04-26T22:10:05.867072+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:10:05.867072+0000","last_undegraded":"2024-04-26T22:10:05.867072+0000","last_fullsized":"2024-04-26T22:10:05.867072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":69686,"num_write_kb":866678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"40'102","reported_seq":162,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842367+0000","last_change":"2024-04-26T21:59:56.647201+0000","last_active":"2024-04-26T22:10:05.842367+0000","last_peered":"2024-04-26T22:10:05.842367+0000","last_clean":"2024-04-26T22:10:05.842367+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:10:05.842367+0000","last_undegraded":"2024-04-26T22:10:05.842367+0000","last_fullsized":"2024-04-26T22:10:05.842367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":98,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"40'12","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.943439+0000","last_change":"2024-04-26T22:00:05.820698+0000","last_active":"2024-04-26T22:10:11.943439+0000","last_peered":"2024-04-26T22:10:11.943439+0000","last_clean":"2024-04-26T22:10:11.943439+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:10:11.943439+0000","last_undegraded":"2024-04-26T22:10:11.943439+0000","last_fullsized":"2024-04-26T22:10:11.943439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:54:40.142002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'4365","reported_seq":4453,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.968703+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:10:11.968703+0000","last_peered":"2024-04-26T22:10:11.968703+0000","last_clean":"2024-04-26T22:10:11.968703+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:10:11.968703+0000","last_undegraded":"2024-04-26T22:10:11.968703+0000","last_fullsized":"2024-04-26T22:10:11.968703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4365,"log_dups_size":0,"ondisk_log_size":4365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:44.028941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":65682,"num_write_kb":806692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"40'5549","reported_seq":5613,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.824597+0000","last_change":"2024-04-26T21:59:57.652599+0000","last_active":"2024-04-26T22:10:05.824597+0000","last_peered":"2024-04-26T22:10:05.824597+0000","last_clean":"2024-04-26T22:10:05.824597+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:10:05.824597+0000","last_undegraded":"2024-04-26T22:10:05.824597+0000","last_fullsized":"2024-04-26T22:10:05.824597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":83744,"num_write_kb":1000289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"40'4","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.846233+0000","last_change":"2024-04-26T21:59:56.648755+0000","last_active":"2024-04-26T22:10:05.846233+0000","last_peered":"2024-04-26T22:10:05.846233+0000","last_clean":"2024-04-26T22:10:05.846233+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:10:05.846233+0000","last_undegraded":"2024-04-26T22:10:05.846233+0000","last_fullsized":"2024-04-26T22:10:05.846233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:57.603603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'6","reported_seq":56,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953872+0000","last_change":"2024-04-26T22:00:05.817463+0000","last_active":"2024-04-26T22:10:11.953872+0000","last_peered":"2024-04-26T22:10:11.953872+0000","last_clean":"2024-04-26T22:10:11.953872+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:10:11.953872+0000","last_undegraded":"2024-04-26T22:10:11.953872+0000","last_fullsized":"2024-04-26T22:10:11.953872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:42.806480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4980","reported_seq":5062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.967201+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:10:11.967201+0000","last_peered":"2024-04-26T22:10:11.967201+0000","last_clean":"2024-04-26T22:10:11.967201+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:10:11.967201+0000","last_undegraded":"2024-04-26T22:10:11.967201+0000","last_fullsized":"2024-04-26T22:10:11.967201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":74941,"num_write_kb":925745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"40'16","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.970996+0000","last_change":"2024-04-26T22:00:05.813022+0000","last_active":"2024-04-26T22:10:11.970996+0000","last_peered":"2024-04-26T22:10:11.970996+0000","last_clean":"2024-04-26T22:10:11.970996+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:10:11.970996+0000","last_undegraded":"2024-04-26T22:10:11.970996+0000","last_fullsized":"2024-04-26T22:10:11.970996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:43.552337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":8,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"40'6345","reported_seq":6407,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874595+0000","last_change":"2024-04-26T21:59:57.646706+0000","last_active":"2024-04-26T22:10:05.874595+0000","last_peered":"2024-04-26T22:10:05.874595+0000","last_clean":"2024-04-26T22:10:05.874595+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:10:05.874595+0000","last_undegraded":"2024-04-26T22:10:05.874595+0000","last_fullsized":"2024-04-26T22:10:05.874595+0000","mapping_epoch":18,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":95977,"num_write_kb":1176758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"40'19","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.772388+0000","last_change":"2024-04-26T21:59:56.645221+0000","last_active":"2024-04-26T22:10:05.772388+0000","last_peered":"2024-04-26T22:10:05.772388+0000","last_clean":"2024-04-26T22:10:05.772388+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:10:05.772388+0000","last_undegraded":"2024-04-26T22:10:05.772388+0000","last_fullsized":"2024-04-26T22:10:05.772388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:13.476887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":28672,"num_write":16,"num_write_kb":8206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"40'4260","reported_seq":4329,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.876539+0000","last_change":"2024-04-26T22:00:06.814238+0000","last_active":"2024-04-26T22:10:11.876539+0000","last_peered":"2024-04-26T22:10:11.876539+0000","last_clean":"2024-04-26T22:10:11.876539+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:10:11.876539+0000","last_undegraded":"2024-04-26T22:10:11.876539+0000","last_fullsized":"2024-04-26T22:10:11.876539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":64151,"num_write_kb":776804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'25","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.953756+0000","last_change":"2024-04-26T22:00:05.819640+0000","last_active":"2024-04-26T22:10:00.953756+0000","last_peered":"2024-04-26T22:10:00.953756+0000","last_clean":"2024-04-26T22:10:00.953756+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:10:00.953756+0000","last_undegraded":"2024-04-26T22:10:00.953756+0000","last_fullsized":"2024-04-26T22:10:00.953756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:50.347140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20512,"num_write":20,"num_write_kb":4102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"40'5323","reported_seq":5382,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874479+0000","last_change":"2024-04-26T21:59:57.658456+0000","last_active":"2024-04-26T22:10:05.874479+0000","last_peered":"2024-04-26T22:10:05.874479+0000","last_clean":"2024-04-26T22:10:05.874479+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:10:05.874479+0000","last_undegraded":"2024-04-26T22:10:05.874479+0000","last_fullsized":"2024-04-26T22:10:05.874479+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":80432,"num_write_kb":1004371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"40'46","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.934549+0000","last_change":"2024-04-26T21:59:56.644013+0000","last_active":"2024-04-26T22:10:00.934549+0000","last_peered":"2024-04-26T22:10:00.934549+0000","last_clean":"2024-04-26T22:10:00.934549+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:10:00.934549+0000","last_undegraded":"2024-04-26T22:10:00.934549+0000","last_fullsized":"2024-04-26T22:10:00.934549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:02.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":41,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'4905","reported_seq":4992,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.989940+0000","last_change":"2024-04-26T22:00:06.824205+0000","last_active":"2024-04-26T22:10:11.989940+0000","last_peered":"2024-04-26T22:10:11.989940+0000","last_clean":"2024-04-26T22:10:11.989940+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:10:11.989940+0000","last_undegraded":"2024-04-26T22:10:11.989940+0000","last_fullsized":"2024-04-26T22:10:11.989940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":74018,"num_write_kb":905043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"40'145","reported_seq":213,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.833871+0000","last_change":"2024-04-26T21:59:56.645015+0000","last_active":"2024-04-26T22:10:05.833871+0000","last_peered":"2024-04-26T22:10:05.833871+0000","last_clean":"2024-04-26T22:10:05.833871+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:10:05.833871+0000","last_undegraded":"2024-04-26T22:10:05.833871+0000","last_fullsized":"2024-04-26T22:10:05.833871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:38.166272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":136,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"40'4250","reported_seq":4307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.802351+0000","last_change":"2024-04-26T21:59:57.642657+0000","last_active":"2024-04-26T22:10:05.802351+0000","last_peered":"2024-04-26T22:10:05.802351+0000","last_clean":"2024-04-26T22:10:05.802351+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:10:05.802351+0000","last_undegraded":"2024-04-26T22:10:05.802351+0000","last_fullsized":"2024-04-26T22:10:05.802351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":64229,"num_write_kb":785523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"40'38","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:07.026729+0000","last_change":"2024-04-26T22:00:05.812030+0000","last_active":"2024-04-26T22:10:07.026729+0000","last_peered":"2024-04-26T22:10:07.026729+0000","last_clean":"2024-04-26T22:10:07.026729+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:10:07.026729+0000","last_undegraded":"2024-04-26T22:10:07.026729+0000","last_fullsized":"2024-04-26T22:10:07.026729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:44:35.622781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":35,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2966","reported_seq":3026,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.990722+0000","last_change":"2024-04-26T22:00:06.817145+0000","last_active":"2024-04-26T22:10:11.990722+0000","last_peered":"2024-04-26T22:10:11.990722+0000","last_clean":"2024-04-26T22:10:11.990722+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:10:11.990722+0000","last_undegraded":"2024-04-26T22:10:11.990722+0000","last_fullsized":"2024-04-26T22:10:11.990722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":2966,"log_dups_size":0,"ondisk_log_size":2966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:35.255833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":44534,"num_write_kb":522679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"40'1255","reported_seq":1319,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:00.935885+0000","last_change":"2024-04-26T21:59:56.645427+0000","last_active":"2024-04-26T22:10:00.935885+0000","last_peered":"2024-04-26T22:10:00.935885+0000","last_clean":"2024-04-26T22:10:00.935885+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:10:00.935885+0000","last_undegraded":"2024-04-26T22:10:00.935885+0000","last_fullsized":"2024-04-26T22:10:00.935885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":1255,"log_dups_size":0,"ondisk_log_size":1255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:10.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":20480,"num_write":1252,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"40'4813","reported_seq":4872,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836289+0000","last_change":"2024-04-26T21:59:57.643226+0000","last_active":"2024-04-26T22:10:05.836289+0000","last_peered":"2024-04-26T22:10:05.836289+0000","last_clean":"2024-04-26T22:10:05.836289+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:10:05.836289+0000","last_undegraded":"2024-04-26T22:10:05.836289+0000","last_fullsized":"2024-04-26T22:10:05.836289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:58.607720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":72672,"num_write_kb":866290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"40'86","reported_seq":166,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953507+0000","last_change":"2024-04-26T22:00:05.820723+0000","last_active":"2024-04-26T22:10:11.953507+0000","last_peered":"2024-04-26T22:10:11.953507+0000","last_clean":"2024-04-26T22:10:11.953507+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:10:11.953507+0000","last_undegraded":"2024-04-26T22:10:11.953507+0000","last_fullsized":"2024-04-26T22:10:11.953507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:04.784649+0000","last_clean_scrub_stamp":"2024-04-26T22:00:04.784649+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:09.914838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":62,"num_read_kb":49200,"num_write":70,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"40'3892","reported_seq":3964,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954209+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:10:11.954209+0000","last_peered":"2024-04-26T22:10:11.954209+0000","last_clean":"2024-04-26T22:10:11.954209+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:10:11.954209+0000","last_undegraded":"2024-04-26T22:10:11.954209+0000","last_fullsized":"2024-04-26T22:10:11.954209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:00.976450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":58573,"num_write_kb":703698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"40'6654","reported_seq":6712,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.836315+0000","last_change":"2024-04-26T21:59:57.639754+0000","last_active":"2024-04-26T22:10:05.836315+0000","last_peered":"2024-04-26T22:10:05.836315+0000","last_clean":"2024-04-26T22:10:05.836315+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:10:05.836315+0000","last_undegraded":"2024-04-26T22:10:05.836315+0000","last_fullsized":"2024-04-26T22:10:05.836315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":100552,"num_write_kb":1253937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"40'4775","reported_seq":4860,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.981653+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:10:11.981653+0000","last_peered":"2024-04-26T22:10:11.981653+0000","last_clean":"2024-04-26T22:10:11.981653+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:10:11.981653+0000","last_undegraded":"2024-04-26T22:10:11.981653+0000","last_fullsized":"2024-04-26T22:10:11.981653+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":71741,"num_write_kb":892635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"40'5861","reported_seq":5917,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.673275+0000","last_change":"2024-04-26T21:59:57.653765+0000","last_active":"2024-04-26T22:10:05.673275+0000","last_peered":"2024-04-26T22:10:05.673275+0000","last_clean":"2024-04-26T22:10:05.673275+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:10:05.673275+0000","last_undegraded":"2024-04-26T22:10:05.673275+0000","last_fullsized":"2024-04-26T22:10:05.673275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:08.952634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88543,"num_write_kb":1086952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"40'3712","reported_seq":3779,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.937208+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:10:11.937208+0000","last_peered":"2024-04-26T22:10:11.937208+0000","last_clean":"2024-04-26T22:10:11.937208+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:10:11.937208+0000","last_undegraded":"2024-04-26T22:10:11.937208+0000","last_fullsized":"2024-04-26T22:10:11.937208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":55802,"num_write_kb":677006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"40'6511","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.793662+0000","last_change":"2024-04-26T22:00:39.200941+0000","last_active":"2024-04-26T22:10:05.793662+0000","last_peered":"2024-04-26T22:10:05.793662+0000","last_clean":"2024-04-26T22:10:05.793662+0000","last_became_active":"2024-04-26T22:00:39.200592+0000","last_became_peered":"2024-04-26T22:00:39.200592+0000","last_unstale":"2024-04-26T22:10:05.793662+0000","last_undegraded":"2024-04-26T22:10:05.793662+0000","last_fullsized":"2024-04-26T22:10:05.793662+0000","mapping_epoch":25,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:59.988559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":98396,"num_write_kb":1218831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"40'4288","reported_seq":4365,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988267+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:10:11.988267+0000","last_peered":"2024-04-26T22:10:11.988267+0000","last_clean":"2024-04-26T22:10:11.988267+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:10:11.988267+0000","last_undegraded":"2024-04-26T22:10:11.988267+0000","last_fullsized":"2024-04-26T22:10:11.988267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":64735,"num_write_kb":800999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"40'4650","reported_seq":4707,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.852016+0000","last_change":"2024-04-26T21:59:57.645588+0000","last_active":"2024-04-26T22:10:05.852016+0000","last_peered":"2024-04-26T22:10:05.852016+0000","last_clean":"2024-04-26T22:10:05.852016+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:10:05.852016+0000","last_undegraded":"2024-04-26T22:10:05.852016+0000","last_fullsized":"2024-04-26T22:10:05.852016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":70263,"num_write_kb":869775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"40'5113","reported_seq":5194,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.915535+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:10:11.915535+0000","last_peered":"2024-04-26T22:10:11.915535+0000","last_clean":"2024-04-26T22:10:11.915535+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:10:11.915535+0000","last_undegraded":"2024-04-26T22:10:11.915535+0000","last_fullsized":"2024-04-26T22:10:11.915535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":76970,"num_write_kb":914783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"40'4980","reported_seq":5038,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.818657+0000","last_change":"2024-04-26T21:59:57.647022+0000","last_active":"2024-04-26T22:10:05.818657+0000","last_peered":"2024-04-26T22:10:05.818657+0000","last_clean":"2024-04-26T22:10:05.818657+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:10:05.818657+0000","last_undegraded":"2024-04-26T22:10:05.818657+0000","last_fullsized":"2024-04-26T22:10:05.818657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":75382,"num_write_kb":914602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"40'3556","reported_seq":3628,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.958144+0000","last_change":"2024-04-26T22:00:06.816073+0000","last_active":"2024-04-26T22:10:11.958144+0000","last_peered":"2024-04-26T22:10:11.958144+0000","last_clean":"2024-04-26T22:10:11.958144+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:10:11.958144+0000","last_undegraded":"2024-04-26T22:10:11.958144+0000","last_fullsized":"2024-04-26T22:10:11.958144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:37.914525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":53572,"num_write_kb":653683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"40'5738","reported_seq":5796,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874637+0000","last_change":"2024-04-26T21:59:57.644835+0000","last_active":"2024-04-26T22:10:05.874637+0000","last_peered":"2024-04-26T22:10:05.874637+0000","last_clean":"2024-04-26T22:10:05.874637+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:10:05.874637+0000","last_undegraded":"2024-04-26T22:10:05.874637+0000","last_fullsized":"2024-04-26T22:10:05.874637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":86675,"num_write_kb":1103247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"40'5159","reported_seq":5238,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.951137+0000","last_change":"2024-04-26T22:00:06.818727+0000","last_active":"2024-04-26T22:10:11.951137+0000","last_peered":"2024-04-26T22:10:11.951137+0000","last_clean":"2024-04-26T22:10:11.951137+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:10:11.951137+0000","last_undegraded":"2024-04-26T22:10:11.951137+0000","last_fullsized":"2024-04-26T22:10:11.951137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:34:25.549984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":77704,"num_write_kb":942965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"40'4895","reported_seq":4967,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842652+0000","last_change":"2024-04-26T21:59:57.649676+0000","last_active":"2024-04-26T22:10:05.842652+0000","last_peered":"2024-04-26T22:10:05.842652+0000","last_clean":"2024-04-26T22:10:05.842652+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:10:05.842652+0000","last_undegraded":"2024-04-26T22:10:05.842652+0000","last_fullsized":"2024-04-26T22:10:05.842652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:26.905400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":363,"num_write":74201,"num_write_kb":908910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"40'4370","reported_seq":4439,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.838189+0000","last_change":"2024-04-26T22:00:06.817091+0000","last_active":"2024-04-26T22:10:11.838189+0000","last_peered":"2024-04-26T22:10:11.838189+0000","last_clean":"2024-04-26T22:10:11.838189+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:10:11.838189+0000","last_undegraded":"2024-04-26T22:10:11.838189+0000","last_fullsized":"2024-04-26T22:10:11.838189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4370,"log_dups_size":0,"ondisk_log_size":4370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:53.580596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":65733,"num_write_kb":804896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"40'6465","reported_seq":6525,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.853311+0000","last_change":"2024-04-26T21:59:57.660376+0000","last_active":"2024-04-26T22:10:05.853311+0000","last_peered":"2024-04-26T22:10:05.853311+0000","last_clean":"2024-04-26T22:10:05.853311+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:10:05.853311+0000","last_undegraded":"2024-04-26T22:10:05.853311+0000","last_fullsized":"2024-04-26T22:10:05.853311+0000","mapping_epoch":18,"log_start":"30'1200","ondisk_log_start":"30'1200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5265,"log_dups_size":0,"ondisk_log_size":5265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":97620,"num_write_kb":1192207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"40'3334","reported_seq":3406,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982941+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:10:11.982941+0000","last_peered":"2024-04-26T22:10:11.982941+0000","last_clean":"2024-04-26T22:10:11.982941+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:10:11.982941+0000","last_undegraded":"2024-04-26T22:10:11.982941+0000","last_fullsized":"2024-04-26T22:10:11.982941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:01.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":49724,"num_write_kb":588291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"40'4839","reported_seq":4896,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.857008+0000","last_change":"2024-04-26T21:59:57.651875+0000","last_active":"2024-04-26T22:10:05.857008+0000","last_peered":"2024-04-26T22:10:05.857008+0000","last_clean":"2024-04-26T22:10:05.857008+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:10:05.857008+0000","last_undegraded":"2024-04-26T22:10:05.857008+0000","last_fullsized":"2024-04-26T22:10:05.857008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:33.097021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":73090,"num_write_kb":888426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"40'5009","reported_seq":5079,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.919327+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:10:11.919327+0000","last_peered":"2024-04-26T22:10:11.919327+0000","last_clean":"2024-04-26T22:10:11.919327+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:10:11.919327+0000","last_undegraded":"2024-04-26T22:10:11.919327+0000","last_fullsized":"2024-04-26T22:10:11.919327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:14.507183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":75285,"num_write_kb":916871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"40'5333","reported_seq":5390,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.697837+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:10:05.697837+0000","last_peered":"2024-04-26T22:10:05.697837+0000","last_clean":"2024-04-26T22:10:05.697837+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:10:05.697837+0000","last_undegraded":"2024-04-26T22:10:05.697837+0000","last_fullsized":"2024-04-26T22:10:05.697837+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:27.129100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80596,"num_write_kb":994453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"40'4157","reported_seq":4231,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.959972+0000","last_change":"2024-04-26T22:00:06.824366+0000","last_active":"2024-04-26T22:10:11.959972+0000","last_peered":"2024-04-26T22:10:11.959972+0000","last_clean":"2024-04-26T22:10:11.959972+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:10:11.959972+0000","last_undegraded":"2024-04-26T22:10:11.959972+0000","last_fullsized":"2024-04-26T22:10:11.959972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:44.559571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":62491,"num_write_kb":760147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"40'5861","reported_seq":5904,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835233+0000","last_change":"2024-04-26T22:00:38.947156+0000","last_active":"2024-04-26T22:10:05.835233+0000","last_peered":"2024-04-26T22:10:05.835233+0000","last_clean":"2024-04-26T22:10:05.835233+0000","last_became_active":"2024-04-26T22:00:38.946980+0000","last_became_peered":"2024-04-26T22:00:38.946980+0000","last_unstale":"2024-04-26T22:10:05.835233+0000","last_undegraded":"2024-04-26T22:10:05.835233+0000","last_fullsized":"2024-04-26T22:10:05.835233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:02.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88698,"num_write_kb":1108811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"40'4156","reported_seq":4228,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.977006+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:10:11.977006+0000","last_peered":"2024-04-26T22:10:11.977006+0000","last_clean":"2024-04-26T22:10:11.977006+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:10:11.977006+0000","last_undegraded":"2024-04-26T22:10:11.977006+0000","last_fullsized":"2024-04-26T22:10:11.977006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62195,"num_write_kb":734457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"40'5569","reported_seq":5629,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.868845+0000","last_change":"2024-04-26T21:59:57.647786+0000","last_active":"2024-04-26T22:10:05.868845+0000","last_peered":"2024-04-26T22:10:05.868845+0000","last_clean":"2024-04-26T22:10:05.868845+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:10:05.868845+0000","last_undegraded":"2024-04-26T22:10:05.868845+0000","last_fullsized":"2024-04-26T22:10:05.868845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5569,"log_dups_size":0,"ondisk_log_size":5569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.336111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":83899,"num_write_kb":1024385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"40'4460","reported_seq":4531,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.955590+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:10:11.955590+0000","last_peered":"2024-04-26T22:10:11.955590+0000","last_clean":"2024-04-26T22:10:11.955590+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:10:11.955590+0000","last_undegraded":"2024-04-26T22:10:11.955590+0000","last_fullsized":"2024-04-26T22:10:11.955590+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:11.984476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":67080,"num_write_kb":827171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"40'5284","reported_seq":5342,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.845154+0000","last_change":"2024-04-26T21:59:57.638366+0000","last_active":"2024-04-26T22:10:05.845154+0000","last_peered":"2024-04-26T22:10:05.845154+0000","last_clean":"2024-04-26T22:10:05.845154+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:10:05.845154+0000","last_undegraded":"2024-04-26T22:10:05.845154+0000","last_fullsized":"2024-04-26T22:10:05.845154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:09.397829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":79924,"num_write_kb":1000899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"40'4361","reported_seq":4441,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972500+0000","last_change":"2024-04-26T22:00:06.816302+0000","last_active":"2024-04-26T22:10:11.972500+0000","last_peered":"2024-04-26T22:10:11.972500+0000","last_clean":"2024-04-26T22:10:11.972500+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:10:11.972500+0000","last_undegraded":"2024-04-26T22:10:11.972500+0000","last_fullsized":"2024-04-26T22:10:11.972500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:04.417722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65464,"num_write_kb":794873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"40'5458","reported_seq":5516,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.860815+0000","last_change":"2024-04-26T21:59:57.655192+0000","last_active":"2024-04-26T22:10:05.860815+0000","last_peered":"2024-04-26T22:10:05.860815+0000","last_clean":"2024-04-26T22:10:05.860815+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:10:05.860815+0000","last_undegraded":"2024-04-26T22:10:05.860815+0000","last_fullsized":"2024-04-26T22:10:05.860815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":82585,"num_write_kb":1000929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"40'3624","reported_seq":3690,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.966367+0000","last_change":"2024-04-26T22:00:06.817689+0000","last_active":"2024-04-26T22:10:11.966367+0000","last_peered":"2024-04-26T22:10:11.966367+0000","last_clean":"2024-04-26T22:10:11.966367+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:10:11.966367+0000","last_undegraded":"2024-04-26T22:10:11.966367+0000","last_fullsized":"2024-04-26T22:10:11.966367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":54495,"num_write_kb":662506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"40'4431","reported_seq":4495,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.870020+0000","last_change":"2024-04-26T21:59:57.651765+0000","last_active":"2024-04-26T22:10:05.870020+0000","last_peered":"2024-04-26T22:10:05.870020+0000","last_clean":"2024-04-26T22:10:05.870020+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:10:05.870020+0000","last_undegraded":"2024-04-26T22:10:05.870020+0000","last_fullsized":"2024-04-26T22:10:05.870020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.890746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":66582,"num_write_kb":814643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"40'4089","reported_seq":4165,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.832376+0000","last_change":"2024-04-26T22:00:06.818342+0000","last_active":"2024-04-26T22:10:11.832376+0000","last_peered":"2024-04-26T22:10:11.832376+0000","last_clean":"2024-04-26T22:10:11.832376+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:10:11.832376+0000","last_undegraded":"2024-04-26T22:10:11.832376+0000","last_fullsized":"2024-04-26T22:10:11.832376+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4089,"log_dups_size":0,"ondisk_log_size":4089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:58.591699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61370,"num_write_kb":735117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"40'6428","reported_seq":6487,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.822613+0000","last_change":"2024-04-26T21:59:57.652651+0000","last_active":"2024-04-26T22:10:05.822613+0000","last_peered":"2024-04-26T22:10:05.822613+0000","last_clean":"2024-04-26T22:10:05.822613+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:10:05.822613+0000","last_undegraded":"2024-04-26T22:10:05.822613+0000","last_fullsized":"2024-04-26T22:10:05.822613+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:37.015775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":97094,"num_write_kb":1219735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"40'4506","reported_seq":4589,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954478+0000","last_change":"2024-04-26T22:00:06.818663+0000","last_active":"2024-04-26T22:10:11.954478+0000","last_peered":"2024-04-26T22:10:11.954478+0000","last_clean":"2024-04-26T22:10:11.954478+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:10:11.954478+0000","last_undegraded":"2024-04-26T22:10:11.954478+0000","last_fullsized":"2024-04-26T22:10:11.954478+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":67596,"num_write_kb":862862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"40'4747","reported_seq":4805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.803411+0000","last_change":"2024-04-26T21:59:57.658356+0000","last_active":"2024-04-26T22:10:05.803411+0000","last_peered":"2024-04-26T22:10:05.803411+0000","last_clean":"2024-04-26T22:10:05.803411+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:10:05.803411+0000","last_undegraded":"2024-04-26T22:10:05.803411+0000","last_fullsized":"2024-04-26T22:10:05.803411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:42.225860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71795,"num_write_kb":917779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"40'4901","reported_seq":4976,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.894392+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:10:11.894392+0000","last_peered":"2024-04-26T22:10:11.894392+0000","last_clean":"2024-04-26T22:10:11.894392+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:10:11.894392+0000","last_undegraded":"2024-04-26T22:10:11.894392+0000","last_fullsized":"2024-04-26T22:10:11.894392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4901,"log_dups_size":0,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:15.700238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":73795,"num_write_kb":926135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"40'5581","reported_seq":5640,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.850772+0000","last_change":"2024-04-26T21:59:57.645554+0000","last_active":"2024-04-26T22:10:05.850772+0000","last_peered":"2024-04-26T22:10:05.850772+0000","last_clean":"2024-04-26T22:10:05.850772+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:10:05.850772+0000","last_undegraded":"2024-04-26T22:10:05.850772+0000","last_fullsized":"2024-04-26T22:10:05.850772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5581,"log_dups_size":0,"ondisk_log_size":5581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":84297,"num_write_kb":1026829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"40'3600","reported_seq":3671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.933325+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:10:11.933325+0000","last_peered":"2024-04-26T22:10:11.933325+0000","last_clean":"2024-04-26T22:10:11.933325+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:10:11.933325+0000","last_undegraded":"2024-04-26T22:10:11.933325+0000","last_fullsized":"2024-04-26T22:10:11.933325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:44.415695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":54091,"num_write_kb":649095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"40'5612","reported_seq":5671,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.789467+0000","last_change":"2024-04-26T21:59:57.643945+0000","last_active":"2024-04-26T22:10:05.789467+0000","last_peered":"2024-04-26T22:10:05.789467+0000","last_clean":"2024-04-26T22:10:05.789467+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:10:05.789467+0000","last_undegraded":"2024-04-26T22:10:05.789467+0000","last_fullsized":"2024-04-26T22:10:05.789467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:13:57.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":84765,"num_write_kb":1047612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"40'3974","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.882273+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:10:11.882273+0000","last_peered":"2024-04-26T22:10:11.882273+0000","last_clean":"2024-04-26T22:10:11.882273+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:10:11.882273+0000","last_undegraded":"2024-04-26T22:10:11.882273+0000","last_fullsized":"2024-04-26T22:10:11.882273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3974,"log_dups_size":0,"ondisk_log_size":3974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:38.743178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":59852,"num_write_kb":718261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"40'4819","reported_seq":4881,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.858034+0000","last_change":"2024-04-26T21:59:57.653621+0000","last_active":"2024-04-26T22:10:05.858034+0000","last_peered":"2024-04-26T22:10:05.858034+0000","last_clean":"2024-04-26T22:10:05.858034+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:10:05.858034+0000","last_undegraded":"2024-04-26T22:10:05.858034+0000","last_fullsized":"2024-04-26T22:10:05.858034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:33.815111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":72748,"num_write_kb":932622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"40'4140","reported_seq":4220,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.931602+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:10:11.931602+0000","last_peered":"2024-04-26T22:10:11.931602+0000","last_clean":"2024-04-26T22:10:11.931602+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:10:11.931602+0000","last_undegraded":"2024-04-26T22:10:11.931602+0000","last_fullsized":"2024-04-26T22:10:11.931602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":62174,"num_write_kb":775596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"40'4503","reported_seq":4562,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.797708+0000","last_change":"2024-04-26T21:59:57.638107+0000","last_active":"2024-04-26T22:10:05.797708+0000","last_peered":"2024-04-26T22:10:05.797708+0000","last_clean":"2024-04-26T22:10:05.797708+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:10:05.797708+0000","last_undegraded":"2024-04-26T22:10:05.797708+0000","last_fullsized":"2024-04-26T22:10:05.797708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:49:39.170389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":68209,"num_write_kb":848035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"40'3768","reported_seq":3848,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.978793+0000","last_change":"2024-04-26T22:00:06.815494+0000","last_active":"2024-04-26T22:10:11.978793+0000","last_peered":"2024-04-26T22:10:11.978793+0000","last_clean":"2024-04-26T22:10:11.978793+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:10:11.978793+0000","last_undegraded":"2024-04-26T22:10:11.978793+0000","last_fullsized":"2024-04-26T22:10:11.978793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:06.178657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":56583,"num_write_kb":708038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"40'5303","reported_seq":5359,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.717590+0000","last_change":"2024-04-26T21:59:57.645280+0000","last_active":"2024-04-26T22:10:05.717590+0000","last_peered":"2024-04-26T22:10:05.717590+0000","last_clean":"2024-04-26T22:10:05.717590+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:10:05.717590+0000","last_undegraded":"2024-04-26T22:10:05.717590+0000","last_fullsized":"2024-04-26T22:10:05.717590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:36.445590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80229,"num_write_kb":963753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"40'3667","reported_seq":3745,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.927871+0000","last_change":"2024-04-26T22:00:06.818652+0000","last_active":"2024-04-26T22:10:11.927871+0000","last_peered":"2024-04-26T22:10:11.927871+0000","last_clean":"2024-04-26T22:10:11.927871+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:10:11.927871+0000","last_undegraded":"2024-04-26T22:10:11.927871+0000","last_fullsized":"2024-04-26T22:10:11.927871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3667,"log_dups_size":0,"ondisk_log_size":3667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":55047,"num_write_kb":687611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"40'6069","reported_seq":6128,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.810736+0000","last_change":"2024-04-26T21:59:57.649292+0000","last_active":"2024-04-26T22:10:05.810736+0000","last_peered":"2024-04-26T22:10:05.810736+0000","last_clean":"2024-04-26T22:10:05.810736+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:10:05.810736+0000","last_undegraded":"2024-04-26T22:10:05.810736+0000","last_fullsized":"2024-04-26T22:10:05.810736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:38.077939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":91606,"num_write_kb":1169870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"40'4173","reported_seq":4252,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.953675+0000","last_change":"2024-04-26T22:00:06.822000+0000","last_active":"2024-04-26T22:10:11.953675+0000","last_peered":"2024-04-26T22:10:11.953675+0000","last_clean":"2024-04-26T22:10:11.953675+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:10:11.953675+0000","last_undegraded":"2024-04-26T22:10:11.953675+0000","last_fullsized":"2024-04-26T22:10:11.953675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:32.716397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":62684,"num_write_kb":755589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"40'6512","reported_seq":6574,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.763471+0000","last_change":"2024-04-26T22:00:39.201110+0000","last_active":"2024-04-26T22:10:05.763471+0000","last_peered":"2024-04-26T22:10:05.763471+0000","last_clean":"2024-04-26T22:10:05.763471+0000","last_became_active":"2024-04-26T22:00:39.200691+0000","last_became_peered":"2024-04-26T22:00:39.200691+0000","last_unstale":"2024-04-26T22:10:05.763471+0000","last_undegraded":"2024-04-26T22:10:05.763471+0000","last_fullsized":"2024-04-26T22:10:05.763471+0000","mapping_epoch":25,"log_start":"29'700","ondisk_log_start":"29'700","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:06.181131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":98505,"num_write_kb":1266779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"40'5161","reported_seq":5247,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982946+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:10:11.982946+0000","last_peered":"2024-04-26T22:10:11.982946+0000","last_clean":"2024-04-26T22:10:11.982946+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:10:11.982946+0000","last_undegraded":"2024-04-26T22:10:11.982946+0000","last_fullsized":"2024-04-26T22:10:11.982946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:15.413960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":77820,"num_write_kb":957388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"40'5846","reported_seq":5903,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.706015+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:10:05.706015+0000","last_peered":"2024-04-26T22:10:05.706015+0000","last_clean":"2024-04-26T22:10:05.706015+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:10:05.706015+0000","last_undegraded":"2024-04-26T22:10:05.706015+0000","last_fullsized":"2024-04-26T22:10:05.706015+0000","mapping_epoch":18,"log_start":"29'600","ondisk_log_start":"29'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88593,"num_write_kb":1072922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"40'3484","reported_seq":3552,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972160+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:10:11.972160+0000","last_peered":"2024-04-26T22:10:11.972160+0000","last_clean":"2024-04-26T22:10:11.972160+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:10:11.972160+0000","last_undegraded":"2024-04-26T22:10:11.972160+0000","last_fullsized":"2024-04-26T22:10:11.972160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3484,"log_dups_size":0,"ondisk_log_size":3484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:10.922072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":52218,"num_write_kb":635931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"40'5213","reported_seq":5273,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874711+0000","last_change":"2024-04-26T21:59:57.656783+0000","last_active":"2024-04-26T22:10:05.874711+0000","last_peered":"2024-04-26T22:10:05.874711+0000","last_clean":"2024-04-26T22:10:05.874711+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:10:05.874711+0000","last_undegraded":"2024-04-26T22:10:05.874711+0000","last_fullsized":"2024-04-26T22:10:05.874711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":78898,"num_write_kb":987567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"40'3779","reported_seq":3859,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:16.022667+0000","last_change":"2024-04-26T22:00:06.820753+0000","last_active":"2024-04-26T22:10:16.022667+0000","last_peered":"2024-04-26T22:10:16.022667+0000","last_clean":"2024-04-26T22:10:16.022667+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:10:16.022667+0000","last_undegraded":"2024-04-26T22:10:16.022667+0000","last_fullsized":"2024-04-26T22:10:16.022667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56904,"num_write_kb":701117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"40'4635","reported_seq":4692,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.724357+0000","last_change":"2024-04-26T21:59:57.658410+0000","last_active":"2024-04-26T22:10:05.724357+0000","last_peered":"2024-04-26T22:10:05.724357+0000","last_clean":"2024-04-26T22:10:05.724357+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:10:05.724357+0000","last_undegraded":"2024-04-26T22:10:05.724357+0000","last_fullsized":"2024-04-26T22:10:05.724357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4635,"log_dups_size":0,"ondisk_log_size":4635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:04:53.933965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":70184,"num_write_kb":869081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"40'4125","reported_seq":4197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.954407+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:10:11.954407+0000","last_peered":"2024-04-26T22:10:11.954407+0000","last_clean":"2024-04-26T22:10:11.954407+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:10:11.954407+0000","last_undegraded":"2024-04-26T22:10:11.954407+0000","last_fullsized":"2024-04-26T22:10:11.954407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:35.739496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62115,"num_write_kb":743778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"40'6598","reported_seq":6661,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.878241+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:10:05.878241+0000","last_peered":"2024-04-26T22:10:05.878241+0000","last_clean":"2024-04-26T22:10:05.878241+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:10:05.878241+0000","last_undegraded":"2024-04-26T22:10:05.878241+0000","last_fullsized":"2024-04-26T22:10:05.878241+0000","mapping_epoch":18,"log_start":"29'800","ondisk_log_start":"29'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":99507,"num_write_kb":1174132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"40'3584","reported_seq":3653,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.979143+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:10:11.979143+0000","last_peered":"2024-04-26T22:10:11.979143+0000","last_clean":"2024-04-26T22:10:11.979143+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:10:11.979143+0000","last_undegraded":"2024-04-26T22:10:11.979143+0000","last_fullsized":"2024-04-26T22:10:11.979143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3584,"log_dups_size":0,"ondisk_log_size":3584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:26:09.191157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":54000,"num_write_kb":647349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"40'4854","reported_seq":4911,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.499989+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:10:05.499989+0000","last_peered":"2024-04-26T22:10:05.499989+0000","last_clean":"2024-04-26T22:10:05.499989+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:10:05.499989+0000","last_undegraded":"2024-04-26T22:10:05.499989+0000","last_fullsized":"2024-04-26T22:10:05.499989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:23.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73549,"num_write_kb":940613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"40'4744","reported_seq":4833,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.993205+0000","last_change":"2024-04-26T22:00:06.817634+0000","last_active":"2024-04-26T22:10:11.993205+0000","last_peered":"2024-04-26T22:10:11.993205+0000","last_clean":"2024-04-26T22:10:11.993205+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:10:11.993205+0000","last_undegraded":"2024-04-26T22:10:11.993205+0000","last_fullsized":"2024-04-26T22:10:11.993205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:28.326675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":71239,"num_write_kb":843376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"40'5453","reported_seq":5511,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.835411+0000","last_change":"2024-04-26T21:59:57.648409+0000","last_active":"2024-04-26T22:10:05.835411+0000","last_peered":"2024-04-26T22:10:05.835411+0000","last_clean":"2024-04-26T22:10:05.835411+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:10:05.835411+0000","last_undegraded":"2024-04-26T22:10:05.835411+0000","last_fullsized":"2024-04-26T22:10:05.835411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5453,"log_dups_size":0,"ondisk_log_size":5453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:34.407953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":82428,"num_write_kb":1036078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"40'4470","reported_seq":4548,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.844612+0000","last_change":"2024-04-26T22:00:06.815431+0000","last_active":"2024-04-26T22:10:11.844612+0000","last_peered":"2024-04-26T22:10:11.844612+0000","last_clean":"2024-04-26T22:10:11.844612+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:10:11.844612+0000","last_undegraded":"2024-04-26T22:10:11.844612+0000","last_fullsized":"2024-04-26T22:10:11.844612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:58.425361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":67360,"num_write_kb":793318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"40'6816","reported_seq":6876,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.874064+0000","last_change":"2024-04-26T21:59:57.657302+0000","last_active":"2024-04-26T22:10:05.874064+0000","last_peered":"2024-04-26T22:10:05.874064+0000","last_clean":"2024-04-26T22:10:05.874064+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:10:05.874064+0000","last_undegraded":"2024-04-26T22:10:05.874064+0000","last_fullsized":"2024-04-26T22:10:05.874064+0000","mapping_epoch":18,"log_start":"29'1100","ondisk_log_start":"29'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":103074,"num_write_kb":1294412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"40'3980","reported_seq":4053,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986399+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:10:11.986399+0000","last_peered":"2024-04-26T22:10:11.986399+0000","last_clean":"2024-04-26T22:10:11.986399+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:10:11.986399+0000","last_undegraded":"2024-04-26T22:10:11.986399+0000","last_fullsized":"2024-04-26T22:10:11.986399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:22.861709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":59871,"num_write_kb":728175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"40'4761","reported_seq":4821,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.861444+0000","last_change":"2024-04-26T21:59:57.653417+0000","last_active":"2024-04-26T22:10:05.861444+0000","last_peered":"2024-04-26T22:10:05.861444+0000","last_clean":"2024-04-26T22:10:05.861444+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:10:05.861444+0000","last_undegraded":"2024-04-26T22:10:05.861444+0000","last_fullsized":"2024-04-26T22:10:05.861444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:42.184751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":72121,"num_write_kb":895283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1800,"num_read_kb":0,"num_write":3978586,"num_write_kb":48381196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264479,"ondisk_log_size":264479,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":109332168,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":691,"num_read_kb":381190,"num_write":3544,"num_write_kb":109340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218726400,"data_stored":218664336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3493,"ondisk_log_size":3493,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":363,"num_write":5266740,"num_write_kb":65270508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338431,"ondisk_log_size":338431,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":143059102,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":553,"num_read_kb":447893,"num_write":4271,"num_write_kb":143037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286179328,"data_stored":286118204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4237,"ondisk_log_size":4237,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525528,"kb_used_data":21200,"kb_used_omap":1,"kb_used_meta":504318,"kb_avail":93846312,"statfs":{"total":96636764160,"available":96098623488,"internally_reserved":0,"allocated":21708800,"data_stored":21470005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516422085},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574974,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848624,"kb_used_data":66920,"kb_used_omap":1,"kb_used_meta":781694,"kb_avail":93523216,"statfs":{"total":96636764160,"available":95767773184,"internally_reserved":0,"allocated":68526080,"data_stored":68282539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":800455114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574352,"kb_used_data":74184,"kb_used_omap":1,"kb_used_meta":500158,"kb_avail":93797488,"statfs":{"total":96636764160,"available":96048627712,"internally_reserved":0,"allocated":75964416,"data_stored":75718280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":512162246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574974,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789860,"kb_used_data":45596,"kb_used_omap":1,"kb_used_meta":744254,"kb_avail":93581980,"statfs":{"total":96636764160,"available":95827947520,"internally_reserved":0,"allocated":46690304,"data_stored":46445706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":762116549},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882668,"kb_used_data":82468,"kb_used_omap":1,"kb_used_meta":800190,"kb_avail":93489172,"statfs":{"total":96636764160,"available":95732912128,"internally_reserved":0,"allocated":84447232,"data_stored":84193910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":819395014},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900648,"kb_used_data":78560,"kb_used_omap":1,"kb_used_meta":822078,"kb_avail":93471192,"statfs":{"total":96636764160,"available":95714500608,"internally_reserved":0,"allocated":80445440,"data_stored":80191557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":841808327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767416,"kb_used_data":86832,"kb_used_omap":1,"kb_used_meta":680574,"kb_avail":93604424,"statfs":{"total":96636764160,"available":95850930176,"internally_reserved":0,"allocated":88915968,"data_stored":88663873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":696908229},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740368,"kb_used_data":41992,"kb_used_omap":1,"kb_used_meta":698366,"kb_avail":93631472,"statfs":{"total":96636764160,"available":95878627328,"internally_reserved":0,"allocated":42999808,"data_stored":42753566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715127237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29372416,"data_stored":29360766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":38112011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12673024,"data_stored":12671093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33652736,"data_stored":33643285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29364224,"data_stored":29360218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":46145536,"data_stored":46138016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50700288,"data_stored":50695345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8585216,"data_stored":8579392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46342144,"data_stored":46328752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37847040,"data_stored":37836920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16875520,"data_stored":16865968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:10:23.299 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.300 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.301 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.302 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.303 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.304 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.305 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.306 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T21:59:45.086527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.307 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.308 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.309 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.310 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.311 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.312 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.313 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-26T22:00:04.784649+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=4, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.314 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.315 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.316 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.317 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.318 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.319 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:23.320 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T22:10:31.666 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:10:31.670 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:10:43.322 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:10:43.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.431+0000 7f93f50d1640 1 Processor -- start 2024-04-26T22:10:43.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.431+0000 7f93f50d1640 1 -- start start 2024-04-26T22:10:43.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93f50d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93f00ab3f0 0x7f93f00c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93f50d1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f93f00c52c0 0x7f93f01574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93f50d1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93f00c3810 con 0x7f93f01570d0 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93f50d1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f93f00c4330 con 0x7f93f00c52c0 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93f50d1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f93f00c3ab0 con 0x7f93f00ab3f0 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f93f00c52c0 0x7f93f01574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f93f00c52c0 0x7f93f01574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42928/0 (socket says 172.21.15.135:42928) 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 -- 172.21.15.135:0/1940581788 learned_addr learned my addr 172.21.15.135:0/1940581788 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 -- 172.21.15.135:0/1940581788 >> v2:172.21.15.139:3300/0 conn(0x7f93f00ab3f0 msgr2=0x7f93f00c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 --2- 172.21.15.135:0/1940581788 >> v2:172.21.15.139:3300/0 conn(0x7f93f00ab3f0 0x7f93f00c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 -- 172.21.15.135:0/1940581788 >> v1:172.21.15.139:6789/0 conn(0x7f93f01570d0 legacy=0x7f93f0150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.432+0000 7f93ef59e640 1 -- 172.21.15.135:0/1940581788 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f00c4580 con 0x7f93f00c52c0 2024-04-26T22:10:43.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.433+0000 7f93ef59e640 1 --2- 172.21.15.135:0/1940581788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f01574b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f93f00c44b0 tx=0x7f93e802c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df980deff1194d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.433+0000 7f93edd9b640 1 -- 172.21.15.135:0/1940581788 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8009600 con 0x7f93f00c52c0 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.433+0000 7f93edd9b640 1 -- 172.21.15.135:0/1940581788 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e80097a0 con 0x7f93f00c52c0 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.433+0000 7f93edd9b640 1 -- 172.21.15.135:0/1940581788 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8009b70 con 0x7f93f00c52c0 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 -- 172.21.15.135:0/1940581788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 msgr2=0x7f93f01574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 --2- 172.21.15.135:0/1940581788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f01574b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f93f00c44b0 tx=0x7f93e802c7c0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 -- 172.21.15.135:0/1940581788 shutdown_connections 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 --2- 172.21.15.135:0/1940581788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f01574b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 --2- 172.21.15.135:0/1940581788 >> v2:172.21.15.139:3300/0 conn(0x7f93f00ab3f0 0x7f93f00c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 -- 172.21.15.135:0/1940581788 >> 172.21.15.135:0/1940581788 conn(0x7f93f006c310 msgr2=0x7f93f006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 -- 172.21.15.135:0/1940581788 shutdown_connections 2024-04-26T22:10:43.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.434+0000 7f93f50d1640 1 -- 172.21.15.135:0/1940581788 wait complete. 2024-04-26T22:10:43.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.435+0000 7f93f50d1640 1 Processor -- start 2024-04-26T22:10:43.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.435+0000 7f93f50d1640 1 -- start start 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93f50d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f00ab3f0 0x7f93f01eba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93f50d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f0145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93f50d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93f01570d0 0x7f93f00c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93f50d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93f01512e0 con 0x7f93f00c52c0 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93f50d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f93f0151160 con 0x7f93f00ab3f0 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93f50d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f93f00aba60 con 0x7f93f01570d0 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f0145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f0145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42934/0 (socket says 172.21.15.135:42934) 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 -- 172.21.15.135:0/3460670855 learned_addr learned my addr 172.21.15.135:0/3460670855 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ef59e640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93f01570d0 0x7f93f00c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93eed9d640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f00ab3f0 0x7f93f01eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:43.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 -- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93f01570d0 msgr2=0x7f93f00c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93f01570d0 0x7f93f00c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 -- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f00ab3f0 msgr2=0x7f93f01eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f00ab3f0 0x7f93f01eba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 -- 172.21.15.135:0/3460670855 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f0155ee0 con 0x7f93f00c52c0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.436+0000 7f93ee59c640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f0145cc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b4d0 tx=0x7f93e400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.437+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4014070 con 0x7f93f00c52c0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.437+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e4010870 con 0x7f93f00c52c0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.437+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f0156110 con 0x7f93f00c52c0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.437+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4010c60 con 0x7f93f00c52c0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.437+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f01566d0 con 0x7f93f00c52c0 2024-04-26T22:10:43.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.438+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f00507a0 con 0x7f93f00c52c0 2024-04-26T22:10:43.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.438+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93e4017070 con 0x7f93f00c52c0 2024-04-26T22:10:43.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.438+0000 7f93d77fe640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f93dc06c8f0 0x7f93dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:10:43.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.439+0000 7f93eed9d640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f93dc06c8f0 0x7f93dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:10:43.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.439+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f93e408c080 con 0x7f93f00c52c0 2024-04-26T22:10:43.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.439+0000 7f93eed9d640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f93dc06c8f0 0x7f93dc06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:10:43.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.443+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93e4059680 con 0x7f93f00c52c0 2024-04-26T22:10:43.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.617+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93f00c4110 con 0x7f93dc06c8f0 2024-04-26T22:10:43.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.628+0000 7f93d77fe640 1 -- 172.21.15.135:0/3460670855 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494015 (secure 0 0 0) 0x7f93f00c4110 con 0x7f93dc06c8f0 2024-04-26T22:10:43.633 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:10:43.637 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f93dc06c8f0 msgr2=0x7f93dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f93dc06c8f0 0x7f93dc06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0009060 comp rx=0 tx=0).stop 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 msgr2=0x7f93f0145cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f0145cc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b4d0 tx=0x7f93e400b990 comp rx=0 tx=0).stop 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 shutdown_connections 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f93dc06c8f0 0x7f93dc06edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93f01570d0 0x7f93f00c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93f00c52c0 0x7f93f0145cc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 --2- 172.21.15.135:0/3460670855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f00ab3f0 0x7f93f01eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:10:43.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 >> 172.21.15.135:0/3460670855 conn(0x7f93f006c310 msgr2=0x7f93f00c2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:10:43.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 shutdown_connections 2024-04-26T22:10:43.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:10:43.632+0000 7f93f50d1640 1 -- 172.21.15.135:0/3460670855 wait complete. 2024-04-26T22:10:43.643 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-26T22:10:43.316522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252981638,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":3350,"num_read_kb":829546,"num_write":9253302,"num_write_kb":113906497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610852,"ondisk_log_size":610852,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6037784,"kb_used_data":497752,"kb_used_omap":12,"kb_used_meta":5539955,"kb_avail":748936936,"statfs":{"total":773094113280,"available":766911422464,"internally_reserved":0,"allocated":509698048,"data_stored":507719436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":5672914481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003917"},"pg_stats":[{"pgid":"5.18","version":"40'4026","reported_seq":4109,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.893457+0000","last_change":"2024-04-26T22:10:36.893482+0000","last_active":"2024-04-26T22:10:36.893457+0000","last_peered":"2024-04-26T22:10:36.893457+0000","last_clean":"2024-04-26T22:10:36.893457+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:10:36.893457+0000","last_undegraded":"2024-04-26T22:10:36.893457+0000","last_fullsized":"2024-04-26T22:10:36.893457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4026","last_scrub_stamp":"2024-04-26T22:10:36.893331+0000","last_deep_scrub":"40'4026","last_deep_scrub_stamp":"2024-04-26T22:10:36.893331+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.893331+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:12.710655+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":60598,"num_write_kb":740487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"40'30","reported_seq":113,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.156080+0000","last_change":"2024-04-26T22:10:26.156123+0000","last_active":"2024-04-26T22:10:26.156080+0000","last_peered":"2024-04-26T22:10:26.156080+0000","last_clean":"2024-04-26T22:10:26.156080+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:10:26.156080+0000","last_undegraded":"2024-04-26T22:10:26.156080+0000","last_fullsized":"2024-04-26T22:10:26.156080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30","last_scrub_stamp":"2024-04-26T22:10:26.155966+0000","last_deep_scrub":"40'30","last_deep_scrub_stamp":"2024-04-26T22:10:26.155966+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.155966+0000","objects_scrubbed":3,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:00.107945+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":16408,"num_write":20,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'5701","reported_seq":5759,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:15.358681+0000","last_change":"2024-04-26T21:59:57.653056+0000","last_active":"2024-04-26T22:10:15.358681+0000","last_peered":"2024-04-26T22:10:15.358681+0000","last_clean":"2024-04-26T22:10:15.358681+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:10:15.358681+0000","last_undegraded":"2024-04-26T22:10:15.358681+0000","last_fullsized":"2024-04-26T22:10:15.358681+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86528,"num_write_kb":1106988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"40'21","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.937217+0000","last_change":"2024-04-26T22:10:23.937241+0000","last_active":"2024-04-26T22:10:23.937217+0000","last_peered":"2024-04-26T22:10:23.937217+0000","last_clean":"2024-04-26T22:10:23.937217+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:10:23.937217+0000","last_undegraded":"2024-04-26T22:10:23.937217+0000","last_fullsized":"2024-04-26T22:10:23.937217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:23.937145+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:23.937145+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.937145+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:20.231325+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190714,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"40'4869","reported_seq":4958,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988315+0000","last_change":"2024-04-26T22:00:06.825419+0000","last_active":"2024-04-26T22:10:11.988315+0000","last_peered":"2024-04-26T22:10:11.988315+0000","last_clean":"2024-04-26T22:10:11.988315+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:10:11.988315+0000","last_undegraded":"2024-04-26T22:10:11.988315+0000","last_fullsized":"2024-04-26T22:10:11.988315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":73297,"num_write_kb":899566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"40'21","reported_seq":93,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.023138+0000","last_change":"2024-04-26T22:10:39.023173+0000","last_active":"2024-04-26T22:10:39.023138+0000","last_peered":"2024-04-26T22:10:39.023138+0000","last_clean":"2024-04-26T22:10:39.023138+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:10:39.023138+0000","last_undegraded":"2024-04-26T22:10:39.023138+0000","last_fullsized":"2024-04-26T22:10:39.023138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:39.023030+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:39.023030+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.023030+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:02.899183+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"40'6511","reported_seq":6583,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.137267+0000","last_change":"2024-04-26T22:10:24.137290+0000","last_active":"2024-04-26T22:10:24.137267+0000","last_peered":"2024-04-26T22:10:24.137267+0000","last_clean":"2024-04-26T22:10:24.137267+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:10:24.137267+0000","last_undegraded":"2024-04-26T22:10:24.137267+0000","last_fullsized":"2024-04-26T22:10:24.137267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6511","last_scrub_stamp":"2024-04-26T22:10:24.137190+0000","last_deep_scrub":"40'6511","last_deep_scrub_stamp":"2024-04-26T22:10:24.137190+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.137190+0000","objects_scrubbed":0,"log_size":6511,"log_dups_size":0,"ondisk_log_size":6511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:54.188579+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":98412,"num_write_kb":1227645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"40'134","reported_seq":199,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.831985+0000","last_change":"2024-04-26T22:10:21.832016+0000","last_active":"2024-04-26T22:10:21.831985+0000","last_peered":"2024-04-26T22:10:21.831985+0000","last_clean":"2024-04-26T22:10:21.831985+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:10:21.831985+0000","last_undegraded":"2024-04-26T22:10:21.831985+0000","last_fullsized":"2024-04-26T22:10:21.831985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'134","last_scrub_stamp":"2024-04-26T22:10:21.831897+0000","last_deep_scrub":"40'134","last_deep_scrub_stamp":"2024-04-26T22:10:21.831897+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.831897+0000","objects_scrubbed":2,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:34.924587+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":127,"num_write_kb":8276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"40'3508","reported_seq":3575,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.970947+0000","last_change":"2024-04-26T22:00:06.820463+0000","last_active":"2024-04-26T22:10:11.970947+0000","last_peered":"2024-04-26T22:10:11.970947+0000","last_clean":"2024-04-26T22:10:11.970947+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:10:11.970947+0000","last_undegraded":"2024-04-26T22:10:11.970947+0000","last_fullsized":"2024-04-26T22:10:11.970947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3508,"log_dups_size":0,"ondisk_log_size":3508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":52619,"num_write_kb":632470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"40'120","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.213875+0000","last_change":"2024-04-26T22:10:37.213912+0000","last_active":"2024-04-26T22:10:37.213875+0000","last_peered":"2024-04-26T22:10:37.213875+0000","last_clean":"2024-04-26T22:10:37.213875+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:10:37.213875+0000","last_undegraded":"2024-04-26T22:10:37.213875+0000","last_fullsized":"2024-04-26T22:10:37.213875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'120","last_scrub_stamp":"2024-04-26T22:10:37.213785+0000","last_deep_scrub":"40'120","last_deep_scrub_stamp":"2024-04-26T22:10:37.213785+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.213785+0000","objects_scrubbed":4,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:26.515957+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":45092,"num_write":102,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"40'3918","reported_seq":3984,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.179306+0000","last_change":"2024-04-26T22:10:39.179334+0000","last_active":"2024-04-26T22:10:39.179306+0000","last_peered":"2024-04-26T22:10:39.179306+0000","last_clean":"2024-04-26T22:10:39.179306+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:10:39.179306+0000","last_undegraded":"2024-04-26T22:10:39.179306+0000","last_fullsized":"2024-04-26T22:10:39.179306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3918","last_scrub_stamp":"2024-04-26T22:10:39.179199+0000","last_deep_scrub":"40'3918","last_deep_scrub_stamp":"2024-04-26T22:10:39.179199+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.179199+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:07.949612+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":59308,"num_write_kb":714963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"40'1269","reported_seq":1335,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.794568+0000","last_change":"2024-04-26T22:10:23.794610+0000","last_active":"2024-04-26T22:10:23.794568+0000","last_peered":"2024-04-26T22:10:23.794568+0000","last_clean":"2024-04-26T22:10:23.794568+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:10:23.794568+0000","last_undegraded":"2024-04-26T22:10:23.794568+0000","last_fullsized":"2024-04-26T22:10:23.794568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'1269","last_scrub_stamp":"2024-04-26T22:10:23.794478+0000","last_deep_scrub":"40'1269","last_deep_scrub_stamp":"2024-04-26T22:10:23.794478+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.794478+0000","objects_scrubbed":1,"log_size":1269,"log_dups_size":0,"ondisk_log_size":1269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:08.939312+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":1265,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'3315","reported_seq":3385,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.989133+0000","last_change":"2024-04-26T22:00:06.828057+0000","last_active":"2024-04-26T22:10:11.989133+0000","last_peered":"2024-04-26T22:10:11.989133+0000","last_clean":"2024-04-26T22:10:11.989133+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:10:11.989133+0000","last_undegraded":"2024-04-26T22:10:11.989133+0000","last_fullsized":"2024-04-26T22:10:11.989133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3315,"log_dups_size":0,"ondisk_log_size":3315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":49749,"num_write_kb":591259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"40'20","reported_seq":93,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.854354+0000","last_change":"2024-04-26T21:59:56.650651+0000","last_active":"2024-04-26T22:10:05.854354+0000","last_peered":"2024-04-26T22:10:05.854354+0000","last_clean":"2024-04-26T22:10:05.854354+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:10:05.854354+0000","last_undegraded":"2024-04-26T22:10:05.854354+0000","last_fullsized":"2024-04-26T22:10:05.854354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":18,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"40'6620","reported_seq":6673,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.398950+0000","last_change":"2024-04-26T22:10:38.398992+0000","last_active":"2024-04-26T22:10:38.398950+0000","last_peered":"2024-04-26T22:10:38.398950+0000","last_clean":"2024-04-26T22:10:38.398950+0000","last_became_active":"2024-04-26T22:00:39.151025+0000","last_became_peered":"2024-04-26T22:00:39.151025+0000","last_unstale":"2024-04-26T22:10:38.398950+0000","last_undegraded":"2024-04-26T22:10:38.398950+0000","last_fullsized":"2024-04-26T22:10:38.398950+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6620","last_scrub_stamp":"2024-04-26T22:10:38.398847+0000","last_deep_scrub":"40'6620","last_deep_scrub_stamp":"2024-04-26T22:10:38.398847+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.398847+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:48.805412+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":100009,"num_write_kb":1224131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"40'43","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.765919+0000","last_change":"2024-04-26T22:10:27.766038+0000","last_active":"2024-04-26T22:10:27.765919+0000","last_peered":"2024-04-26T22:10:27.765919+0000","last_clean":"2024-04-26T22:10:27.765919+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:10:27.765919+0000","last_undegraded":"2024-04-26T22:10:27.765919+0000","last_fullsized":"2024-04-26T22:10:27.765919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'43","last_scrub_stamp":"2024-04-26T22:10:27.765838+0000","last_deep_scrub":"40'43","last_deep_scrub_stamp":"2024-04-26T22:10:27.765838+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.765838+0000","objects_scrubbed":2,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:18.805686+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":30,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"40'4367","reported_seq":4456,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.389651+0000","last_change":"2024-04-26T22:10:37.389677+0000","last_active":"2024-04-26T22:10:37.389651+0000","last_peered":"2024-04-26T22:10:37.389651+0000","last_clean":"2024-04-26T22:10:37.389651+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:10:37.389651+0000","last_undegraded":"2024-04-26T22:10:37.389651+0000","last_fullsized":"2024-04-26T22:10:37.389651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4367","last_scrub_stamp":"2024-04-26T22:10:37.389553+0000","last_deep_scrub":"40'4367","last_deep_scrub_stamp":"2024-04-26T22:10:37.389553+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.389553+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:09.719845+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65356,"num_write_kb":790948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"40'4740","reported_seq":4806,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.342201+0000","last_change":"2024-04-26T22:10:33.342228+0000","last_active":"2024-04-26T22:10:33.342201+0000","last_peered":"2024-04-26T22:10:33.342201+0000","last_clean":"2024-04-26T22:10:33.342201+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:10:33.342201+0000","last_undegraded":"2024-04-26T22:10:33.342201+0000","last_fullsized":"2024-04-26T22:10:33.342201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4740","last_scrub_stamp":"2024-04-26T22:10:33.342090+0000","last_deep_scrub":"40'4740","last_deep_scrub_stamp":"2024-04-26T22:10:33.342090+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.342090+0000","objects_scrubbed":0,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:59.003154+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":71515,"num_write_kb":926848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"40'209","reported_seq":292,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.183050+0000","last_change":"2024-04-26T22:10:36.183083+0000","last_active":"2024-04-26T22:10:36.183050+0000","last_peered":"2024-04-26T22:10:36.183050+0000","last_clean":"2024-04-26T22:10:36.183050+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:10:36.183050+0000","last_undegraded":"2024-04-26T22:10:36.183050+0000","last_fullsized":"2024-04-26T22:10:36.183050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'209","last_scrub_stamp":"2024-04-26T22:10:36.182952+0000","last_deep_scrub":"40'209","last_deep_scrub_stamp":"2024-04-26T22:10:36.182952+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.182952+0000","objects_scrubbed":3,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:39.012653+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":29169,"num_write":192,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"40'41","reported_seq":114,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.883143+0000","last_change":"2024-04-26T22:10:35.883186+0000","last_active":"2024-04-26T22:10:35.883143+0000","last_peered":"2024-04-26T22:10:35.883143+0000","last_clean":"2024-04-26T22:10:35.883143+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:10:35.883143+0000","last_undegraded":"2024-04-26T22:10:35.883143+0000","last_fullsized":"2024-04-26T22:10:35.883143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'41","last_scrub_stamp":"2024-04-26T22:10:35.883051+0000","last_deep_scrub":"40'41","last_deep_scrub_stamp":"2024-04-26T22:10:35.883051+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.883051+0000","objects_scrubbed":2,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:09.957036+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":13991,"num_write":36,"num_write_kb":4124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'3984","reported_seq":4070,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.486129+0000","last_change":"2024-04-26T22:10:22.486153+0000","last_active":"2024-04-26T22:10:22.486129+0000","last_peered":"2024-04-26T22:10:22.486129+0000","last_clean":"2024-04-26T22:10:22.486129+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:10:22.486129+0000","last_undegraded":"2024-04-26T22:10:22.486129+0000","last_fullsized":"2024-04-26T22:10:22.486129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3984","last_scrub_stamp":"2024-04-26T22:10:22.486045+0000","last_deep_scrub":"40'3984","last_deep_scrub_stamp":"2024-04-26T22:10:22.486045+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.486045+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:18.725892+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59797,"num_write_kb":746756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"40'4602","reported_seq":4670,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.012894+0000","last_change":"2024-04-26T22:10:28.012925+0000","last_active":"2024-04-26T22:10:28.012894+0000","last_peered":"2024-04-26T22:10:28.012894+0000","last_clean":"2024-04-26T22:10:28.012894+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:10:28.012894+0000","last_undegraded":"2024-04-26T22:10:28.012894+0000","last_fullsized":"2024-04-26T22:10:28.012894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4602","last_scrub_stamp":"2024-04-26T22:10:28.012760+0000","last_deep_scrub":"40'4602","last_deep_scrub_stamp":"2024-04-26T22:10:28.012760+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.012760+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:08.470714+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69468,"num_write_kb":843448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"40'25","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.309293+0000","last_change":"2024-04-26T22:10:33.309321+0000","last_active":"2024-04-26T22:10:33.309293+0000","last_peered":"2024-04-26T22:10:33.309293+0000","last_clean":"2024-04-26T22:10:33.309293+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:10:33.309293+0000","last_undegraded":"2024-04-26T22:10:33.309293+0000","last_fullsized":"2024-04-26T22:10:33.309293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'25","last_scrub_stamp":"2024-04-26T22:10:33.309216+0000","last_deep_scrub":"40'25","last_deep_scrub_stamp":"2024-04-26T22:10:33.309216+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.309216+0000","objects_scrubbed":1,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:45.314332+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":20,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"40'3","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.792564+0000","last_change":"2024-04-26T22:10:25.792588+0000","last_active":"2024-04-26T22:10:25.792564+0000","last_peered":"2024-04-26T22:10:25.792564+0000","last_clean":"2024-04-26T22:10:25.792564+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:10:25.792564+0000","last_undegraded":"2024-04-26T22:10:25.792564+0000","last_fullsized":"2024-04-26T22:10:25.792564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:25.792472+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:25.792472+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.792472+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:00.793620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"40'3976","reported_seq":4054,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.941365+0000","last_change":"2024-04-26T22:00:06.825150+0000","last_active":"2024-04-26T22:10:11.941365+0000","last_peered":"2024-04-26T22:10:11.941365+0000","last_clean":"2024-04-26T22:10:11.941365+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:10:11.941365+0000","last_undegraded":"2024-04-26T22:10:11.941365+0000","last_fullsized":"2024-04-26T22:10:11.941365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3976,"log_dups_size":0,"ondisk_log_size":3976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59917,"num_write_kb":706723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"40'5423","reported_seq":5481,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.783413+0000","last_change":"2024-04-26T21:59:57.648296+0000","last_active":"2024-04-26T22:10:05.783413+0000","last_peered":"2024-04-26T22:10:05.783413+0000","last_clean":"2024-04-26T22:10:05.783413+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:10:05.783413+0000","last_undegraded":"2024-04-26T22:10:05.783413+0000","last_fullsized":"2024-04-26T22:10:05.783413+0000","mapping_epoch":18,"log_start":"28'200","ondisk_log_start":"28'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":81931,"num_write_kb":1035269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'44","reported_seq":113,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.369198+0000","last_change":"2024-04-26T22:10:30.369244+0000","last_active":"2024-04-26T22:10:30.369198+0000","last_peered":"2024-04-26T22:10:30.369198+0000","last_clean":"2024-04-26T22:10:30.369198+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:10:30.369198+0000","last_undegraded":"2024-04-26T22:10:30.369198+0000","last_fullsized":"2024-04-26T22:10:30.369198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'44","last_scrub_stamp":"2024-04-26T22:10:30.368976+0000","last_deep_scrub":"40'44","last_deep_scrub_stamp":"2024-04-26T22:10:30.368976+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.368976+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:03.853474+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363115,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":324,"num_write":38,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"40'87","reported_seq":157,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.446504+0000","last_change":"2024-04-26T22:10:27.446567+0000","last_active":"2024-04-26T22:10:27.446504+0000","last_peered":"2024-04-26T22:10:27.446504+0000","last_clean":"2024-04-26T22:10:27.446504+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:10:27.446504+0000","last_undegraded":"2024-04-26T22:10:27.446504+0000","last_fullsized":"2024-04-26T22:10:27.446504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'87","last_scrub_stamp":"2024-04-26T22:10:27.446409+0000","last_deep_scrub":"40'87","last_deep_scrub_stamp":"2024-04-26T22:10:27.446409+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.446409+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:53.107473+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":6841,"num_write":81,"num_write_kb":4168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"40'4751","reported_seq":4838,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.249776+0000","last_change":"2024-04-26T22:10:31.249803+0000","last_active":"2024-04-26T22:10:31.249776+0000","last_peered":"2024-04-26T22:10:31.249776+0000","last_clean":"2024-04-26T22:10:31.249776+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:10:31.249776+0000","last_undegraded":"2024-04-26T22:10:31.249776+0000","last_fullsized":"2024-04-26T22:10:31.249776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4751","last_scrub_stamp":"2024-04-26T22:10:31.249660+0000","last_deep_scrub":"40'4751","last_deep_scrub_stamp":"2024-04-26T22:10:31.249660+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.249660+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:21.922150+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":71494,"num_write_kb":858502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"40'5174","reported_seq":5242,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.837084+0000","last_change":"2024-04-26T22:10:32.837126+0000","last_active":"2024-04-26T22:10:32.837084+0000","last_peered":"2024-04-26T22:10:32.837084+0000","last_clean":"2024-04-26T22:10:32.837084+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:10:32.837084+0000","last_undegraded":"2024-04-26T22:10:32.837084+0000","last_fullsized":"2024-04-26T22:10:32.837084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5174","last_scrub_stamp":"2024-04-26T22:10:32.836923+0000","last_deep_scrub":"40'5174","last_deep_scrub_stamp":"2024-04-26T22:10:32.836923+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.836923+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:42.665499+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":78246,"num_write_kb":941647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"40'8","reported_seq":72,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.842541+0000","last_change":"2024-04-26T21:59:56.649872+0000","last_active":"2024-04-26T22:10:05.842541+0000","last_peered":"2024-04-26T22:10:05.842541+0000","last_clean":"2024-04-26T22:10:05.842541+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:10:05.842541+0000","last_undegraded":"2024-04-26T22:10:05.842541+0000","last_fullsized":"2024-04-26T22:10:05.842541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:55.618144+0000","last_clean_scrub_stamp":"2024-04-26T21:59:55.618144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":4,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"40'39","reported_seq":116,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.808525+0000","last_change":"2024-04-26T22:10:22.808582+0000","last_active":"2024-04-26T22:10:22.808525+0000","last_peered":"2024-04-26T22:10:22.808525+0000","last_clean":"2024-04-26T22:10:22.808525+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:10:22.808525+0000","last_undegraded":"2024-04-26T22:10:22.808525+0000","last_fullsized":"2024-04-26T22:10:22.808525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'39","last_scrub_stamp":"2024-04-26T22:10:22.808431+0000","last_deep_scrub":"40'39","last_deep_scrub_stamp":"2024-04-26T22:10:22.808431+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.808431+0000","objects_scrubbed":2,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:44.877319+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":31,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"40'3913","reported_seq":4008,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.020278+0000","last_change":"2024-04-26T22:10:35.020307+0000","last_active":"2024-04-26T22:10:35.020278+0000","last_peered":"2024-04-26T22:10:35.020278+0000","last_clean":"2024-04-26T22:10:35.020278+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:10:35.020278+0000","last_undegraded":"2024-04-26T22:10:35.020278+0000","last_fullsized":"2024-04-26T22:10:35.020278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3913","last_scrub_stamp":"2024-04-26T22:10:35.020128+0000","last_deep_scrub":"40'3913","last_deep_scrub_stamp":"2024-04-26T22:10:35.020128+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.020128+0000","objects_scrubbed":0,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:37.849829+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":58791,"num_write_kb":740377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'4599","reported_seq":4666,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.164092+0000","last_change":"2024-04-26T22:10:29.164144+0000","last_active":"2024-04-26T22:10:29.164092+0000","last_peered":"2024-04-26T22:10:29.164092+0000","last_clean":"2024-04-26T22:10:29.164092+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:10:29.164092+0000","last_undegraded":"2024-04-26T22:10:29.164092+0000","last_fullsized":"2024-04-26T22:10:29.164092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4599","last_scrub_stamp":"2024-04-26T22:10:29.163964+0000","last_deep_scrub":"40'4599","last_deep_scrub_stamp":"2024-04-26T22:10:29.163964+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.163964+0000","objects_scrubbed":0,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:04.165113+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69140,"num_write_kb":847801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"40'9","reported_seq":88,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.364604+0000","last_change":"2024-04-26T22:10:30.364630+0000","last_active":"2024-04-26T22:10:30.364604+0000","last_peered":"2024-04-26T22:10:30.364604+0000","last_clean":"2024-04-26T22:10:30.364604+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:10:30.364604+0000","last_undegraded":"2024-04-26T22:10:30.364604+0000","last_fullsized":"2024-04-26T22:10:30.364604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'9","last_scrub_stamp":"2024-04-26T22:10:30.364522+0000","last_deep_scrub":"40'9","last_deep_scrub_stamp":"2024-04-26T22:10:30.364522+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.364522+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:57.094328+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"40'23","reported_seq":110,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.810252+0000","last_change":"2024-04-26T22:10:34.810290+0000","last_active":"2024-04-26T22:10:34.810252+0000","last_peered":"2024-04-26T22:10:34.810252+0000","last_clean":"2024-04-26T22:10:34.810252+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:10:34.810252+0000","last_undegraded":"2024-04-26T22:10:34.810252+0000","last_fullsized":"2024-04-26T22:10:34.810252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'23","last_scrub_stamp":"2024-04-26T22:10:34.810145+0000","last_deep_scrub":"40'23","last_deep_scrub_stamp":"2024-04-26T22:10:34.810145+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.810145+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:57.149020+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":98,"num_write":20,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"40'3929","reported_seq":4000,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.651621+0000","last_change":"2024-04-26T22:00:06.819864+0000","last_active":"2024-04-26T22:10:21.651621+0000","last_peered":"2024-04-26T22:10:21.651621+0000","last_clean":"2024-04-26T22:10:21.651621+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:10:21.651621+0000","last_undegraded":"2024-04-26T22:10:21.651621+0000","last_fullsized":"2024-04-26T22:10:21.651621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":59023,"num_write_kb":718389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'5311","reported_seq":5377,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.992181+0000","last_change":"2024-04-26T22:10:33.992216+0000","last_active":"2024-04-26T22:10:33.992181+0000","last_peered":"2024-04-26T22:10:33.992181+0000","last_clean":"2024-04-26T22:10:33.992181+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:10:33.992181+0000","last_undegraded":"2024-04-26T22:10:33.992181+0000","last_fullsized":"2024-04-26T22:10:33.992181+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5311","last_scrub_stamp":"2024-04-26T22:10:33.992048+0000","last_deep_scrub":"40'5311","last_deep_scrub_stamp":"2024-04-26T22:10:33.992048+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.992048+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:52.407619+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80379,"num_write_kb":1004221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'218","reported_seq":291,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.350013+0000","last_change":"2024-04-26T22:10:32.350054+0000","last_active":"2024-04-26T22:10:32.350013+0000","last_peered":"2024-04-26T22:10:32.350013+0000","last_clean":"2024-04-26T22:10:32.350013+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:10:32.350013+0000","last_undegraded":"2024-04-26T22:10:32.350013+0000","last_fullsized":"2024-04-26T22:10:32.350013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'218","last_scrub_stamp":"2024-04-26T22:10:32.349915+0000","last_deep_scrub":"40'218","last_deep_scrub_stamp":"2024-04-26T22:10:32.349915+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.349915+0000","objects_scrubbed":1,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:12.807870+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":61,"num_write":416,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"40'24","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.950133+0000","last_change":"2024-04-26T22:10:22.950185+0000","last_active":"2024-04-26T22:10:22.950133+0000","last_peered":"2024-04-26T22:10:22.950133+0000","last_clean":"2024-04-26T22:10:22.950133+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:10:22.950133+0000","last_undegraded":"2024-04-26T22:10:22.950133+0000","last_fullsized":"2024-04-26T22:10:22.950133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'24","last_scrub_stamp":"2024-04-26T22:10:22.950054+0000","last_deep_scrub":"40'24","last_deep_scrub_stamp":"2024-04-26T22:10:22.950054+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.950054+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:48.611118+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":15,"num_write_kb":8199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"40'4334","reported_seq":4419,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.373617+0000","last_change":"2024-04-26T22:10:38.373663+0000","last_active":"2024-04-26T22:10:38.373617+0000","last_peered":"2024-04-26T22:10:38.373617+0000","last_clean":"2024-04-26T22:10:38.373617+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:10:38.373617+0000","last_undegraded":"2024-04-26T22:10:38.373617+0000","last_fullsized":"2024-04-26T22:10:38.373617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4334","last_scrub_stamp":"2024-04-26T22:10:38.373498+0000","last_deep_scrub":"40'4334","last_deep_scrub_stamp":"2024-04-26T22:10:38.373498+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.373498+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:37.285923+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":65365,"num_write_kb":782662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"40'5496","reported_seq":5566,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.426140+0000","last_change":"2024-04-26T22:10:24.426140+0000","last_active":"2024-04-26T22:10:24.426140+0000","last_peered":"2024-04-26T22:10:24.426140+0000","last_clean":"2024-04-26T22:10:24.426140+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:10:24.426140+0000","last_undegraded":"2024-04-26T22:10:24.426140+0000","last_fullsized":"2024-04-26T22:10:24.426140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5496","last_scrub_stamp":"2024-04-26T22:10:24.425962+0000","last_deep_scrub":"40'5496","last_deep_scrub_stamp":"2024-04-26T22:10:24.425962+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.425962+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:59.427110+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":82892,"num_write_kb":1050745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'36","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.219085+0000","last_change":"2024-04-26T22:10:39.219085+0000","last_active":"2024-04-26T22:10:39.219085+0000","last_peered":"2024-04-26T22:10:39.219085+0000","last_clean":"2024-04-26T22:10:39.219085+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:10:39.219085+0000","last_undegraded":"2024-04-26T22:10:39.219085+0000","last_fullsized":"2024-04-26T22:10:39.219085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36","last_scrub_stamp":"2024-04-26T22:10:39.218959+0000","last_deep_scrub":"40'36","last_deep_scrub_stamp":"2024-04-26T22:10:39.218959+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.218959+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:07.989371+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":29,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"28'1","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.240571+0000","last_change":"2024-04-26T22:10:36.240630+0000","last_active":"2024-04-26T22:10:36.240571+0000","last_peered":"2024-04-26T22:10:36.240571+0000","last_clean":"2024-04-26T22:10:36.240571+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:10:36.240571+0000","last_undegraded":"2024-04-26T22:10:36.240571+0000","last_fullsized":"2024-04-26T22:10:36.240571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'1","last_scrub_stamp":"2024-04-26T22:10:36.240429+0000","last_deep_scrub":"28'1","last_deep_scrub_stamp":"2024-04-26T22:10:36.240429+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.240429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:46.646994+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"40'3789","reported_seq":3872,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.302299+0000","last_change":"2024-04-26T22:10:37.302344+0000","last_active":"2024-04-26T22:10:37.302299+0000","last_peered":"2024-04-26T22:10:37.302299+0000","last_clean":"2024-04-26T22:10:37.302299+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:10:37.302299+0000","last_undegraded":"2024-04-26T22:10:37.302299+0000","last_fullsized":"2024-04-26T22:10:37.302299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3789","last_scrub_stamp":"2024-04-26T22:10:37.302131+0000","last_deep_scrub":"40'3789","last_deep_scrub_stamp":"2024-04-26T22:10:37.302131+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.302131+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:36.214557+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":56913,"num_write_kb":707433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"40'4759","reported_seq":4827,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.169618+0000","last_change":"2024-04-26T22:10:38.169647+0000","last_active":"2024-04-26T22:10:38.169618+0000","last_peered":"2024-04-26T22:10:38.169618+0000","last_clean":"2024-04-26T22:10:38.169618+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:10:38.169618+0000","last_undegraded":"2024-04-26T22:10:38.169618+0000","last_fullsized":"2024-04-26T22:10:38.169618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4759","last_scrub_stamp":"2024-04-26T22:10:38.169516+0000","last_deep_scrub":"40'4759","last_deep_scrub_stamp":"2024-04-26T22:10:38.169516+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.169516+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:56.916728+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71808,"num_write_kb":921006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"40'7","reported_seq":75,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.354144+0000","last_change":"2024-04-26T22:10:31.354168+0000","last_active":"2024-04-26T22:10:31.354144+0000","last_peered":"2024-04-26T22:10:31.354144+0000","last_clean":"2024-04-26T22:10:31.354144+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:10:31.354144+0000","last_undegraded":"2024-04-26T22:10:31.354144+0000","last_fullsized":"2024-04-26T22:10:31.354144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'7","last_scrub_stamp":"2024-04-26T22:10:31.354057+0000","last_deep_scrub":"40'7","last_deep_scrub_stamp":"2024-04-26T22:10:31.354057+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.354057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:41.760623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"40'185","reported_seq":253,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.486350+0000","last_change":"2024-04-26T22:10:21.486350+0000","last_active":"2024-04-26T22:10:21.486350+0000","last_peered":"2024-04-26T22:10:21.486350+0000","last_clean":"2024-04-26T22:10:21.486350+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:10:21.486350+0000","last_undegraded":"2024-04-26T22:10:21.486350+0000","last_fullsized":"2024-04-26T22:10:21.486350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'185","last_scrub_stamp":"2024-04-26T22:10:21.486206+0000","last_deep_scrub":"40'185","last_deep_scrub_stamp":"2024-04-26T22:10:21.486206+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.486206+0000","objects_scrubbed":2,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:43.555095+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":27847,"num_write":173,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"40'3541","reported_seq":3619,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.309280+0000","last_change":"2024-04-26T22:10:39.309309+0000","last_active":"2024-04-26T22:10:39.309280+0000","last_peered":"2024-04-26T22:10:39.309280+0000","last_clean":"2024-04-26T22:10:39.309280+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:10:39.309280+0000","last_undegraded":"2024-04-26T22:10:39.309280+0000","last_fullsized":"2024-04-26T22:10:39.309280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3541","last_scrub_stamp":"2024-04-26T22:10:39.309160+0000","last_deep_scrub":"40'3541","last_deep_scrub_stamp":"2024-04-26T22:10:39.309160+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.309160+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:01.648035+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":53073,"num_write_kb":661602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'5106","reported_seq":5174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.815261+0000","last_change":"2024-04-26T22:10:30.815288+0000","last_active":"2024-04-26T22:10:30.815261+0000","last_peered":"2024-04-26T22:10:30.815261+0000","last_clean":"2024-04-26T22:10:30.815261+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:10:30.815261+0000","last_undegraded":"2024-04-26T22:10:30.815261+0000","last_fullsized":"2024-04-26T22:10:30.815261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5106","last_scrub_stamp":"2024-04-26T22:10:30.815149+0000","last_deep_scrub":"40'5106","last_deep_scrub_stamp":"2024-04-26T22:10:30.815149+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.815149+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:40.838350+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":77293,"num_write_kb":982493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"40'223","reported_seq":298,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.260885+0000","last_change":"2024-04-26T22:10:36.260916+0000","last_active":"2024-04-26T22:10:36.260885+0000","last_peered":"2024-04-26T22:10:36.260885+0000","last_clean":"2024-04-26T22:10:36.260885+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:10:36.260885+0000","last_undegraded":"2024-04-26T22:10:36.260885+0000","last_fullsized":"2024-04-26T22:10:36.260885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'223","last_scrub_stamp":"2024-04-26T22:10:36.260804+0000","last_deep_scrub":"40'223","last_deep_scrub_stamp":"2024-04-26T22:10:36.260804+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.260804+0000","objects_scrubbed":2,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:08.428051+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":213,"num_write_kb":8367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"40'5","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:20.475608+0000","last_change":"2024-04-26T22:10:20.475608+0000","last_active":"2024-04-26T22:10:20.475608+0000","last_peered":"2024-04-26T22:10:20.475608+0000","last_clean":"2024-04-26T22:10:20.475608+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:10:20.475608+0000","last_undegraded":"2024-04-26T22:10:20.475608+0000","last_fullsized":"2024-04-26T22:10:20.475608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:20.475457+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:20.475457+0000","last_clean_scrub_stamp":"2024-04-26T22:10:20.475457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:02.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"40'3541","reported_seq":3623,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.431937+0000","last_change":"2024-04-26T22:10:34.431964+0000","last_active":"2024-04-26T22:10:34.431937+0000","last_peered":"2024-04-26T22:10:34.431937+0000","last_clean":"2024-04-26T22:10:34.431937+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:10:34.431937+0000","last_undegraded":"2024-04-26T22:10:34.431937+0000","last_fullsized":"2024-04-26T22:10:34.431937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3541","last_scrub_stamp":"2024-04-26T22:10:34.431799+0000","last_deep_scrub":"40'3541","last_deep_scrub_stamp":"2024-04-26T22:10:34.431799+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.431799+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:00.092863+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":52805,"num_write_kb":629973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'5114","reported_seq":5183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.456582+0000","last_change":"2024-04-26T22:10:26.456626+0000","last_active":"2024-04-26T22:10:26.456582+0000","last_peered":"2024-04-26T22:10:26.456582+0000","last_clean":"2024-04-26T22:10:26.456582+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:10:26.456582+0000","last_undegraded":"2024-04-26T22:10:26.456582+0000","last_fullsized":"2024-04-26T22:10:26.456582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5114","last_scrub_stamp":"2024-04-26T22:10:26.456451+0000","last_deep_scrub":"40'5114","last_deep_scrub_stamp":"2024-04-26T22:10:26.456451+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.456451+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:17.496298+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":77095,"num_write_kb":967489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"40'19","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.557467+0000","last_change":"2024-04-26T22:10:24.557508+0000","last_active":"2024-04-26T22:10:24.557467+0000","last_peered":"2024-04-26T22:10:24.557467+0000","last_clean":"2024-04-26T22:10:24.557467+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:10:24.557467+0000","last_undegraded":"2024-04-26T22:10:24.557467+0000","last_fullsized":"2024-04-26T22:10:24.557467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'19","last_scrub_stamp":"2024-04-26T22:10:24.557372+0000","last_deep_scrub":"40'19","last_deep_scrub_stamp":"2024-04-26T22:10:24.557372+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.557372+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:31.018707+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88181,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":134,"num_write":15,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"40'6","reported_seq":81,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.408971+0000","last_change":"2024-04-26T22:10:25.409016+0000","last_active":"2024-04-26T22:10:25.408971+0000","last_peered":"2024-04-26T22:10:25.408971+0000","last_clean":"2024-04-26T22:10:25.408971+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:10:25.408971+0000","last_undegraded":"2024-04-26T22:10:25.408971+0000","last_fullsized":"2024-04-26T22:10:25.408971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:25.408850+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:25.408850+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.408850+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:11.494130+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'4139","reported_seq":4217,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.982615+0000","last_change":"2024-04-26T22:00:06.832200+0000","last_active":"2024-04-26T22:10:11.982615+0000","last_peered":"2024-04-26T22:10:11.982615+0000","last_clean":"2024-04-26T22:10:11.982615+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:10:11.982615+0000","last_undegraded":"2024-04-26T22:10:11.982615+0000","last_fullsized":"2024-04-26T22:10:11.982615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":62199,"num_write_kb":728313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"40'5239","reported_seq":5307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.132755+0000","last_change":"2024-04-26T22:10:27.132813+0000","last_active":"2024-04-26T22:10:27.132755+0000","last_peered":"2024-04-26T22:10:27.132755+0000","last_clean":"2024-04-26T22:10:27.132755+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:10:27.132755+0000","last_undegraded":"2024-04-26T22:10:27.132755+0000","last_fullsized":"2024-04-26T22:10:27.132755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5239","last_scrub_stamp":"2024-04-26T22:10:27.132630+0000","last_deep_scrub":"40'5239","last_deep_scrub_stamp":"2024-04-26T22:10:27.132630+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.132630+0000","objects_scrubbed":0,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:12.277463+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":79195,"num_write_kb":994121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"40'32","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.198699+0000","last_change":"2024-04-26T22:10:38.198739+0000","last_active":"2024-04-26T22:10:38.198699+0000","last_peered":"2024-04-26T22:10:38.198699+0000","last_clean":"2024-04-26T22:10:38.198699+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:10:38.198699+0000","last_undegraded":"2024-04-26T22:10:38.198699+0000","last_fullsized":"2024-04-26T22:10:38.198699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32","last_scrub_stamp":"2024-04-26T22:10:38.198618+0000","last_deep_scrub":"40'32","last_deep_scrub_stamp":"2024-04-26T22:10:38.198618+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.198618+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:23.794950+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":8201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"40'212","reported_seq":278,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.174042+0000","last_change":"2024-04-26T22:10:34.174067+0000","last_active":"2024-04-26T22:10:34.174042+0000","last_peered":"2024-04-26T22:10:34.174042+0000","last_clean":"2024-04-26T22:10:34.174042+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:10:34.174042+0000","last_undegraded":"2024-04-26T22:10:34.174042+0000","last_fullsized":"2024-04-26T22:10:34.174042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'212","last_scrub_stamp":"2024-04-26T22:10:34.173950+0000","last_deep_scrub":"40'212","last_deep_scrub_stamp":"2024-04-26T22:10:34.173950+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.173950+0000","objects_scrubbed":1,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.197151+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":74,"num_write":426,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"40'4145","reported_seq":4231,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.269577+0000","last_change":"2024-04-26T22:10:29.269577+0000","last_active":"2024-04-26T22:10:29.269577+0000","last_peered":"2024-04-26T22:10:29.269577+0000","last_clean":"2024-04-26T22:10:29.269577+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:10:29.269577+0000","last_undegraded":"2024-04-26T22:10:29.269577+0000","last_fullsized":"2024-04-26T22:10:29.269577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4145","last_scrub_stamp":"2024-04-26T22:10:29.269445+0000","last_deep_scrub":"40'4145","last_deep_scrub_stamp":"2024-04-26T22:10:29.269445+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.269445+0000","objects_scrubbed":0,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:09.727400+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":62372,"num_write_kb":775201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"40'5748","reported_seq":5815,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.011933+0000","last_change":"2024-04-26T22:10:32.011960+0000","last_active":"2024-04-26T22:10:32.011933+0000","last_peered":"2024-04-26T22:10:32.011933+0000","last_clean":"2024-04-26T22:10:32.011933+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:10:32.011933+0000","last_undegraded":"2024-04-26T22:10:32.011933+0000","last_fullsized":"2024-04-26T22:10:32.011933+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5748","last_scrub_stamp":"2024-04-26T22:10:32.011809+0000","last_deep_scrub":"40'5748","last_deep_scrub_stamp":"2024-04-26T22:10:32.011809+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.011809+0000","objects_scrubbed":0,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:17.844177+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":87080,"num_write_kb":1065746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"39'3","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.237247+0000","last_change":"2024-04-26T22:10:33.237288+0000","last_active":"2024-04-26T22:10:33.237247+0000","last_peered":"2024-04-26T22:10:33.237247+0000","last_clean":"2024-04-26T22:10:33.237247+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:10:33.237247+0000","last_undegraded":"2024-04-26T22:10:33.237247+0000","last_fullsized":"2024-04-26T22:10:33.237247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3","last_scrub_stamp":"2024-04-26T22:10:33.237154+0000","last_deep_scrub":"39'3","last_deep_scrub_stamp":"2024-04-26T22:10:33.237154+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.237154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:09.709068+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"40'119","reported_seq":226,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.205515+0000","last_change":"2024-04-26T22:10:35.205563+0000","last_active":"2024-04-26T22:10:35.205515+0000","last_peered":"2024-04-26T22:10:35.205515+0000","last_clean":"2024-04-26T22:10:35.205515+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:10:35.205515+0000","last_undegraded":"2024-04-26T22:10:35.205515+0000","last_fullsized":"2024-04-26T22:10:35.205515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'119","last_scrub_stamp":"2024-04-26T22:10:35.205248+0000","last_deep_scrub":"40'119","last_deep_scrub_stamp":"2024-04-26T22:10:35.205248+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.205248+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:53.620818+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"40'6","reported_seq":66,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.314475+0000","last_change":"2024-04-26T22:10:37.314501+0000","last_active":"2024-04-26T22:10:37.314475+0000","last_peered":"2024-04-26T22:10:37.314475+0000","last_clean":"2024-04-26T22:10:37.314475+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:10:37.314475+0000","last_undegraded":"2024-04-26T22:10:37.314475+0000","last_fullsized":"2024-04-26T22:10:37.314475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:37.314391+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:37.314391+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.314391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:11.459169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"40'3379","reported_seq":3469,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.455658+0000","last_change":"2024-04-26T22:10:30.455687+0000","last_active":"2024-04-26T22:10:30.455658+0000","last_peered":"2024-04-26T22:10:30.455658+0000","last_clean":"2024-04-26T22:10:30.455658+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:10:30.455658+0000","last_undegraded":"2024-04-26T22:10:30.455658+0000","last_fullsized":"2024-04-26T22:10:30.455658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3379","last_scrub_stamp":"2024-04-26T22:10:30.455550+0000","last_deep_scrub":"40'3379","last_deep_scrub_stamp":"2024-04-26T22:10:30.455550+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.455550+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:57.185356+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":50792,"num_write_kb":610820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"40'5433","reported_seq":5503,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.215101+0000","last_change":"2024-04-26T22:10:30.215131+0000","last_active":"2024-04-26T22:10:30.215101+0000","last_peered":"2024-04-26T22:10:30.215101+0000","last_clean":"2024-04-26T22:10:30.215101+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:10:30.215101+0000","last_undegraded":"2024-04-26T22:10:30.215101+0000","last_fullsized":"2024-04-26T22:10:30.215101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5433","last_scrub_stamp":"2024-04-26T22:10:30.214972+0000","last_deep_scrub":"40'5433","last_deep_scrub_stamp":"2024-04-26T22:10:30.214972+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.214972+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:52.553847+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":81848,"num_write_kb":1014508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"40'3","reported_seq":70,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.564717+0000","last_change":"2024-04-26T22:10:26.564744+0000","last_active":"2024-04-26T22:10:26.564717+0000","last_peered":"2024-04-26T22:10:26.564717+0000","last_clean":"2024-04-26T22:10:26.564717+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:10:26.564717+0000","last_undegraded":"2024-04-26T22:10:26.564717+0000","last_fullsized":"2024-04-26T22:10:26.564717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:26.564626+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:26.564626+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.564626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:56.616015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"40'202","reported_seq":262,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.285096+0000","last_change":"2024-04-26T22:10:36.285126+0000","last_active":"2024-04-26T22:10:36.285096+0000","last_peered":"2024-04-26T22:10:36.285096+0000","last_clean":"2024-04-26T22:10:36.285096+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:10:36.285096+0000","last_undegraded":"2024-04-26T22:10:36.285096+0000","last_fullsized":"2024-04-26T22:10:36.285096+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'202","last_scrub_stamp":"2024-04-26T22:10:36.285007+0000","last_deep_scrub":"40'202","last_deep_scrub_stamp":"2024-04-26T22:10:36.285007+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.285007+0000","objects_scrubbed":1,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:09.769505+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":196,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"40'3780","reported_seq":3866,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.399702+0000","last_change":"2024-04-26T22:10:33.399727+0000","last_active":"2024-04-26T22:10:33.399702+0000","last_peered":"2024-04-26T22:10:33.399702+0000","last_clean":"2024-04-26T22:10:33.399702+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:10:33.399702+0000","last_undegraded":"2024-04-26T22:10:33.399702+0000","last_fullsized":"2024-04-26T22:10:33.399702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3780","last_scrub_stamp":"2024-04-26T22:10:33.399366+0000","last_deep_scrub":"40'3780","last_deep_scrub_stamp":"2024-04-26T22:10:33.399366+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.399366+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:24.439213+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":56837,"num_write_kb":699720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"40'5733","reported_seq":5803,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.254368+0000","last_change":"2024-04-26T22:10:32.254411+0000","last_active":"2024-04-26T22:10:32.254368+0000","last_peered":"2024-04-26T22:10:32.254368+0000","last_clean":"2024-04-26T22:10:32.254368+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:10:32.254368+0000","last_undegraded":"2024-04-26T22:10:32.254368+0000","last_fullsized":"2024-04-26T22:10:32.254368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5733","last_scrub_stamp":"2024-04-26T22:10:32.254230+0000","last_deep_scrub":"40'5733","last_deep_scrub_stamp":"2024-04-26T22:10:32.254230+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.254230+0000","objects_scrubbed":0,"log_size":5733,"log_dups_size":0,"ondisk_log_size":5733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:36.887990+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":86585,"num_write_kb":1068440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"40'21","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.386012+0000","last_change":"2024-04-26T22:10:29.386047+0000","last_active":"2024-04-26T22:10:29.386012+0000","last_peered":"2024-04-26T22:10:29.386012+0000","last_clean":"2024-04-26T22:10:29.386012+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:10:29.386012+0000","last_undegraded":"2024-04-26T22:10:29.386012+0000","last_fullsized":"2024-04-26T22:10:29.386012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:29.385929+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:29.385929+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.385929+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:39.409130+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16390,"num_write":17,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"40'382","reported_seq":458,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.224869+0000","last_change":"2024-04-26T22:10:35.224908+0000","last_active":"2024-04-26T22:10:35.224869+0000","last_peered":"2024-04-26T22:10:35.224869+0000","last_clean":"2024-04-26T22:10:35.224869+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:10:35.224869+0000","last_undegraded":"2024-04-26T22:10:35.224869+0000","last_fullsized":"2024-04-26T22:10:35.224869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'382","last_scrub_stamp":"2024-04-26T22:10:35.224766+0000","last_deep_scrub":"40'382","last_deep_scrub_stamp":"2024-04-26T22:10:35.224766+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.224766+0000","objects_scrubbed":3,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:51.709281+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":25852,"num_write":364,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"40'3909","reported_seq":3988,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.963314+0000","last_change":"2024-04-26T22:00:06.822398+0000","last_active":"2024-04-26T22:10:11.963314+0000","last_peered":"2024-04-26T22:10:11.963314+0000","last_clean":"2024-04-26T22:10:11.963314+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:10:11.963314+0000","last_undegraded":"2024-04-26T22:10:11.963314+0000","last_fullsized":"2024-04-26T22:10:11.963314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":58819,"num_write_kb":699051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"40'6248","reported_seq":6315,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.199932+0000","last_change":"2024-04-26T22:10:24.199962+0000","last_active":"2024-04-26T22:10:24.199932+0000","last_peered":"2024-04-26T22:10:24.199932+0000","last_clean":"2024-04-26T22:10:24.199932+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:10:24.199932+0000","last_undegraded":"2024-04-26T22:10:24.199932+0000","last_fullsized":"2024-04-26T22:10:24.199932+0000","mapping_epoch":18,"log_start":"28'400","ondisk_log_start":"28'400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6248","last_scrub_stamp":"2024-04-26T22:10:24.199807+0000","last_deep_scrub":"40'6248","last_deep_scrub_stamp":"2024-04-26T22:10:24.199807+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.199807+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:49.860871+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":94569,"num_write_kb":1160134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"40'113","reported_seq":194,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.541477+0000","last_change":"2024-04-26T22:10:25.541510+0000","last_active":"2024-04-26T22:10:25.541477+0000","last_peered":"2024-04-26T22:10:25.541477+0000","last_clean":"2024-04-26T22:10:25.541477+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:10:25.541477+0000","last_undegraded":"2024-04-26T22:10:25.541477+0000","last_fullsized":"2024-04-26T22:10:25.541477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'113","last_scrub_stamp":"2024-04-26T22:10:25.541375+0000","last_deep_scrub":"40'113","last_deep_scrub_stamp":"2024-04-26T22:10:25.541375+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.541375+0000","objects_scrubbed":2,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:18.885415+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":106,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"40'14","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.252540+0000","last_change":"2024-04-26T22:10:23.252568+0000","last_active":"2024-04-26T22:10:23.252540+0000","last_peered":"2024-04-26T22:10:23.252540+0000","last_clean":"2024-04-26T22:10:23.252540+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:10:23.252540+0000","last_undegraded":"2024-04-26T22:10:23.252540+0000","last_fullsized":"2024-04-26T22:10:23.252540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'14","last_scrub_stamp":"2024-04-26T22:10:23.252456+0000","last_deep_scrub":"40'14","last_deep_scrub_stamp":"2024-04-26T22:10:23.252456+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.252456+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:38.945444+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"40'4399","reported_seq":4482,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.441280+0000","last_change":"2024-04-26T22:10:31.441307+0000","last_active":"2024-04-26T22:10:31.441280+0000","last_peered":"2024-04-26T22:10:31.441280+0000","last_clean":"2024-04-26T22:10:31.441280+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:10:31.441280+0000","last_undegraded":"2024-04-26T22:10:31.441280+0000","last_fullsized":"2024-04-26T22:10:31.441280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4399","last_scrub_stamp":"2024-04-26T22:10:31.441157+0000","last_deep_scrub":"40'4399","last_deep_scrub_stamp":"2024-04-26T22:10:31.441157+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.441157+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:06.442305+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":66147,"num_write_kb":822005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"40'5224","reported_seq":5291,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.212449+0000","last_change":"2024-04-26T22:10:23.212477+0000","last_active":"2024-04-26T22:10:23.212449+0000","last_peered":"2024-04-26T22:10:23.212449+0000","last_clean":"2024-04-26T22:10:23.212449+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:10:23.212449+0000","last_undegraded":"2024-04-26T22:10:23.212449+0000","last_fullsized":"2024-04-26T22:10:23.212449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5224","last_scrub_stamp":"2024-04-26T22:10:23.212329+0000","last_deep_scrub":"40'5224","last_deep_scrub_stamp":"2024-04-26T22:10:23.212329+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.212329+0000","objects_scrubbed":0,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:14.252177+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":78957,"num_write_kb":961695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"40'18","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.803568+0000","last_change":"2024-04-26T22:10:28.803600+0000","last_active":"2024-04-26T22:10:28.803568+0000","last_peered":"2024-04-26T22:10:28.803568+0000","last_clean":"2024-04-26T22:10:28.803568+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:10:28.803568+0000","last_undegraded":"2024-04-26T22:10:28.803568+0000","last_fullsized":"2024-04-26T22:10:28.803568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'18","last_scrub_stamp":"2024-04-26T22:10:28.803483+0000","last_deep_scrub":"40'18","last_deep_scrub_stamp":"2024-04-26T22:10:28.803483+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.803483+0000","objects_scrubbed":2,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:54.464547+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":12294,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"40'3","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.753414+0000","last_change":"2024-04-26T22:10:26.753439+0000","last_active":"2024-04-26T22:10:26.753414+0000","last_peered":"2024-04-26T22:10:26.753414+0000","last_clean":"2024-04-26T22:10:26.753414+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:10:26.753414+0000","last_undegraded":"2024-04-26T22:10:26.753414+0000","last_fullsized":"2024-04-26T22:10:26.753414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:26.753319+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:26.753319+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.753319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:12.838598+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"40'3562","reported_seq":3639,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988853+0000","last_change":"2024-04-26T22:00:06.827598+0000","last_active":"2024-04-26T22:10:11.988853+0000","last_peered":"2024-04-26T22:10:11.988853+0000","last_clean":"2024-04-26T22:10:11.988853+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:10:11.988853+0000","last_undegraded":"2024-04-26T22:10:11.988853+0000","last_fullsized":"2024-04-26T22:10:11.988853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3562,"log_dups_size":0,"ondisk_log_size":3562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":53350,"num_write_kb":651982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"40'5336","reported_seq":5391,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.370312+0000","last_change":"2024-04-26T22:10:37.370338+0000","last_active":"2024-04-26T22:10:37.370312+0000","last_peered":"2024-04-26T22:10:37.370312+0000","last_clean":"2024-04-26T22:10:37.370312+0000","last_became_active":"2024-04-26T22:00:39.150992+0000","last_became_peered":"2024-04-26T22:00:39.150992+0000","last_unstale":"2024-04-26T22:10:37.370312+0000","last_undegraded":"2024-04-26T22:10:37.370312+0000","last_fullsized":"2024-04-26T22:10:37.370312+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5336","last_scrub_stamp":"2024-04-26T22:10:37.370208+0000","last_deep_scrub":"40'5336","last_deep_scrub_stamp":"2024-04-26T22:10:37.370208+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.370208+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:10.854706+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":80461,"num_write_kb":1019956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"40'293","reported_seq":398,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.535350+0000","last_change":"2024-04-26T22:10:23.535380+0000","last_active":"2024-04-26T22:10:23.535350+0000","last_peered":"2024-04-26T22:10:23.535350+0000","last_clean":"2024-04-26T22:10:23.535350+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:10:23.535350+0000","last_undegraded":"2024-04-26T22:10:23.535350+0000","last_fullsized":"2024-04-26T22:10:23.535350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'293","last_scrub_stamp":"2024-04-26T22:10:23.535274+0000","last_deep_scrub":"40'293","last_deep_scrub_stamp":"2024-04-26T22:10:23.535274+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.535274+0000","objects_scrubbed":6,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:00.794998+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":12387,"num_write":284,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"40'4","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.154205+0000","last_change":"2024-04-26T22:10:23.154226+0000","last_active":"2024-04-26T22:10:23.154205+0000","last_peered":"2024-04-26T22:10:23.154205+0000","last_clean":"2024-04-26T22:10:23.154205+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:10:23.154205+0000","last_undegraded":"2024-04-26T22:10:23.154205+0000","last_fullsized":"2024-04-26T22:10:23.154205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4","last_scrub_stamp":"2024-04-26T22:10:23.154129+0000","last_deep_scrub":"40'4","last_deep_scrub_stamp":"2024-04-26T22:10:23.154129+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.154129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:16.498170+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"40'4185","reported_seq":4271,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.989837+0000","last_change":"2024-04-26T22:00:06.832637+0000","last_active":"2024-04-26T22:10:11.989837+0000","last_peered":"2024-04-26T22:10:11.989837+0000","last_clean":"2024-04-26T22:10:11.989837+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:10:11.989837+0000","last_undegraded":"2024-04-26T22:10:11.989837+0000","last_fullsized":"2024-04-26T22:10:11.989837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":62914,"num_write_kb":762533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'5750","reported_seq":5819,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.343589+0000","last_change":"2024-04-26T22:10:32.343618+0000","last_active":"2024-04-26T22:10:32.343589+0000","last_peered":"2024-04-26T22:10:32.343589+0000","last_clean":"2024-04-26T22:10:32.343589+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:10:32.343589+0000","last_undegraded":"2024-04-26T22:10:32.343589+0000","last_fullsized":"2024-04-26T22:10:32.343589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5750","last_scrub_stamp":"2024-04-26T22:10:32.343474+0000","last_deep_scrub":"40'5750","last_deep_scrub_stamp":"2024-04-26T22:10:32.343474+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.343474+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:43.677224+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":86794,"num_write_kb":1073964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"40'6","reported_seq":73,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.354569+0000","last_change":"2024-04-26T22:10:28.354613+0000","last_active":"2024-04-26T22:10:28.354569+0000","last_peered":"2024-04-26T22:10:28.354569+0000","last_clean":"2024-04-26T22:10:28.354569+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:10:28.354569+0000","last_undegraded":"2024-04-26T22:10:28.354569+0000","last_fullsized":"2024-04-26T22:10:28.354569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:28.354466+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:28.354466+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.354466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:10.649352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"40'3","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.967991+0000","last_change":"2024-04-26T22:10:21.968014+0000","last_active":"2024-04-26T22:10:21.967991+0000","last_peered":"2024-04-26T22:10:21.967991+0000","last_clean":"2024-04-26T22:10:21.967991+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:10:21.967991+0000","last_undegraded":"2024-04-26T22:10:21.967991+0000","last_fullsized":"2024-04-26T22:10:21.967991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:21.967918+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:21.967918+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.967918+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:13.007765+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"40'3717","reported_seq":3787,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.883943+0000","last_change":"2024-04-26T22:00:06.832004+0000","last_active":"2024-04-26T22:10:11.883943+0000","last_peered":"2024-04-26T22:10:11.883943+0000","last_clean":"2024-04-26T22:10:11.883943+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:10:11.883943+0000","last_undegraded":"2024-04-26T22:10:11.883943+0000","last_fullsized":"2024-04-26T22:10:11.883943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3717,"log_dups_size":0,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":55831,"num_write_kb":673382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"40'6101","reported_seq":6170,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.165399+0000","last_change":"2024-04-26T22:10:25.165431+0000","last_active":"2024-04-26T22:10:25.165399+0000","last_peered":"2024-04-26T22:10:25.165399+0000","last_clean":"2024-04-26T22:10:25.165399+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:10:25.165399+0000","last_undegraded":"2024-04-26T22:10:25.165399+0000","last_fullsized":"2024-04-26T22:10:25.165399+0000","mapping_epoch":18,"log_start":"28'300","ondisk_log_start":"28'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6101","last_scrub_stamp":"2024-04-26T22:10:25.165276+0000","last_deep_scrub":"40'6101","last_deep_scrub_stamp":"2024-04-26T22:10:25.165276+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.165276+0000","objects_scrubbed":0,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:47.066166+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":92157,"num_write_kb":1133252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"40'5","reported_seq":72,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.207081+0000","last_change":"2024-04-26T22:10:37.207109+0000","last_active":"2024-04-26T22:10:37.207081+0000","last_peered":"2024-04-26T22:10:37.207081+0000","last_clean":"2024-04-26T22:10:37.207081+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:10:37.207081+0000","last_undegraded":"2024-04-26T22:10:37.207081+0000","last_fullsized":"2024-04-26T22:10:37.207081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:37.206986+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:37.206986+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.206986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:46.662405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"40'377","reported_seq":479,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.994206+0000","last_change":"2024-04-26T22:10:26.994244+0000","last_active":"2024-04-26T22:10:26.994206+0000","last_peered":"2024-04-26T22:10:26.994206+0000","last_clean":"2024-04-26T22:10:26.994206+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:10:26.994206+0000","last_undegraded":"2024-04-26T22:10:26.994206+0000","last_fullsized":"2024-04-26T22:10:26.994206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'377","last_scrub_stamp":"2024-04-26T22:10:26.994121+0000","last_deep_scrub":"40'377","last_deep_scrub_stamp":"2024-04-26T22:10:26.994121+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.994121+0000","objects_scrubbed":5,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:36.822697+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":31157,"num_write":363,"num_write_kb":8442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"40'5464","reported_seq":5546,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.990237+0000","last_change":"2024-04-26T22:00:06.819748+0000","last_active":"2024-04-26T22:10:11.990237+0000","last_peered":"2024-04-26T22:10:11.990237+0000","last_clean":"2024-04-26T22:10:11.990237+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:10:11.990237+0000","last_undegraded":"2024-04-26T22:10:11.990237+0000","last_fullsized":"2024-04-26T22:10:11.990237+0000","mapping_epoch":22,"log_start":"31'200","ondisk_log_start":"31'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":82133,"num_write_kb":980051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'5077","reported_seq":5142,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.164714+0000","last_change":"2024-04-26T22:10:21.164737+0000","last_active":"2024-04-26T22:10:21.164714+0000","last_peered":"2024-04-26T22:10:21.164714+0000","last_clean":"2024-04-26T22:10:21.164714+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:10:21.164714+0000","last_undegraded":"2024-04-26T22:10:21.164714+0000","last_fullsized":"2024-04-26T22:10:21.164714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5077","last_scrub_stamp":"2024-04-26T22:10:21.164618+0000","last_deep_scrub":"40'5077","last_deep_scrub_stamp":"2024-04-26T22:10:21.164618+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.164618+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:58.424342+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":76618,"num_write_kb":958363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"40'23","reported_seq":109,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.119062+0000","last_change":"2024-04-26T22:10:25.119109+0000","last_active":"2024-04-26T22:10:25.119062+0000","last_peered":"2024-04-26T22:10:25.119062+0000","last_clean":"2024-04-26T22:10:25.119062+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:10:25.119062+0000","last_undegraded":"2024-04-26T22:10:25.119062+0000","last_fullsized":"2024-04-26T22:10:25.119062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'23","last_scrub_stamp":"2024-04-26T22:10:25.118971+0000","last_deep_scrub":"40'23","last_deep_scrub_stamp":"2024-04-26T22:10:25.118971+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.118971+0000","objects_scrubbed":4,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:38.211661+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":32816,"num_write":13,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"40'16","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.253601+0000","last_change":"2024-04-26T22:10:25.253631+0000","last_active":"2024-04-26T22:10:25.253601+0000","last_peered":"2024-04-26T22:10:25.253601+0000","last_clean":"2024-04-26T22:10:25.253601+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:10:25.253601+0000","last_undegraded":"2024-04-26T22:10:25.253601+0000","last_fullsized":"2024-04-26T22:10:25.253601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'16","last_scrub_stamp":"2024-04-26T22:10:25.253519+0000","last_deep_scrub":"40'16","last_deep_scrub_stamp":"2024-04-26T22:10:25.253519+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.253519+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:21.547699+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"40'4714","reported_seq":4806,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.998967+0000","last_change":"2024-04-26T22:10:32.999009+0000","last_active":"2024-04-26T22:10:32.998967+0000","last_peered":"2024-04-26T22:10:32.998967+0000","last_clean":"2024-04-26T22:10:32.998967+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:10:32.998967+0000","last_undegraded":"2024-04-26T22:10:32.998967+0000","last_fullsized":"2024-04-26T22:10:32.998967+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4714","last_scrub_stamp":"2024-04-26T22:10:32.998840+0000","last_deep_scrub":"40'4714","last_deep_scrub_stamp":"2024-04-26T22:10:32.998840+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.998840+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:05.329132+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":70866,"num_write_kb":843772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"40'5114","reported_seq":5179,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.217373+0000","last_change":"2024-04-26T22:10:28.217402+0000","last_active":"2024-04-26T22:10:28.217373+0000","last_peered":"2024-04-26T22:10:28.217373+0000","last_clean":"2024-04-26T22:10:28.217373+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:10:28.217373+0000","last_undegraded":"2024-04-26T22:10:28.217373+0000","last_fullsized":"2024-04-26T22:10:28.217373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5114","last_scrub_stamp":"2024-04-26T22:10:28.217250+0000","last_deep_scrub":"40'5114","last_deep_scrub_stamp":"2024-04-26T22:10:28.217250+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.217250+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:38.045826+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77180,"num_write_kb":958350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"40'829","reported_seq":931,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.400024+0000","last_change":"2024-04-26T22:10:27.400067+0000","last_active":"2024-04-26T22:10:27.400024+0000","last_peered":"2024-04-26T22:10:27.400024+0000","last_clean":"2024-04-26T22:10:27.400024+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:10:27.400024+0000","last_undegraded":"2024-04-26T22:10:27.400024+0000","last_fullsized":"2024-04-26T22:10:27.400024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'829","last_scrub_stamp":"2024-04-26T22:10:27.399913+0000","last_deep_scrub":"40'829","last_deep_scrub_stamp":"2024-04-26T22:10:27.399913+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.399913+0000","objects_scrubbed":6,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:01.351892+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":36914,"num_write":813,"num_write_kb":13003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"40'5","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.444749+0000","last_change":"2024-04-26T22:10:23.444787+0000","last_active":"2024-04-26T22:10:23.444749+0000","last_peered":"2024-04-26T22:10:23.444749+0000","last_clean":"2024-04-26T22:10:23.444749+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:10:23.444749+0000","last_undegraded":"2024-04-26T22:10:23.444749+0000","last_fullsized":"2024-04-26T22:10:23.444749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:23.444637+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:23.444637+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.444637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:50.174442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"40'3745","reported_seq":3824,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.928243+0000","last_change":"2024-04-26T22:00:06.826690+0000","last_active":"2024-04-26T22:10:11.928243+0000","last_peered":"2024-04-26T22:10:11.928243+0000","last_clean":"2024-04-26T22:10:11.928243+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:10:11.928243+0000","last_undegraded":"2024-04-26T22:10:11.928243+0000","last_fullsized":"2024-04-26T22:10:11.928243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56184,"num_write_kb":680067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"40'7267","reported_seq":7335,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.796950+0000","last_change":"2024-04-26T22:10:33.797021+0000","last_active":"2024-04-26T22:10:33.796950+0000","last_peered":"2024-04-26T22:10:33.796950+0000","last_clean":"2024-04-26T22:10:33.796950+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:10:33.796950+0000","last_undegraded":"2024-04-26T22:10:33.796950+0000","last_fullsized":"2024-04-26T22:10:33.796950+0000","mapping_epoch":18,"log_start":"30'1800","ondisk_log_start":"30'1800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'7267","last_scrub_stamp":"2024-04-26T22:10:33.796788+0000","last_deep_scrub":"40'7267","last_deep_scrub_stamp":"2024-04-26T22:10:33.796788+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.796788+0000","objects_scrubbed":0,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:14.254743+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":109924,"num_write_kb":1323267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"40'372","reported_seq":458,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.286226+0000","last_change":"2024-04-26T22:10:34.286266+0000","last_active":"2024-04-26T22:10:34.286226+0000","last_peered":"2024-04-26T22:10:34.286226+0000","last_clean":"2024-04-26T22:10:34.286226+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:10:34.286226+0000","last_undegraded":"2024-04-26T22:10:34.286226+0000","last_fullsized":"2024-04-26T22:10:34.286226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'372","last_scrub_stamp":"2024-04-26T22:10:34.286145+0000","last_deep_scrub":"40'372","last_deep_scrub_stamp":"2024-04-26T22:10:34.286145+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.286145+0000","objects_scrubbed":3,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:24.958635+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":367,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"40'235","reported_seq":320,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.790617+0000","last_change":"2024-04-26T22:10:24.790685+0000","last_active":"2024-04-26T22:10:24.790617+0000","last_peered":"2024-04-26T22:10:24.790617+0000","last_clean":"2024-04-26T22:10:24.790617+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:10:24.790617+0000","last_undegraded":"2024-04-26T22:10:24.790617+0000","last_fullsized":"2024-04-26T22:10:24.790617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'235","last_scrub_stamp":"2024-04-26T22:10:24.790516+0000","last_deep_scrub":"40'235","last_deep_scrub_stamp":"2024-04-26T22:10:24.790516+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.790516+0000","objects_scrubbed":4,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:51.520322+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":33257,"num_write":223,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"40'4111","reported_seq":4197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.367556+0000","last_change":"2024-04-26T22:10:39.367581+0000","last_active":"2024-04-26T22:10:39.367556+0000","last_peered":"2024-04-26T22:10:39.367556+0000","last_clean":"2024-04-26T22:10:39.367556+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:10:39.367556+0000","last_undegraded":"2024-04-26T22:10:39.367556+0000","last_fullsized":"2024-04-26T22:10:39.367556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4111","last_scrub_stamp":"2024-04-26T22:10:39.367450+0000","last_deep_scrub":"40'4111","last_deep_scrub_stamp":"2024-04-26T22:10:39.367450+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.367450+0000","objects_scrubbed":0,"log_size":4111,"log_dups_size":0,"ondisk_log_size":4111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:56.087982+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61736,"num_write_kb":766472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"40'4628","reported_seq":4697,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.106830+0000","last_change":"2024-04-26T22:10:31.106858+0000","last_active":"2024-04-26T22:10:31.106830+0000","last_peered":"2024-04-26T22:10:31.106830+0000","last_clean":"2024-04-26T22:10:31.106830+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:10:31.106830+0000","last_undegraded":"2024-04-26T22:10:31.106830+0000","last_fullsized":"2024-04-26T22:10:31.106830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4628","last_scrub_stamp":"2024-04-26T22:10:31.106732+0000","last_deep_scrub":"40'4628","last_deep_scrub_stamp":"2024-04-26T22:10:31.106732+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.106732+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:22.146579+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":69686,"num_write_kb":866678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"40'103","reported_seq":172,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.305006+0000","last_change":"2024-04-26T22:10:35.305061+0000","last_active":"2024-04-26T22:10:35.305006+0000","last_peered":"2024-04-26T22:10:35.305006+0000","last_clean":"2024-04-26T22:10:35.305006+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:10:35.305006+0000","last_undegraded":"2024-04-26T22:10:35.305006+0000","last_fullsized":"2024-04-26T22:10:35.305006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'103","last_scrub_stamp":"2024-04-26T22:10:35.304882+0000","last_deep_scrub":"40'103","last_deep_scrub_stamp":"2024-04-26T22:10:35.304882+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.304882+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.441615+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":98,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"40'16","reported_seq":118,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.011647+0000","last_change":"2024-04-26T22:10:29.011685+0000","last_active":"2024-04-26T22:10:29.011647+0000","last_peered":"2024-04-26T22:10:29.011647+0000","last_clean":"2024-04-26T22:10:29.011647+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:10:29.011647+0000","last_undegraded":"2024-04-26T22:10:29.011647+0000","last_fullsized":"2024-04-26T22:10:29.011647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'16","last_scrub_stamp":"2024-04-26T22:10:29.011558+0000","last_deep_scrub":"40'16","last_deep_scrub_stamp":"2024-04-26T22:10:29.011558+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.011558+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:51.350434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'4365","reported_seq":4454,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.969040+0000","last_change":"2024-04-26T22:00:06.819475+0000","last_active":"2024-04-26T22:10:11.969040+0000","last_peered":"2024-04-26T22:10:11.969040+0000","last_clean":"2024-04-26T22:10:11.969040+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:10:11.969040+0000","last_undegraded":"2024-04-26T22:10:11.969040+0000","last_fullsized":"2024-04-26T22:10:11.969040+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4365,"log_dups_size":0,"ondisk_log_size":4365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":65682,"num_write_kb":806692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"40'5549","reported_seq":5622,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.230468+0000","last_change":"2024-04-26T22:10:28.230494+0000","last_active":"2024-04-26T22:10:28.230468+0000","last_peered":"2024-04-26T22:10:28.230468+0000","last_clean":"2024-04-26T22:10:28.230468+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:10:28.230468+0000","last_undegraded":"2024-04-26T22:10:28.230468+0000","last_fullsized":"2024-04-26T22:10:28.230468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5549","last_scrub_stamp":"2024-04-26T22:10:28.230358+0000","last_deep_scrub":"40'5549","last_deep_scrub_stamp":"2024-04-26T22:10:28.230358+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.230358+0000","objects_scrubbed":0,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:02.375137+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":83744,"num_write_kb":1000289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"40'4","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.348286+0000","last_change":"2024-04-26T22:10:31.348313+0000","last_active":"2024-04-26T22:10:31.348286+0000","last_peered":"2024-04-26T22:10:31.348286+0000","last_clean":"2024-04-26T22:10:31.348286+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:10:31.348286+0000","last_undegraded":"2024-04-26T22:10:31.348286+0000","last_fullsized":"2024-04-26T22:10:31.348286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4","last_scrub_stamp":"2024-04-26T22:10:31.348178+0000","last_deep_scrub":"40'4","last_deep_scrub_stamp":"2024-04-26T22:10:31.348178+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.348178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.937782+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'6","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.951899+0000","last_change":"2024-04-26T22:10:24.951929+0000","last_active":"2024-04-26T22:10:24.951899+0000","last_peered":"2024-04-26T22:10:24.951899+0000","last_clean":"2024-04-26T22:10:24.951899+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:10:24.951899+0000","last_undegraded":"2024-04-26T22:10:24.951899+0000","last_fullsized":"2024-04-26T22:10:24.951899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:24.951797+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:24.951797+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.951797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:34.974998+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4980","reported_seq":5064,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.527996+0000","last_change":"2024-04-26T22:00:06.821258+0000","last_active":"2024-04-26T22:10:22.527996+0000","last_peered":"2024-04-26T22:10:22.527996+0000","last_clean":"2024-04-26T22:10:22.527996+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:10:22.527996+0000","last_undegraded":"2024-04-26T22:10:22.527996+0000","last_fullsized":"2024-04-26T22:10:22.527996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":74941,"num_write_kb":925745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"40'17","reported_seq":81,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.140744+0000","last_change":"2024-04-26T22:10:33.140770+0000","last_active":"2024-04-26T22:10:33.140744+0000","last_peered":"2024-04-26T22:10:33.140744+0000","last_clean":"2024-04-26T22:10:33.140744+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:10:33.140744+0000","last_undegraded":"2024-04-26T22:10:33.140744+0000","last_fullsized":"2024-04-26T22:10:33.140744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'17","last_scrub_stamp":"2024-04-26T22:10:33.140667+0000","last_deep_scrub":"40'17","last_deep_scrub_stamp":"2024-04-26T22:10:33.140667+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.140667+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:29.434847+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":8,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"40'6345","reported_seq":6415,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.408724+0000","last_change":"2024-04-26T22:10:28.408756+0000","last_active":"2024-04-26T22:10:28.408724+0000","last_peered":"2024-04-26T22:10:28.408724+0000","last_clean":"2024-04-26T22:10:28.408724+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:10:28.408724+0000","last_undegraded":"2024-04-26T22:10:28.408724+0000","last_fullsized":"2024-04-26T22:10:28.408724+0000","mapping_epoch":18,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6345","last_scrub_stamp":"2024-04-26T22:10:28.408608+0000","last_deep_scrub":"40'6345","last_deep_scrub_stamp":"2024-04-26T22:10:28.408608+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.408608+0000","objects_scrubbed":0,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:24.702788+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":95977,"num_write_kb":1176758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"40'21","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.263535+0000","last_change":"2024-04-26T22:10:34.263586+0000","last_active":"2024-04-26T22:10:34.263535+0000","last_peered":"2024-04-26T22:10:34.263535+0000","last_clean":"2024-04-26T22:10:34.263535+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:10:34.263535+0000","last_undegraded":"2024-04-26T22:10:34.263535+0000","last_fullsized":"2024-04-26T22:10:34.263535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:34.263431+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:34.263431+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.263431+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:06.593723+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":28672,"num_write":16,"num_write_kb":8206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"40'4260","reported_seq":4339,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.279258+0000","last_change":"2024-04-26T22:10:38.279283+0000","last_active":"2024-04-26T22:10:38.279258+0000","last_peered":"2024-04-26T22:10:38.279258+0000","last_clean":"2024-04-26T22:10:38.279258+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:10:38.279258+0000","last_undegraded":"2024-04-26T22:10:38.279258+0000","last_fullsized":"2024-04-26T22:10:38.279258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4260","last_scrub_stamp":"2024-04-26T22:10:38.279159+0000","last_deep_scrub":"40'4260","last_deep_scrub_stamp":"2024-04-26T22:10:38.279159+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.279159+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:54.999691+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":64151,"num_write_kb":776804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"40'28","reported_seq":109,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.318402+0000","last_change":"2024-04-26T22:10:35.318445+0000","last_active":"2024-04-26T22:10:35.318402+0000","last_peered":"2024-04-26T22:10:35.318402+0000","last_clean":"2024-04-26T22:10:35.318402+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:10:35.318402+0000","last_undegraded":"2024-04-26T22:10:35.318402+0000","last_fullsized":"2024-04-26T22:10:35.318402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28","last_scrub_stamp":"2024-04-26T22:10:35.318315+0000","last_deep_scrub":"40'28","last_deep_scrub_stamp":"2024-04-26T22:10:35.318315+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.318315+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:45.341516+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20512,"num_write":20,"num_write_kb":4102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"40'5323","reported_seq":5391,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.010733+0000","last_change":"2024-04-26T22:10:30.010762+0000","last_active":"2024-04-26T22:10:30.010733+0000","last_peered":"2024-04-26T22:10:30.010733+0000","last_clean":"2024-04-26T22:10:30.010733+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:10:30.010733+0000","last_undegraded":"2024-04-26T22:10:30.010733+0000","last_fullsized":"2024-04-26T22:10:30.010733+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5323","last_scrub_stamp":"2024-04-26T22:10:30.010619+0000","last_deep_scrub":"40'5323","last_deep_scrub_stamp":"2024-04-26T22:10:30.010619+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.010619+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:04.084603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":80432,"num_write_kb":1004371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"40'47","reported_seq":116,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.065136+0000","last_change":"2024-04-26T22:10:37.065172+0000","last_active":"2024-04-26T22:10:37.065136+0000","last_peered":"2024-04-26T22:10:37.065136+0000","last_clean":"2024-04-26T22:10:37.065136+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:10:37.065136+0000","last_undegraded":"2024-04-26T22:10:37.065136+0000","last_fullsized":"2024-04-26T22:10:37.065136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'47","last_scrub_stamp":"2024-04-26T22:10:37.065038+0000","last_deep_scrub":"40'47","last_deep_scrub_stamp":"2024-04-26T22:10:37.065038+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.065038+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:22.661370+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":41,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'4905","reported_seq":5000,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.404486+0000","last_change":"2024-04-26T22:10:40.404515+0000","last_active":"2024-04-26T22:10:40.404486+0000","last_peered":"2024-04-26T22:10:40.404486+0000","last_clean":"2024-04-26T22:10:40.404486+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:10:40.404486+0000","last_undegraded":"2024-04-26T22:10:40.404486+0000","last_fullsized":"2024-04-26T22:10:40.404486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4905","last_scrub_stamp":"2024-04-26T22:10:40.404383+0000","last_deep_scrub":"40'4905","last_deep_scrub_stamp":"2024-04-26T22:10:40.404383+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.404383+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:29.706555+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":74018,"num_write_kb":905043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"40'148","reported_seq":225,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.396878+0000","last_change":"2024-04-26T22:10:29.396913+0000","last_active":"2024-04-26T22:10:29.396878+0000","last_peered":"2024-04-26T22:10:29.396878+0000","last_clean":"2024-04-26T22:10:29.396878+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:10:29.396878+0000","last_undegraded":"2024-04-26T22:10:29.396878+0000","last_fullsized":"2024-04-26T22:10:29.396878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'148","last_scrub_stamp":"2024-04-26T22:10:29.396793+0000","last_deep_scrub":"40'148","last_deep_scrub_stamp":"2024-04-26T22:10:29.396793+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.396793+0000","objects_scrubbed":3,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:31.794704+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":136,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"40'4250","reported_seq":4316,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.169415+0000","last_change":"2024-04-26T22:10:28.169480+0000","last_active":"2024-04-26T22:10:28.169415+0000","last_peered":"2024-04-26T22:10:28.169415+0000","last_clean":"2024-04-26T22:10:28.169415+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:10:28.169415+0000","last_undegraded":"2024-04-26T22:10:28.169415+0000","last_fullsized":"2024-04-26T22:10:28.169415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4250","last_scrub_stamp":"2024-04-26T22:10:28.169303+0000","last_deep_scrub":"40'4250","last_deep_scrub_stamp":"2024-04-26T22:10:28.169303+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.169303+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:09.367910+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":64229,"num_write_kb":785523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"40'40","reported_seq":117,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.507144+0000","last_change":"2024-04-26T22:10:29.507176+0000","last_active":"2024-04-26T22:10:29.507144+0000","last_peered":"2024-04-26T22:10:29.507144+0000","last_clean":"2024-04-26T22:10:29.507144+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:10:29.507144+0000","last_undegraded":"2024-04-26T22:10:29.507144+0000","last_fullsized":"2024-04-26T22:10:29.507144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'40","last_scrub_stamp":"2024-04-26T22:10:29.507051+0000","last_deep_scrub":"40'40","last_deep_scrub_stamp":"2024-04-26T22:10:29.507051+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.507051+0000","objects_scrubbed":2,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.651885+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":35,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2966","reported_seq":3035,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.882227+0000","last_change":"2024-04-26T22:10:39.882255+0000","last_active":"2024-04-26T22:10:39.882227+0000","last_peered":"2024-04-26T22:10:39.882227+0000","last_clean":"2024-04-26T22:10:39.882227+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:10:39.882227+0000","last_undegraded":"2024-04-26T22:10:39.882227+0000","last_fullsized":"2024-04-26T22:10:39.882227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'2966","last_scrub_stamp":"2024-04-26T22:10:39.882100+0000","last_deep_scrub":"40'2966","last_deep_scrub_stamp":"2024-04-26T22:10:39.882100+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.882100+0000","objects_scrubbed":0,"log_size":2966,"log_dups_size":0,"ondisk_log_size":2966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:58.297671+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":44534,"num_write_kb":522679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"40'1256","reported_seq":1329,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.034755+0000","last_change":"2024-04-26T22:10:38.034811+0000","last_active":"2024-04-26T22:10:38.034755+0000","last_peered":"2024-04-26T22:10:38.034755+0000","last_clean":"2024-04-26T22:10:38.034755+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:10:38.034755+0000","last_undegraded":"2024-04-26T22:10:38.034755+0000","last_fullsized":"2024-04-26T22:10:38.034755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'1256","last_scrub_stamp":"2024-04-26T22:10:38.034645+0000","last_deep_scrub":"40'1256","last_deep_scrub_stamp":"2024-04-26T22:10:38.034645+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.034645+0000","objects_scrubbed":1,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:06.805058+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":20480,"num_write":1252,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"40'4813","reported_seq":4881,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.809356+0000","last_change":"2024-04-26T22:10:31.809402+0000","last_active":"2024-04-26T22:10:31.809356+0000","last_peered":"2024-04-26T22:10:31.809356+0000","last_clean":"2024-04-26T22:10:31.809356+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:10:31.809356+0000","last_undegraded":"2024-04-26T22:10:31.809356+0000","last_fullsized":"2024-04-26T22:10:31.809356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4813","last_scrub_stamp":"2024-04-26T22:10:31.809248+0000","last_deep_scrub":"40'4813","last_deep_scrub_stamp":"2024-04-26T22:10:31.809248+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.809248+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:05.293746+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":72672,"num_write_kb":866290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"40'90","reported_seq":179,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.039273+0000","last_change":"2024-04-26T22:10:31.039314+0000","last_active":"2024-04-26T22:10:31.039273+0000","last_peered":"2024-04-26T22:10:31.039273+0000","last_clean":"2024-04-26T22:10:31.039273+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:10:31.039273+0000","last_undegraded":"2024-04-26T22:10:31.039273+0000","last_fullsized":"2024-04-26T22:10:31.039273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'90","last_scrub_stamp":"2024-04-26T22:10:31.039155+0000","last_deep_scrub":"40'90","last_deep_scrub_stamp":"2024-04-26T22:10:31.039155+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.039155+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:06.856478+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":62,"num_read_kb":49200,"num_write":70,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"40'3892","reported_seq":3965,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.972669+0000","last_change":"2024-04-26T22:00:06.814149+0000","last_active":"2024-04-26T22:10:11.972669+0000","last_peered":"2024-04-26T22:10:11.972669+0000","last_clean":"2024-04-26T22:10:11.972669+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:10:11.972669+0000","last_undegraded":"2024-04-26T22:10:11.972669+0000","last_fullsized":"2024-04-26T22:10:11.972669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":58573,"num_write_kb":703698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"40'6654","reported_seq":6722,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.435185+0000","last_change":"2024-04-26T22:10:26.435215+0000","last_active":"2024-04-26T22:10:26.435185+0000","last_peered":"2024-04-26T22:10:26.435185+0000","last_clean":"2024-04-26T22:10:26.435185+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:10:26.435185+0000","last_undegraded":"2024-04-26T22:10:26.435185+0000","last_fullsized":"2024-04-26T22:10:26.435185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6654","last_scrub_stamp":"2024-04-26T22:10:26.435070+0000","last_deep_scrub":"40'6654","last_deep_scrub_stamp":"2024-04-26T22:10:26.435070+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.435070+0000","objects_scrubbed":0,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:33.327046+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":100552,"num_write_kb":1253937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"40'4775","reported_seq":4860,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.981653+0000","last_change":"2024-04-26T22:00:06.824484+0000","last_active":"2024-04-26T22:10:11.981653+0000","last_peered":"2024-04-26T22:10:11.981653+0000","last_clean":"2024-04-26T22:10:11.981653+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:10:11.981653+0000","last_undegraded":"2024-04-26T22:10:11.981653+0000","last_fullsized":"2024-04-26T22:10:11.981653+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":71741,"num_write_kb":892635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"40'5861","reported_seq":5927,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.562664+0000","last_change":"2024-04-26T22:10:28.562664+0000","last_active":"2024-04-26T22:10:28.562664+0000","last_peered":"2024-04-26T22:10:28.562664+0000","last_clean":"2024-04-26T22:10:28.562664+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:10:28.562664+0000","last_undegraded":"2024-04-26T22:10:28.562664+0000","last_fullsized":"2024-04-26T22:10:28.562664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5861","last_scrub_stamp":"2024-04-26T22:10:28.562516+0000","last_deep_scrub":"40'5861","last_deep_scrub_stamp":"2024-04-26T22:10:28.562516+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.562516+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:50.631404+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88543,"num_write_kb":1086952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"40'3712","reported_seq":3780,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.989117+0000","last_change":"2024-04-26T22:00:06.828076+0000","last_active":"2024-04-26T22:10:11.989117+0000","last_peered":"2024-04-26T22:10:11.989117+0000","last_clean":"2024-04-26T22:10:11.989117+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:10:11.989117+0000","last_undegraded":"2024-04-26T22:10:11.989117+0000","last_fullsized":"2024-04-26T22:10:11.989117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":55802,"num_write_kb":677006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"40'6511","reported_seq":6576,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.089067+0000","last_change":"2024-04-26T22:00:39.200941+0000","last_active":"2024-04-26T22:10:22.089067+0000","last_peered":"2024-04-26T22:10:22.089067+0000","last_clean":"2024-04-26T22:10:22.089067+0000","last_became_active":"2024-04-26T22:00:39.200592+0000","last_became_peered":"2024-04-26T22:00:39.200592+0000","last_unstale":"2024-04-26T22:10:22.089067+0000","last_undegraded":"2024-04-26T22:10:22.089067+0000","last_fullsized":"2024-04-26T22:10:22.089067+0000","mapping_epoch":25,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":98396,"num_write_kb":1218831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"40'4288","reported_seq":4365,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988267+0000","last_change":"2024-04-26T22:00:06.824598+0000","last_active":"2024-04-26T22:10:11.988267+0000","last_peered":"2024-04-26T22:10:11.988267+0000","last_clean":"2024-04-26T22:10:11.988267+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:10:11.988267+0000","last_undegraded":"2024-04-26T22:10:11.988267+0000","last_fullsized":"2024-04-26T22:10:11.988267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":64735,"num_write_kb":800999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"40'4650","reported_seq":4717,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.121706+0000","last_change":"2024-04-26T22:10:32.121706+0000","last_active":"2024-04-26T22:10:32.121706+0000","last_peered":"2024-04-26T22:10:32.121706+0000","last_clean":"2024-04-26T22:10:32.121706+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:10:32.121706+0000","last_undegraded":"2024-04-26T22:10:32.121706+0000","last_fullsized":"2024-04-26T22:10:32.121706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4650","last_scrub_stamp":"2024-04-26T22:10:32.121543+0000","last_deep_scrub":"40'4650","last_deep_scrub_stamp":"2024-04-26T22:10:32.121543+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.121543+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:57.782607+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":70263,"num_write_kb":869775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"40'5113","reported_seq":5196,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.964863+0000","last_change":"2024-04-26T22:00:06.822567+0000","last_active":"2024-04-26T22:10:22.964863+0000","last_peered":"2024-04-26T22:10:22.964863+0000","last_clean":"2024-04-26T22:10:22.964863+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:10:22.964863+0000","last_undegraded":"2024-04-26T22:10:22.964863+0000","last_fullsized":"2024-04-26T22:10:22.964863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":76970,"num_write_kb":914783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"40'4980","reported_seq":5047,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.029135+0000","last_change":"2024-04-26T22:10:40.029163+0000","last_active":"2024-04-26T22:10:40.029135+0000","last_peered":"2024-04-26T22:10:40.029135+0000","last_clean":"2024-04-26T22:10:40.029135+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:10:40.029135+0000","last_undegraded":"2024-04-26T22:10:40.029135+0000","last_fullsized":"2024-04-26T22:10:40.029135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4980","last_scrub_stamp":"2024-04-26T22:10:40.029039+0000","last_deep_scrub":"40'4980","last_deep_scrub_stamp":"2024-04-26T22:10:40.029039+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.029039+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:33.324013+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":75382,"num_write_kb":914602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"40'3556","reported_seq":3637,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.874839+0000","last_change":"2024-04-26T22:10:37.874865+0000","last_active":"2024-04-26T22:10:37.874839+0000","last_peered":"2024-04-26T22:10:37.874839+0000","last_clean":"2024-04-26T22:10:37.874839+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:10:37.874839+0000","last_undegraded":"2024-04-26T22:10:37.874839+0000","last_fullsized":"2024-04-26T22:10:37.874839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3556","last_scrub_stamp":"2024-04-26T22:10:37.874735+0000","last_deep_scrub":"40'3556","last_deep_scrub_stamp":"2024-04-26T22:10:37.874735+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.874735+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:23.707103+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":53572,"num_write_kb":653683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"40'5738","reported_seq":5804,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.324997+0000","last_change":"2024-04-26T22:10:35.325023+0000","last_active":"2024-04-26T22:10:35.324997+0000","last_peered":"2024-04-26T22:10:35.324997+0000","last_clean":"2024-04-26T22:10:35.324997+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:10:35.324997+0000","last_undegraded":"2024-04-26T22:10:35.324997+0000","last_fullsized":"2024-04-26T22:10:35.324997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5738","last_scrub_stamp":"2024-04-26T22:10:35.324891+0000","last_deep_scrub":"40'5738","last_deep_scrub_stamp":"2024-04-26T22:10:35.324891+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.324891+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:39.958651+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":86675,"num_write_kb":1103247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"40'5159","reported_seq":5248,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.257786+0000","last_change":"2024-04-26T22:10:33.257786+0000","last_active":"2024-04-26T22:10:33.257786+0000","last_peered":"2024-04-26T22:10:33.257786+0000","last_clean":"2024-04-26T22:10:33.257786+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:10:33.257786+0000","last_undegraded":"2024-04-26T22:10:33.257786+0000","last_fullsized":"2024-04-26T22:10:33.257786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5159","last_scrub_stamp":"2024-04-26T22:10:33.257606+0000","last_deep_scrub":"40'5159","last_deep_scrub_stamp":"2024-04-26T22:10:33.257606+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.257606+0000","objects_scrubbed":0,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:19.089974+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":77704,"num_write_kb":942965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"40'4895","reported_seq":4976,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.446815+0000","last_change":"2024-04-26T22:10:25.446854+0000","last_active":"2024-04-26T22:10:25.446815+0000","last_peered":"2024-04-26T22:10:25.446815+0000","last_clean":"2024-04-26T22:10:25.446815+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:10:25.446815+0000","last_undegraded":"2024-04-26T22:10:25.446815+0000","last_fullsized":"2024-04-26T22:10:25.446815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4895","last_scrub_stamp":"2024-04-26T22:10:25.446699+0000","last_deep_scrub":"40'4895","last_deep_scrub_stamp":"2024-04-26T22:10:25.446699+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.446699+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:18.790740+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":363,"num_write":74201,"num_write_kb":908910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"40'4370","reported_seq":4449,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.382324+0000","last_change":"2024-04-26T22:10:40.382362+0000","last_active":"2024-04-26T22:10:40.382324+0000","last_peered":"2024-04-26T22:10:40.382324+0000","last_clean":"2024-04-26T22:10:40.382324+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:10:40.382324+0000","last_undegraded":"2024-04-26T22:10:40.382324+0000","last_fullsized":"2024-04-26T22:10:40.382324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4370","last_scrub_stamp":"2024-04-26T22:10:40.382222+0000","last_deep_scrub":"40'4370","last_deep_scrub_stamp":"2024-04-26T22:10:40.382222+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.382222+0000","objects_scrubbed":0,"log_size":4370,"log_dups_size":0,"ondisk_log_size":4370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.054712+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":65733,"num_write_kb":804896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"40'6465","reported_seq":6534,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.949449+0000","last_change":"2024-04-26T22:10:25.949476+0000","last_active":"2024-04-26T22:10:25.949449+0000","last_peered":"2024-04-26T22:10:25.949449+0000","last_clean":"2024-04-26T22:10:25.949449+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:10:25.949449+0000","last_undegraded":"2024-04-26T22:10:25.949449+0000","last_fullsized":"2024-04-26T22:10:25.949449+0000","mapping_epoch":18,"log_start":"30'1200","ondisk_log_start":"30'1200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6465","last_scrub_stamp":"2024-04-26T22:10:25.949356+0000","last_deep_scrub":"40'6465","last_deep_scrub_stamp":"2024-04-26T22:10:25.949356+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.949356+0000","objects_scrubbed":0,"log_size":5265,"log_dups_size":0,"ondisk_log_size":5265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:59.433854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":97620,"num_write_kb":1192207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"40'3334","reported_seq":3407,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988724+0000","last_change":"2024-04-26T22:00:06.815770+0000","last_active":"2024-04-26T22:10:11.988724+0000","last_peered":"2024-04-26T22:10:11.988724+0000","last_clean":"2024-04-26T22:10:11.988724+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:10:11.988724+0000","last_undegraded":"2024-04-26T22:10:11.988724+0000","last_fullsized":"2024-04-26T22:10:11.988724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":49724,"num_write_kb":588291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"40'4839","reported_seq":4905,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.219008+0000","last_change":"2024-04-26T22:10:30.219051+0000","last_active":"2024-04-26T22:10:30.219008+0000","last_peered":"2024-04-26T22:10:30.219008+0000","last_clean":"2024-04-26T22:10:30.219008+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:10:30.219008+0000","last_undegraded":"2024-04-26T22:10:30.219008+0000","last_fullsized":"2024-04-26T22:10:30.219008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4839","last_scrub_stamp":"2024-04-26T22:10:30.218860+0000","last_deep_scrub":"40'4839","last_deep_scrub_stamp":"2024-04-26T22:10:30.218860+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.218860+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:46.939392+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":73090,"num_write_kb":888426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"40'5009","reported_seq":5080,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.919461+0000","last_change":"2024-04-26T22:00:06.824874+0000","last_active":"2024-04-26T22:10:11.919461+0000","last_peered":"2024-04-26T22:10:11.919461+0000","last_clean":"2024-04-26T22:10:11.919461+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:10:11.919461+0000","last_undegraded":"2024-04-26T22:10:11.919461+0000","last_fullsized":"2024-04-26T22:10:11.919461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":75285,"num_write_kb":916871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"40'5333","reported_seq":5391,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.876227+0000","last_change":"2024-04-26T21:59:57.652724+0000","last_active":"2024-04-26T22:10:05.876227+0000","last_peered":"2024-04-26T22:10:05.876227+0000","last_clean":"2024-04-26T22:10:05.876227+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:10:05.876227+0000","last_undegraded":"2024-04-26T22:10:05.876227+0000","last_fullsized":"2024-04-26T22:10:05.876227+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80596,"num_write_kb":994453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"40'4157","reported_seq":4240,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.386617+0000","last_change":"2024-04-26T22:10:35.386643+0000","last_active":"2024-04-26T22:10:35.386617+0000","last_peered":"2024-04-26T22:10:35.386617+0000","last_clean":"2024-04-26T22:10:35.386617+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:10:35.386617+0000","last_undegraded":"2024-04-26T22:10:35.386617+0000","last_fullsized":"2024-04-26T22:10:35.386617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4157","last_scrub_stamp":"2024-04-26T22:10:35.386499+0000","last_deep_scrub":"40'4157","last_deep_scrub_stamp":"2024-04-26T22:10:35.386499+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.386499+0000","objects_scrubbed":0,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:31.680679+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":62491,"num_write_kb":760147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"40'5861","reported_seq":5913,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.426860+0000","last_change":"2024-04-26T22:10:36.426889+0000","last_active":"2024-04-26T22:10:36.426860+0000","last_peered":"2024-04-26T22:10:36.426860+0000","last_clean":"2024-04-26T22:10:36.426860+0000","last_became_active":"2024-04-26T22:00:38.946980+0000","last_became_peered":"2024-04-26T22:00:38.946980+0000","last_unstale":"2024-04-26T22:10:36.426860+0000","last_undegraded":"2024-04-26T22:10:36.426860+0000","last_fullsized":"2024-04-26T22:10:36.426860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5861","last_scrub_stamp":"2024-04-26T22:10:36.426732+0000","last_deep_scrub":"40'5861","last_deep_scrub_stamp":"2024-04-26T22:10:36.426732+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.426732+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:46.449932+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88698,"num_write_kb":1108811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"40'4156","reported_seq":4229,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.991318+0000","last_change":"2024-04-26T22:00:06.821876+0000","last_active":"2024-04-26T22:10:11.991318+0000","last_peered":"2024-04-26T22:10:11.991318+0000","last_clean":"2024-04-26T22:10:11.991318+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:10:11.991318+0000","last_undegraded":"2024-04-26T22:10:11.991318+0000","last_fullsized":"2024-04-26T22:10:11.991318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62195,"num_write_kb":734457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"40'5569","reported_seq":5638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.136600+0000","last_change":"2024-04-26T22:10:22.136624+0000","last_active":"2024-04-26T22:10:22.136600+0000","last_peered":"2024-04-26T22:10:22.136600+0000","last_clean":"2024-04-26T22:10:22.136600+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:10:22.136600+0000","last_undegraded":"2024-04-26T22:10:22.136600+0000","last_fullsized":"2024-04-26T22:10:22.136600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5569","last_scrub_stamp":"2024-04-26T22:10:22.136506+0000","last_deep_scrub":"40'5569","last_deep_scrub_stamp":"2024-04-26T22:10:22.136506+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.136506+0000","objects_scrubbed":0,"log_size":5569,"log_dups_size":0,"ondisk_log_size":5569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:22.727401+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":83899,"num_write_kb":1024385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"40'4460","reported_seq":4532,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.980639+0000","last_change":"2024-04-26T22:00:06.818235+0000","last_active":"2024-04-26T22:10:11.980639+0000","last_peered":"2024-04-26T22:10:11.980639+0000","last_clean":"2024-04-26T22:10:11.980639+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:10:11.980639+0000","last_undegraded":"2024-04-26T22:10:11.980639+0000","last_fullsized":"2024-04-26T22:10:11.980639+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":67080,"num_write_kb":827171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"40'5284","reported_seq":5351,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.333765+0000","last_change":"2024-04-26T22:10:34.333790+0000","last_active":"2024-04-26T22:10:34.333765+0000","last_peered":"2024-04-26T22:10:34.333765+0000","last_clean":"2024-04-26T22:10:34.333765+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:10:34.333765+0000","last_undegraded":"2024-04-26T22:10:34.333765+0000","last_fullsized":"2024-04-26T22:10:34.333765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5284","last_scrub_stamp":"2024-04-26T22:10:34.333648+0000","last_deep_scrub":"40'5284","last_deep_scrub_stamp":"2024-04-26T22:10:34.333648+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.333648+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:30.627828+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":79924,"num_write_kb":1000899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"40'4361","reported_seq":4450,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.884600+0000","last_change":"2024-04-26T22:10:38.884652+0000","last_active":"2024-04-26T22:10:38.884600+0000","last_peered":"2024-04-26T22:10:38.884600+0000","last_clean":"2024-04-26T22:10:38.884600+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:10:38.884600+0000","last_undegraded":"2024-04-26T22:10:38.884600+0000","last_fullsized":"2024-04-26T22:10:38.884600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4361","last_scrub_stamp":"2024-04-26T22:10:38.884458+0000","last_deep_scrub":"40'4361","last_deep_scrub_stamp":"2024-04-26T22:10:38.884458+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.884458+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:11.214750+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65464,"num_write_kb":794873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"40'5458","reported_seq":5525,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.302249+0000","last_change":"2024-04-26T22:10:32.302278+0000","last_active":"2024-04-26T22:10:32.302249+0000","last_peered":"2024-04-26T22:10:32.302249+0000","last_clean":"2024-04-26T22:10:32.302249+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:10:32.302249+0000","last_undegraded":"2024-04-26T22:10:32.302249+0000","last_fullsized":"2024-04-26T22:10:32.302249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5458","last_scrub_stamp":"2024-04-26T22:10:32.302152+0000","last_deep_scrub":"40'5458","last_deep_scrub_stamp":"2024-04-26T22:10:32.302152+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.302152+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:08.119475+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":82585,"num_write_kb":1000929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"40'3624","reported_seq":3699,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.235149+0000","last_change":"2024-04-26T22:10:27.235175+0000","last_active":"2024-04-26T22:10:27.235149+0000","last_peered":"2024-04-26T22:10:27.235149+0000","last_clean":"2024-04-26T22:10:27.235149+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:10:27.235149+0000","last_undegraded":"2024-04-26T22:10:27.235149+0000","last_fullsized":"2024-04-26T22:10:27.235149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3624","last_scrub_stamp":"2024-04-26T22:10:27.235043+0000","last_deep_scrub":"40'3624","last_deep_scrub_stamp":"2024-04-26T22:10:27.235043+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.235043+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:00.719541+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":54495,"num_write_kb":662506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"40'4431","reported_seq":4504,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.594748+0000","last_change":"2024-04-26T22:10:27.594797+0000","last_active":"2024-04-26T22:10:27.594748+0000","last_peered":"2024-04-26T22:10:27.594748+0000","last_clean":"2024-04-26T22:10:27.594748+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:10:27.594748+0000","last_undegraded":"2024-04-26T22:10:27.594748+0000","last_fullsized":"2024-04-26T22:10:27.594748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4431","last_scrub_stamp":"2024-04-26T22:10:27.594561+0000","last_deep_scrub":"40'4431","last_deep_scrub_stamp":"2024-04-26T22:10:27.594561+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.594561+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:40.687251+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":66582,"num_write_kb":814643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"40'4089","reported_seq":4174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.319106+0000","last_change":"2024-04-26T22:10:40.319135+0000","last_active":"2024-04-26T22:10:40.319106+0000","last_peered":"2024-04-26T22:10:40.319106+0000","last_clean":"2024-04-26T22:10:40.319106+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:10:40.319106+0000","last_undegraded":"2024-04-26T22:10:40.319106+0000","last_fullsized":"2024-04-26T22:10:40.319106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4089","last_scrub_stamp":"2024-04-26T22:10:40.318986+0000","last_deep_scrub":"40'4089","last_deep_scrub_stamp":"2024-04-26T22:10:40.318986+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.318986+0000","objects_scrubbed":0,"log_size":4089,"log_dups_size":0,"ondisk_log_size":4089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:44.952746+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61370,"num_write_kb":735117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"40'6428","reported_seq":6496,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.245094+0000","last_change":"2024-04-26T22:10:26.245119+0000","last_active":"2024-04-26T22:10:26.245094+0000","last_peered":"2024-04-26T22:10:26.245094+0000","last_clean":"2024-04-26T22:10:26.245094+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:10:26.245094+0000","last_undegraded":"2024-04-26T22:10:26.245094+0000","last_fullsized":"2024-04-26T22:10:26.245094+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6428","last_scrub_stamp":"2024-04-26T22:10:26.244996+0000","last_deep_scrub":"40'6428","last_deep_scrub_stamp":"2024-04-26T22:10:26.244996+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.244996+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:36.268196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":97094,"num_write_kb":1219735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"40'4506","reported_seq":4597,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.384356+0000","last_change":"2024-04-26T22:10:38.384380+0000","last_active":"2024-04-26T22:10:38.384356+0000","last_peered":"2024-04-26T22:10:38.384356+0000","last_clean":"2024-04-26T22:10:38.384356+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:10:38.384356+0000","last_undegraded":"2024-04-26T22:10:38.384356+0000","last_fullsized":"2024-04-26T22:10:38.384356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4506","last_scrub_stamp":"2024-04-26T22:10:38.384255+0000","last_deep_scrub":"40'4506","last_deep_scrub_stamp":"2024-04-26T22:10:38.384255+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.384255+0000","objects_scrubbed":0,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:56.799825+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":67596,"num_write_kb":862862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"40'4747","reported_seq":4814,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.266486+0000","last_change":"2024-04-26T22:10:24.266514+0000","last_active":"2024-04-26T22:10:24.266486+0000","last_peered":"2024-04-26T22:10:24.266486+0000","last_clean":"2024-04-26T22:10:24.266486+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:10:24.266486+0000","last_undegraded":"2024-04-26T22:10:24.266486+0000","last_fullsized":"2024-04-26T22:10:24.266486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4747","last_scrub_stamp":"2024-04-26T22:10:24.266388+0000","last_deep_scrub":"40'4747","last_deep_scrub_stamp":"2024-04-26T22:10:24.266388+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.266388+0000","objects_scrubbed":0,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:41.401313+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71795,"num_write_kb":917779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"40'4901","reported_seq":4977,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.988219+0000","last_change":"2024-04-26T22:00:06.822413+0000","last_active":"2024-04-26T22:10:11.988219+0000","last_peered":"2024-04-26T22:10:11.988219+0000","last_clean":"2024-04-26T22:10:11.988219+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:10:11.988219+0000","last_undegraded":"2024-04-26T22:10:11.988219+0000","last_fullsized":"2024-04-26T22:10:11.988219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4901,"log_dups_size":0,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":73795,"num_write_kb":926135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"40'5581","reported_seq":5648,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.353780+0000","last_change":"2024-04-26T22:10:36.353808+0000","last_active":"2024-04-26T22:10:36.353780+0000","last_peered":"2024-04-26T22:10:36.353780+0000","last_clean":"2024-04-26T22:10:36.353780+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:10:36.353780+0000","last_undegraded":"2024-04-26T22:10:36.353780+0000","last_fullsized":"2024-04-26T22:10:36.353780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5581","last_scrub_stamp":"2024-04-26T22:10:36.353677+0000","last_deep_scrub":"40'5581","last_deep_scrub_stamp":"2024-04-26T22:10:36.353677+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.353677+0000","objects_scrubbed":0,"log_size":5581,"log_dups_size":0,"ondisk_log_size":5581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.186045+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":84297,"num_write_kb":1026829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"40'3600","reported_seq":3672,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.990406+0000","last_change":"2024-04-26T22:00:06.832631+0000","last_active":"2024-04-26T22:10:11.990406+0000","last_peered":"2024-04-26T22:10:11.990406+0000","last_clean":"2024-04-26T22:10:11.990406+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:10:11.990406+0000","last_undegraded":"2024-04-26T22:10:11.990406+0000","last_fullsized":"2024-04-26T22:10:11.990406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":54091,"num_write_kb":649095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"40'5612","reported_seq":5680,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.152606+0000","last_change":"2024-04-26T22:10:30.152659+0000","last_active":"2024-04-26T22:10:30.152606+0000","last_peered":"2024-04-26T22:10:30.152606+0000","last_clean":"2024-04-26T22:10:30.152606+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:10:30.152606+0000","last_undegraded":"2024-04-26T22:10:30.152606+0000","last_fullsized":"2024-04-26T22:10:30.152606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5612","last_scrub_stamp":"2024-04-26T22:10:30.152464+0000","last_deep_scrub":"40'5612","last_deep_scrub_stamp":"2024-04-26T22:10:30.152464+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.152464+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:16.237743+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":84765,"num_write_kb":1047612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"40'3974","reported_seq":4054,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.883390+0000","last_change":"2024-04-26T22:00:06.823395+0000","last_active":"2024-04-26T22:10:11.883390+0000","last_peered":"2024-04-26T22:10:11.883390+0000","last_clean":"2024-04-26T22:10:11.883390+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:10:11.883390+0000","last_undegraded":"2024-04-26T22:10:11.883390+0000","last_fullsized":"2024-04-26T22:10:11.883390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3974,"log_dups_size":0,"ondisk_log_size":3974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":59852,"num_write_kb":718261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"40'4819","reported_seq":4890,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.027511+0000","last_change":"2024-04-26T22:10:36.027563+0000","last_active":"2024-04-26T22:10:36.027511+0000","last_peered":"2024-04-26T22:10:36.027511+0000","last_clean":"2024-04-26T22:10:36.027511+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:10:36.027511+0000","last_undegraded":"2024-04-26T22:10:36.027511+0000","last_fullsized":"2024-04-26T22:10:36.027511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4819","last_scrub_stamp":"2024-04-26T22:10:36.027399+0000","last_deep_scrub":"40'4819","last_deep_scrub_stamp":"2024-04-26T22:10:36.027399+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.027399+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:25.329571+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":72748,"num_write_kb":932622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"40'4140","reported_seq":4221,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.991202+0000","last_change":"2024-04-26T22:00:06.815355+0000","last_active":"2024-04-26T22:10:11.991202+0000","last_peered":"2024-04-26T22:10:11.991202+0000","last_clean":"2024-04-26T22:10:11.991202+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:10:11.991202+0000","last_undegraded":"2024-04-26T22:10:11.991202+0000","last_fullsized":"2024-04-26T22:10:11.991202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":62174,"num_write_kb":775596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"40'4503","reported_seq":4571,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.829753+0000","last_change":"2024-04-26T22:10:29.829807+0000","last_active":"2024-04-26T22:10:29.829753+0000","last_peered":"2024-04-26T22:10:29.829753+0000","last_clean":"2024-04-26T22:10:29.829753+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:10:29.829753+0000","last_undegraded":"2024-04-26T22:10:29.829753+0000","last_fullsized":"2024-04-26T22:10:29.829753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4503","last_scrub_stamp":"2024-04-26T22:10:29.829629+0000","last_deep_scrub":"40'4503","last_deep_scrub_stamp":"2024-04-26T22:10:29.829629+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.829629+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:26.123809+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":68209,"num_write_kb":848035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"40'3768","reported_seq":3857,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.462225+0000","last_change":"2024-04-26T22:10:21.462249+0000","last_active":"2024-04-26T22:10:21.462225+0000","last_peered":"2024-04-26T22:10:21.462225+0000","last_clean":"2024-04-26T22:10:21.462225+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:10:21.462225+0000","last_undegraded":"2024-04-26T22:10:21.462225+0000","last_fullsized":"2024-04-26T22:10:21.462225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3768","last_scrub_stamp":"2024-04-26T22:10:21.462125+0000","last_deep_scrub":"40'3768","last_deep_scrub_stamp":"2024-04-26T22:10:21.462125+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.462125+0000","objects_scrubbed":0,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:39.714111+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":56583,"num_write_kb":708038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"40'5303","reported_seq":5368,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.423568+0000","last_change":"2024-04-26T22:10:24.423594+0000","last_active":"2024-04-26T22:10:24.423568+0000","last_peered":"2024-04-26T22:10:24.423568+0000","last_clean":"2024-04-26T22:10:24.423568+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:10:24.423568+0000","last_undegraded":"2024-04-26T22:10:24.423568+0000","last_fullsized":"2024-04-26T22:10:24.423568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5303","last_scrub_stamp":"2024-04-26T22:10:24.423454+0000","last_deep_scrub":"40'5303","last_deep_scrub_stamp":"2024-04-26T22:10:24.423454+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.423454+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:25.014349+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80229,"num_write_kb":963753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"40'3667","reported_seq":3754,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.266169+0000","last_change":"2024-04-26T22:10:31.266196+0000","last_active":"2024-04-26T22:10:31.266169+0000","last_peered":"2024-04-26T22:10:31.266169+0000","last_clean":"2024-04-26T22:10:31.266169+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:10:31.266169+0000","last_undegraded":"2024-04-26T22:10:31.266169+0000","last_fullsized":"2024-04-26T22:10:31.266169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3667","last_scrub_stamp":"2024-04-26T22:10:31.266071+0000","last_deep_scrub":"40'3667","last_deep_scrub_stamp":"2024-04-26T22:10:31.266071+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.266071+0000","objects_scrubbed":0,"log_size":3667,"log_dups_size":0,"ondisk_log_size":3667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:05.340055+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":55047,"num_write_kb":687611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"40'6069","reported_seq":6137,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.459956+0000","last_change":"2024-04-26T22:10:23.459996+0000","last_active":"2024-04-26T22:10:23.459956+0000","last_peered":"2024-04-26T22:10:23.459956+0000","last_clean":"2024-04-26T22:10:23.459956+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:10:23.459956+0000","last_undegraded":"2024-04-26T22:10:23.459956+0000","last_fullsized":"2024-04-26T22:10:23.459956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6069","last_scrub_stamp":"2024-04-26T22:10:23.459828+0000","last_deep_scrub":"40'6069","last_deep_scrub_stamp":"2024-04-26T22:10:23.459828+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.459828+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:35.043456+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":91606,"num_write_kb":1169870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"40'4173","reported_seq":4261,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.232685+0000","last_change":"2024-04-26T22:10:29.232712+0000","last_active":"2024-04-26T22:10:29.232685+0000","last_peered":"2024-04-26T22:10:29.232685+0000","last_clean":"2024-04-26T22:10:29.232685+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:10:29.232685+0000","last_undegraded":"2024-04-26T22:10:29.232685+0000","last_fullsized":"2024-04-26T22:10:29.232685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4173","last_scrub_stamp":"2024-04-26T22:10:29.232579+0000","last_deep_scrub":"40'4173","last_deep_scrub_stamp":"2024-04-26T22:10:29.232579+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.232579+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:28.145004+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":62684,"num_write_kb":755589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"40'6512","reported_seq":6575,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.875653+0000","last_change":"2024-04-26T22:00:39.201110+0000","last_active":"2024-04-26T22:10:05.875653+0000","last_peered":"2024-04-26T22:10:05.875653+0000","last_clean":"2024-04-26T22:10:05.875653+0000","last_became_active":"2024-04-26T22:00:39.200691+0000","last_became_peered":"2024-04-26T22:00:39.200691+0000","last_unstale":"2024-04-26T22:10:05.875653+0000","last_undegraded":"2024-04-26T22:10:05.875653+0000","last_fullsized":"2024-04-26T22:10:05.875653+0000","mapping_epoch":25,"log_start":"29'700","ondisk_log_start":"29'700","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":98505,"num_write_kb":1266779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"40'5161","reported_seq":5249,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.216489+0000","last_change":"2024-04-26T22:00:06.814345+0000","last_active":"2024-04-26T22:10:21.216489+0000","last_peered":"2024-04-26T22:10:21.216489+0000","last_clean":"2024-04-26T22:10:21.216489+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:10:21.216489+0000","last_undegraded":"2024-04-26T22:10:21.216489+0000","last_fullsized":"2024-04-26T22:10:21.216489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":77820,"num_write_kb":957388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"40'5846","reported_seq":5904,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:05.776723+0000","last_change":"2024-04-26T21:59:57.648543+0000","last_active":"2024-04-26T22:10:05.776723+0000","last_peered":"2024-04-26T22:10:05.776723+0000","last_clean":"2024-04-26T22:10:05.776723+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:10:05.776723+0000","last_undegraded":"2024-04-26T22:10:05.776723+0000","last_fullsized":"2024-04-26T22:10:05.776723+0000","mapping_epoch":18,"log_start":"29'600","ondisk_log_start":"29'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88593,"num_write_kb":1072922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"40'3484","reported_seq":3554,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.528015+0000","last_change":"2024-04-26T22:00:06.822536+0000","last_active":"2024-04-26T22:10:22.528015+0000","last_peered":"2024-04-26T22:10:22.528015+0000","last_clean":"2024-04-26T22:10:22.528015+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:10:22.528015+0000","last_undegraded":"2024-04-26T22:10:22.528015+0000","last_fullsized":"2024-04-26T22:10:22.528015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3484,"log_dups_size":0,"ondisk_log_size":3484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":52218,"num_write_kb":635931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"40'5213","reported_seq":5282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.212704+0000","last_change":"2024-04-26T22:10:26.212754+0000","last_active":"2024-04-26T22:10:26.212704+0000","last_peered":"2024-04-26T22:10:26.212704+0000","last_clean":"2024-04-26T22:10:26.212704+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:10:26.212704+0000","last_undegraded":"2024-04-26T22:10:26.212704+0000","last_fullsized":"2024-04-26T22:10:26.212704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5213","last_scrub_stamp":"2024-04-26T22:10:26.212563+0000","last_deep_scrub":"40'5213","last_deep_scrub_stamp":"2024-04-26T22:10:26.212563+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.212563+0000","objects_scrubbed":0,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:59.766447+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":78898,"num_write_kb":987567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"40'3779","reported_seq":3867,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.377643+0000","last_change":"2024-04-26T22:10:39.377705+0000","last_active":"2024-04-26T22:10:39.377643+0000","last_peered":"2024-04-26T22:10:39.377643+0000","last_clean":"2024-04-26T22:10:39.377643+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:10:39.377643+0000","last_undegraded":"2024-04-26T22:10:39.377643+0000","last_fullsized":"2024-04-26T22:10:39.377643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3779","last_scrub_stamp":"2024-04-26T22:10:39.377496+0000","last_deep_scrub":"40'3779","last_deep_scrub_stamp":"2024-04-26T22:10:39.377496+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.377496+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:42.207197+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56904,"num_write_kb":701117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"40'4635","reported_seq":4701,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.204396+0000","last_change":"2024-04-26T22:10:22.204423+0000","last_active":"2024-04-26T22:10:22.204396+0000","last_peered":"2024-04-26T22:10:22.204396+0000","last_clean":"2024-04-26T22:10:22.204396+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:10:22.204396+0000","last_undegraded":"2024-04-26T22:10:22.204396+0000","last_fullsized":"2024-04-26T22:10:22.204396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4635","last_scrub_stamp":"2024-04-26T22:10:22.204275+0000","last_deep_scrub":"40'4635","last_deep_scrub_stamp":"2024-04-26T22:10:22.204275+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.204275+0000","objects_scrubbed":0,"log_size":4635,"log_dups_size":0,"ondisk_log_size":4635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:08.289555+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":70184,"num_write_kb":869081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"40'4125","reported_seq":4199,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.089456+0000","last_change":"2024-04-26T22:00:06.831864+0000","last_active":"2024-04-26T22:10:22.089456+0000","last_peered":"2024-04-26T22:10:22.089456+0000","last_clean":"2024-04-26T22:10:22.089456+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:10:22.089456+0000","last_undegraded":"2024-04-26T22:10:22.089456+0000","last_fullsized":"2024-04-26T22:10:22.089456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62115,"num_write_kb":743778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"40'6598","reported_seq":6662,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.110544+0000","last_change":"2024-04-26T21:59:57.651851+0000","last_active":"2024-04-26T22:10:10.110544+0000","last_peered":"2024-04-26T22:10:10.110544+0000","last_clean":"2024-04-26T22:10:10.110544+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:10:10.110544+0000","last_undegraded":"2024-04-26T22:10:10.110544+0000","last_fullsized":"2024-04-26T22:10:10.110544+0000","mapping_epoch":18,"log_start":"29'800","ondisk_log_start":"29'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":99507,"num_write_kb":1174132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"40'3584","reported_seq":3654,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:15.358630+0000","last_change":"2024-04-26T22:00:06.827081+0000","last_active":"2024-04-26T22:10:15.358630+0000","last_peered":"2024-04-26T22:10:15.358630+0000","last_clean":"2024-04-26T22:10:15.358630+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:10:15.358630+0000","last_undegraded":"2024-04-26T22:10:15.358630+0000","last_fullsized":"2024-04-26T22:10:15.358630+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3584,"log_dups_size":0,"ondisk_log_size":3584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":54000,"num_write_kb":647349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"40'4854","reported_seq":4912,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:10.150078+0000","last_change":"2024-04-26T21:59:57.649033+0000","last_active":"2024-04-26T22:10:10.150078+0000","last_peered":"2024-04-26T22:10:10.150078+0000","last_clean":"2024-04-26T22:10:10.150078+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:10:10.150078+0000","last_undegraded":"2024-04-26T22:10:10.150078+0000","last_fullsized":"2024-04-26T22:10:10.150078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:59:56.622261+0000","last_clean_scrub_stamp":"2024-04-26T21:59:56.622261+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73549,"num_write_kb":940613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"40'4744","reported_seq":4842,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.378872+0000","last_change":"2024-04-26T22:10:32.378912+0000","last_active":"2024-04-26T22:10:32.378872+0000","last_peered":"2024-04-26T22:10:32.378872+0000","last_clean":"2024-04-26T22:10:32.378872+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:10:32.378872+0000","last_undegraded":"2024-04-26T22:10:32.378872+0000","last_fullsized":"2024-04-26T22:10:32.378872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4744","last_scrub_stamp":"2024-04-26T22:10:32.378741+0000","last_deep_scrub":"40'4744","last_deep_scrub_stamp":"2024-04-26T22:10:32.378741+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.378741+0000","objects_scrubbed":0,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:18.464020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":71239,"num_write_kb":843376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"40'5453","reported_seq":5521,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:20.152937+0000","last_change":"2024-04-26T22:10:20.152937+0000","last_active":"2024-04-26T22:10:20.152937+0000","last_peered":"2024-04-26T22:10:20.152937+0000","last_clean":"2024-04-26T22:10:20.152937+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:10:20.152937+0000","last_undegraded":"2024-04-26T22:10:20.152937+0000","last_fullsized":"2024-04-26T22:10:20.152937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5453","last_scrub_stamp":"2024-04-26T22:10:20.152763+0000","last_deep_scrub":"40'5453","last_deep_scrub_stamp":"2024-04-26T22:10:20.152763+0000","last_clean_scrub_stamp":"2024-04-26T22:10:20.152763+0000","objects_scrubbed":0,"log_size":5453,"log_dups_size":0,"ondisk_log_size":5453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:16.392610+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":82428,"num_write_kb":1036078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"40'4470","reported_seq":4557,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.187484+0000","last_change":"2024-04-26T22:10:27.187511+0000","last_active":"2024-04-26T22:10:27.187484+0000","last_peered":"2024-04-26T22:10:27.187484+0000","last_clean":"2024-04-26T22:10:27.187484+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:10:27.187484+0000","last_undegraded":"2024-04-26T22:10:27.187484+0000","last_fullsized":"2024-04-26T22:10:27.187484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4470","last_scrub_stamp":"2024-04-26T22:10:27.187360+0000","last_deep_scrub":"40'4470","last_deep_scrub_stamp":"2024-04-26T22:10:27.187360+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.187360+0000","objects_scrubbed":0,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:43.671876+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":67360,"num_write_kb":793318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"40'6816","reported_seq":6885,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.462659+0000","last_change":"2024-04-26T22:10:22.462659+0000","last_active":"2024-04-26T22:10:22.462659+0000","last_peered":"2024-04-26T22:10:22.462659+0000","last_clean":"2024-04-26T22:10:22.462659+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:10:22.462659+0000","last_undegraded":"2024-04-26T22:10:22.462659+0000","last_fullsized":"2024-04-26T22:10:22.462659+0000","mapping_epoch":18,"log_start":"29'1100","ondisk_log_start":"29'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6816","last_scrub_stamp":"2024-04-26T22:10:22.462544+0000","last_deep_scrub":"40'6816","last_deep_scrub_stamp":"2024-04-26T22:10:22.462544+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.462544+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:24.860454+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":103074,"num_write_kb":1294412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"40'3980","reported_seq":4054,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:11.986619+0000","last_change":"2024-04-26T22:00:06.824238+0000","last_active":"2024-04-26T22:10:11.986619+0000","last_peered":"2024-04-26T22:10:11.986619+0000","last_clean":"2024-04-26T22:10:11.986619+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:10:11.986619+0000","last_undegraded":"2024-04-26T22:10:11.986619+0000","last_fullsized":"2024-04-26T22:10:11.986619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:00:05.791883+0000","last_clean_scrub_stamp":"2024-04-26T22:00:05.791883+0000","objects_scrubbed":0,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":59871,"num_write_kb":728175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"40'4761","reported_seq":4831,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.234316+0000","last_change":"2024-04-26T22:10:34.234342+0000","last_active":"2024-04-26T22:10:34.234316+0000","last_peered":"2024-04-26T22:10:34.234316+0000","last_clean":"2024-04-26T22:10:34.234316+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:10:34.234316+0000","last_undegraded":"2024-04-26T22:10:34.234316+0000","last_fullsized":"2024-04-26T22:10:34.234316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4761","last_scrub_stamp":"2024-04-26T22:10:34.234178+0000","last_deep_scrub":"40'4761","last_deep_scrub_stamp":"2024-04-26T22:10:34.234178+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.234178+0000","objects_scrubbed":0,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:06.564470+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":72121,"num_write_kb":895283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1800,"num_read_kb":0,"num_write":3978586,"num_write_kb":48381196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264479,"ondisk_log_size":264479,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":109332168,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":691,"num_read_kb":381190,"num_write":3546,"num_write_kb":109340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218726400,"data_stored":218664336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3537,"ondisk_log_size":3537,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":363,"num_write":5266740,"num_write_kb":65270508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338431,"ondisk_log_size":338431,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":143059102,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":553,"num_read_kb":447893,"num_write":4271,"num_write_kb":143037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286179328,"data_stored":286118204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4286,"ondisk_log_size":4286,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529688,"kb_used_data":21200,"kb_used_omap":1,"kb_used_meta":508478,"kb_avail":93842152,"statfs":{"total":96636764160,"available":96094363648,"internally_reserved":0,"allocated":21708800,"data_stored":21470005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":520681925},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848624,"kb_used_data":66920,"kb_used_omap":1,"kb_used_meta":781694,"kb_avail":93523216,"statfs":{"total":96636764160,"available":95767773184,"internally_reserved":0,"allocated":68526080,"data_stored":68282539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":800455114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574352,"kb_used_data":74184,"kb_used_omap":1,"kb_used_meta":500158,"kb_avail":93797488,"statfs":{"total":96636764160,"available":96048627712,"internally_reserved":0,"allocated":75964416,"data_stored":75718280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":512162246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794020,"kb_used_data":45596,"kb_used_omap":1,"kb_used_meta":748414,"kb_avail":93577820,"statfs":{"total":96636764160,"available":95823687680,"internally_reserved":0,"allocated":46690304,"data_stored":46445706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":766376389},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882668,"kb_used_data":82468,"kb_used_omap":1,"kb_used_meta":800190,"kb_avail":93489172,"statfs":{"total":96636764160,"available":95732912128,"internally_reserved":0,"allocated":84447232,"data_stored":84193910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":819395014},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900648,"kb_used_data":78560,"kb_used_omap":1,"kb_used_meta":822078,"kb_avail":93471192,"statfs":{"total":96636764160,"available":95714500608,"internally_reserved":0,"allocated":80445440,"data_stored":80191557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":841808327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767416,"kb_used_data":86832,"kb_used_omap":1,"kb_used_meta":680574,"kb_avail":93604424,"statfs":{"total":96636764160,"available":95850930176,"internally_reserved":0,"allocated":88915968,"data_stored":88663873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":696908229},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740368,"kb_used_data":41992,"kb_used_omap":1,"kb_used_meta":698366,"kb_avail":93631472,"statfs":{"total":96636764160,"available":95878627328,"internally_reserved":0,"allocated":42999808,"data_stored":42753566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715127237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29372416,"data_stored":29360766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":38112011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12673024,"data_stored":12671093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33652736,"data_stored":33643285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29364224,"data_stored":29360218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":46145536,"data_stored":46138016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50700288,"data_stored":50695345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8585216,"data_stored":8579392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46342144,"data_stored":46328752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37847040,"data_stored":37836920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16875520,"data_stored":16865968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:10:43.704 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.705 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-26T21:59:55.618144+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=55, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.706 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.706 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.706 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.707 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.708 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.708 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.708 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.708 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.708 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.708 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.709 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.710 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.711 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.711 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-26T21:59:56.622261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=59, tm_sec=56, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.711 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-04-26T22:00:05.791883+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=22, tm_min=10, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T22:10:43.711 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T22:11:01.750 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:11:01.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:11:03.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:11:03.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.821+0000 7efc755ac640 1 Processor -- start 2024-04-26T22:11:03.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.821+0000 7efc755ac640 1 -- start start 2024-04-26T22:11:03.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.821+0000 7efc755ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 0x7efc700c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.821+0000 7efc755ac640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7015a530 con 0x7efc700740e0 2024-04-26T22:11:03.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.821+0000 7efc755ac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7015a810 con 0x7efc700ca3a0 2024-04-26T22:11:03.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.821+0000 7efc755ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc70072930 con 0x7efc700c6e20 2024-04-26T22:11:03.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6e7fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7efc700740e0 0x7efc70156f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57632/0 (socket says 172.21.15.135:57632) 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6e7fc640 1 -- 172.21.15.135:0/195402258 learned_addr learned my addr 172.21.15.135:0/195402258 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6effd640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 0x7efc700c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267730497 0 0) 0x7efc7015a530 con 0x7efc700740e0 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc58003520 con 0x7efc700740e0 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 659089058 0 0) 0x7efc58003520 con 0x7efc700740e0 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 msgr2=0x7efc700c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 0x7efc700c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 >> v1:172.21.15.139:6789/0 conn(0x7efc700ca3a0 legacy=0x7efc70161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7015aaf0 con 0x7efc700740e0 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7efc64001670 con 0x7efc700740e0 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 >> v1:172.21.15.135:6789/0 conn(0x7efc700740e0 legacy=0x7efc70156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc580039e0 0x7efc58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 0x7efc58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc58004d40 0x7efc580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800ba70 con 0x7efc58004d40 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc5800bbf0 con 0x7efc580039e0 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc5800bdf0 con 0x7efc700c6e20 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6f7fe640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc58004d40 0x7efc580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6effd640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc580039e0 0x7efc58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.822+0000 7efc6e7fc640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 0x7efc58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6f7fe640 1 -- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 msgr2=0x7efc58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6f7fe640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc700c6e20 0x7efc58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6f7fe640 1 -- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc580039e0 msgr2=0x7efc58003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6f7fe640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc580039e0 0x7efc58003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6f7fe640 1 -- 172.21.15.135:0/195402258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc58003520 con 0x7efc58004d40 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6f7fe640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc58004d40 0x7efc580051c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efc68001f60 tx=0x7efc58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1c022a0b610b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68001b30 con 0x7efc58004d40 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc68031620 con 0x7efc58004d40 2024-04-26T22:11:03.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.823+0000 7efc6dffb640 1 -- 172.21.15.135:0/195402258 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68031a10 con 0x7efc58004d40 2024-04-26T22:11:03.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc755ac640 1 -- 172.21.15.135:0/195402258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc58004d40 msgr2=0x7efc580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc755ac640 1 --2- 172.21.15.135:0/195402258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc58004d40 0x7efc580051c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efc68001f60 tx=0x7efc58005830 comp rx=0 tx=0).stop 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc6f7fe640 1 -- 172.21.15.135:0/195402258 reap_dead start 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc755ac640 1 -- 172.21.15.135:0/195402258 shutdown_connections 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc755ac640 1 -- 172.21.15.135:0/195402258 >> 172.21.15.135:0/195402258 conn(0x7efc700ae0b0 msgr2=0x7efc700ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc755ac640 1 -- 172.21.15.135:0/195402258 shutdown_connections 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.824+0000 7efc755ac640 1 -- 172.21.15.135:0/195402258 wait complete. 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.825+0000 7efc755ac640 1 Processor -- start 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.825+0000 7efc755ac640 1 -- start start 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc755ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc70142850 0x7efc701ed210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc755ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 0x7efc70153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc755ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc701ed750 0x7efc701f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc755ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc70154540 con 0x7efc70155f40 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc755ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc701543c0 con 0x7efc70142850 2024-04-26T22:11:03.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc755ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc7015a0c0 con 0x7efc701ed750 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 0x7efc70153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 0x7efc70153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39274/0 (socket says 172.21.15.135:39274) 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 -- 172.21.15.135:0/3441377691 learned_addr learned my addr 172.21.15.135:0/3441377691 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6effd640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc70142850 0x7efc701ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6f7fe640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc701ed750 0x7efc701f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 -- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc701ed750 msgr2=0x7efc701f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc701ed750 0x7efc701f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 -- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc70142850 msgr2=0x7efc701ed210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc70142850 0x7efc701ed210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.826+0000 7efc6e7fc640 1 -- 172.21.15.135:0/3441377691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc70156440 con 0x7efc70155f40 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.827+0000 7efc6e7fc640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 0x7efc70153490 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efc64007fc0 tx=0x7efc640074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.827+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64007740 con 0x7efc70155f40 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.827+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc64018070 con 0x7efc70155f40 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.827+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc70156720 con 0x7efc70155f40 2024-04-26T22:11:03.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.827+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6400fba0 con 0x7efc70155f40 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.827+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701eccf0 con 0x7efc70155f40 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.828+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc701ecef0 con 0x7efc70155f40 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.828+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc6400fd40 con 0x7efc70155f40 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.829+0000 7efc577fe640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc5c06cac0 0x7efc5c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.829+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7efc6408c370 con 0x7efc70155f40 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.829+0000 7efc6effd640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc5c06cac0 0x7efc5c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:03.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.829+0000 7efc6effd640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc5c06cac0 0x7efc5c06ef80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efc60009b00 tx=0x7efc60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:03.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:03.833+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc701ecef0 con 0x7efc70155f40 2024-04-26T22:11:04.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.011+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 --> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc701ecae0 con 0x7efc5c06cac0 2024-04-26T22:11:04.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.022+0000 7efc577fe640 1 -- 172.21.15.135:0/3441377691 <== mgr.4104 v2:172.21.15.135:6832/3236798717 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495925 (secure 0 0 0) 0x7efc701ecae0 con 0x7efc5c06cac0 2024-04-26T22:11:04.028 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:04.032 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-26T22:11:04.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc5c06cac0 msgr2=0x7efc5c06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc5c06cac0 0x7efc5c06ef80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efc60009b00 tx=0x7efc60008040 comp rx=0 tx=0).stop 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 msgr2=0x7efc70153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 0x7efc70153490 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efc64007fc0 tx=0x7efc640074f0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 shutdown_connections 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7efc5c06cac0 0x7efc5c06ef80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc701ed750 0x7efc701f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70155f40 0x7efc70153490 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 --2- 172.21.15.135:0/3441377691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc70142850 0x7efc701ed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 >> 172.21.15.135:0/3441377691 conn(0x7efc700ae0b0 msgr2=0x7efc700c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 shutdown_connections 2024-04-26T22:11:04.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.026+0000 7efc755ac640 1 -- 172.21.15.135:0/3441377691 wait complete. 2024-04-26T22:11:04.038 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-26T22:11:03.322718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252981638,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":3350,"num_read_kb":829546,"num_write":9253302,"num_write_kb":113906497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":610855,"ondisk_log_size":610855,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6041944,"kb_used_data":497752,"kb_used_omap":12,"kb_used_meta":5544115,"kb_avail":748932776,"statfs":{"total":773094113280,"available":766907162624,"internally_reserved":0,"allocated":509698048,"data_stored":507719436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":5677174321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003811"},"pg_stats":[{"pgid":"5.18","version":"40'4026","reported_seq":4109,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.893457+0000","last_change":"2024-04-26T22:10:36.893482+0000","last_active":"2024-04-26T22:10:36.893457+0000","last_peered":"2024-04-26T22:10:36.893457+0000","last_clean":"2024-04-26T22:10:36.893457+0000","last_became_active":"2024-04-26T22:00:06.809338+0000","last_became_peered":"2024-04-26T22:00:06.809338+0000","last_unstale":"2024-04-26T22:10:36.893457+0000","last_undegraded":"2024-04-26T22:10:36.893457+0000","last_fullsized":"2024-04-26T22:10:36.893457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4026","last_scrub_stamp":"2024-04-26T22:10:36.893331+0000","last_deep_scrub":"40'4026","last_deep_scrub_stamp":"2024-04-26T22:10:36.893331+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.893331+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:12.710655+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":60598,"num_write_kb":740487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"40'30","reported_seq":113,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.156080+0000","last_change":"2024-04-26T22:10:26.156123+0000","last_active":"2024-04-26T22:10:26.156080+0000","last_peered":"2024-04-26T22:10:26.156080+0000","last_clean":"2024-04-26T22:10:26.156080+0000","last_became_active":"2024-04-26T21:59:56.643719+0000","last_became_peered":"2024-04-26T21:59:56.643719+0000","last_unstale":"2024-04-26T22:10:26.156080+0000","last_undegraded":"2024-04-26T22:10:26.156080+0000","last_fullsized":"2024-04-26T22:10:26.156080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'30","last_scrub_stamp":"2024-04-26T22:10:26.155966+0000","last_deep_scrub":"40'30","last_deep_scrub_stamp":"2024-04-26T22:10:26.155966+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.155966+0000","objects_scrubbed":3,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:00.107945+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":16408,"num_write":20,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'5701","reported_seq":5767,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:43.970289+0000","last_change":"2024-04-26T22:10:43.970311+0000","last_active":"2024-04-26T22:10:43.970289+0000","last_peered":"2024-04-26T22:10:43.970289+0000","last_clean":"2024-04-26T22:10:43.970289+0000","last_became_active":"2024-04-26T21:59:57.652660+0000","last_became_peered":"2024-04-26T21:59:57.652660+0000","last_unstale":"2024-04-26T22:10:43.970289+0000","last_undegraded":"2024-04-26T22:10:43.970289+0000","last_fullsized":"2024-04-26T22:10:43.970289+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5701","last_scrub_stamp":"2024-04-26T22:10:43.970203+0000","last_deep_scrub":"40'5701","last_deep_scrub_stamp":"2024-04-26T22:10:43.970203+0000","last_clean_scrub_stamp":"2024-04-26T22:10:43.970203+0000","objects_scrubbed":0,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:22.256723+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86528,"num_write_kb":1106988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"40'21","reported_seq":80,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.937217+0000","last_change":"2024-04-26T22:10:23.937241+0000","last_active":"2024-04-26T22:10:23.937217+0000","last_peered":"2024-04-26T22:10:23.937217+0000","last_clean":"2024-04-26T22:10:23.937217+0000","last_became_active":"2024-04-26T22:00:05.813095+0000","last_became_peered":"2024-04-26T22:00:05.813095+0000","last_unstale":"2024-04-26T22:10:23.937217+0000","last_undegraded":"2024-04-26T22:10:23.937217+0000","last_fullsized":"2024-04-26T22:10:23.937217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:23.937145+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:23.937145+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.937145+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:20.231325+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190714,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"40'4869","reported_seq":4966,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:43.333084+0000","last_change":"2024-04-26T22:10:43.333129+0000","last_active":"2024-04-26T22:10:43.333084+0000","last_peered":"2024-04-26T22:10:43.333084+0000","last_clean":"2024-04-26T22:10:43.333084+0000","last_became_active":"2024-04-26T22:00:06.825195+0000","last_became_peered":"2024-04-26T22:00:06.825195+0000","last_unstale":"2024-04-26T22:10:43.333084+0000","last_undegraded":"2024-04-26T22:10:43.333084+0000","last_fullsized":"2024-04-26T22:10:43.333084+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4869","last_scrub_stamp":"2024-04-26T22:10:43.332954+0000","last_deep_scrub":"40'4869","last_deep_scrub_stamp":"2024-04-26T22:10:43.332954+0000","last_clean_scrub_stamp":"2024-04-26T22:10:43.332954+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:07.209107+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":73297,"num_write_kb":899566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"40'21","reported_seq":93,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.023138+0000","last_change":"2024-04-26T22:10:39.023173+0000","last_active":"2024-04-26T22:10:39.023138+0000","last_peered":"2024-04-26T22:10:39.023138+0000","last_clean":"2024-04-26T22:10:39.023138+0000","last_became_active":"2024-04-26T21:59:56.645774+0000","last_became_peered":"2024-04-26T21:59:56.645774+0000","last_unstale":"2024-04-26T22:10:39.023138+0000","last_undegraded":"2024-04-26T22:10:39.023138+0000","last_fullsized":"2024-04-26T22:10:39.023138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:39.023030+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:39.023030+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.023030+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:02.899183+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"40'6511","reported_seq":6583,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.137267+0000","last_change":"2024-04-26T22:10:24.137290+0000","last_active":"2024-04-26T22:10:24.137267+0000","last_peered":"2024-04-26T22:10:24.137267+0000","last_clean":"2024-04-26T22:10:24.137267+0000","last_became_active":"2024-04-26T21:59:57.646114+0000","last_became_peered":"2024-04-26T21:59:57.646114+0000","last_unstale":"2024-04-26T22:10:24.137267+0000","last_undegraded":"2024-04-26T22:10:24.137267+0000","last_fullsized":"2024-04-26T22:10:24.137267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6511","last_scrub_stamp":"2024-04-26T22:10:24.137190+0000","last_deep_scrub":"40'6511","last_deep_scrub_stamp":"2024-04-26T22:10:24.137190+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.137190+0000","objects_scrubbed":0,"log_size":6511,"log_dups_size":0,"ondisk_log_size":6511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:54.188579+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":98412,"num_write_kb":1227645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"40'134","reported_seq":199,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.831985+0000","last_change":"2024-04-26T22:10:21.832016+0000","last_active":"2024-04-26T22:10:21.831985+0000","last_peered":"2024-04-26T22:10:21.831985+0000","last_clean":"2024-04-26T22:10:21.831985+0000","last_became_active":"2024-04-26T22:00:05.819825+0000","last_became_peered":"2024-04-26T22:00:05.819825+0000","last_unstale":"2024-04-26T22:10:21.831985+0000","last_undegraded":"2024-04-26T22:10:21.831985+0000","last_fullsized":"2024-04-26T22:10:21.831985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'134","last_scrub_stamp":"2024-04-26T22:10:21.831897+0000","last_deep_scrub":"40'134","last_deep_scrub_stamp":"2024-04-26T22:10:21.831897+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.831897+0000","objects_scrubbed":2,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:34.924587+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":127,"num_write_kb":8276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"40'3508","reported_seq":3583,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:44.156754+0000","last_change":"2024-04-26T22:10:44.156779+0000","last_active":"2024-04-26T22:10:44.156754+0000","last_peered":"2024-04-26T22:10:44.156754+0000","last_clean":"2024-04-26T22:10:44.156754+0000","last_became_active":"2024-04-26T22:00:06.819968+0000","last_became_peered":"2024-04-26T22:00:06.819968+0000","last_unstale":"2024-04-26T22:10:44.156754+0000","last_undegraded":"2024-04-26T22:10:44.156754+0000","last_fullsized":"2024-04-26T22:10:44.156754+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3508","last_scrub_stamp":"2024-04-26T22:10:44.156647+0000","last_deep_scrub":"40'3508","last_deep_scrub_stamp":"2024-04-26T22:10:44.156647+0000","last_clean_scrub_stamp":"2024-04-26T22:10:44.156647+0000","objects_scrubbed":0,"log_size":3508,"log_dups_size":0,"ondisk_log_size":3508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:21.236176+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":52619,"num_write_kb":632470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"40'120","reported_seq":210,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.213875+0000","last_change":"2024-04-26T22:10:37.213912+0000","last_active":"2024-04-26T22:10:37.213875+0000","last_peered":"2024-04-26T22:10:37.213875+0000","last_clean":"2024-04-26T22:10:37.213875+0000","last_became_active":"2024-04-26T21:59:56.647170+0000","last_became_peered":"2024-04-26T21:59:56.647170+0000","last_unstale":"2024-04-26T22:10:37.213875+0000","last_undegraded":"2024-04-26T22:10:37.213875+0000","last_fullsized":"2024-04-26T22:10:37.213875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'120","last_scrub_stamp":"2024-04-26T22:10:37.213785+0000","last_deep_scrub":"40'120","last_deep_scrub_stamp":"2024-04-26T22:10:37.213785+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.213785+0000","objects_scrubbed":4,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:26.515957+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":47,"num_read_kb":45092,"num_write":102,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"40'3918","reported_seq":3984,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.179306+0000","last_change":"2024-04-26T22:10:39.179334+0000","last_active":"2024-04-26T22:10:39.179306+0000","last_peered":"2024-04-26T22:10:39.179306+0000","last_clean":"2024-04-26T22:10:39.179306+0000","last_became_active":"2024-04-26T21:59:57.645738+0000","last_became_peered":"2024-04-26T21:59:57.645738+0000","last_unstale":"2024-04-26T22:10:39.179306+0000","last_undegraded":"2024-04-26T22:10:39.179306+0000","last_fullsized":"2024-04-26T22:10:39.179306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3918","last_scrub_stamp":"2024-04-26T22:10:39.179199+0000","last_deep_scrub":"40'3918","last_deep_scrub_stamp":"2024-04-26T22:10:39.179199+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.179199+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:07.949612+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":59308,"num_write_kb":714963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"40'1269","reported_seq":1335,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.794568+0000","last_change":"2024-04-26T22:10:23.794610+0000","last_active":"2024-04-26T22:10:23.794568+0000","last_peered":"2024-04-26T22:10:23.794568+0000","last_clean":"2024-04-26T22:10:23.794568+0000","last_became_active":"2024-04-26T22:00:05.815763+0000","last_became_peered":"2024-04-26T22:00:05.815763+0000","last_unstale":"2024-04-26T22:10:23.794568+0000","last_undegraded":"2024-04-26T22:10:23.794568+0000","last_fullsized":"2024-04-26T22:10:23.794568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'1269","last_scrub_stamp":"2024-04-26T22:10:23.794478+0000","last_deep_scrub":"40'1269","last_deep_scrub_stamp":"2024-04-26T22:10:23.794478+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.794478+0000","objects_scrubbed":1,"log_size":1269,"log_dups_size":0,"ondisk_log_size":1269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:08.939312+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":1265,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'3315","reported_seq":3393,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:41.158585+0000","last_change":"2024-04-26T22:10:41.158613+0000","last_active":"2024-04-26T22:10:41.158585+0000","last_peered":"2024-04-26T22:10:41.158585+0000","last_clean":"2024-04-26T22:10:41.158585+0000","last_became_active":"2024-04-26T22:00:06.827699+0000","last_became_peered":"2024-04-26T22:00:06.827699+0000","last_unstale":"2024-04-26T22:10:41.158585+0000","last_undegraded":"2024-04-26T22:10:41.158585+0000","last_fullsized":"2024-04-26T22:10:41.158585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3315","last_scrub_stamp":"2024-04-26T22:10:41.158442+0000","last_deep_scrub":"40'3315","last_deep_scrub_stamp":"2024-04-26T22:10:41.158442+0000","last_clean_scrub_stamp":"2024-04-26T22:10:41.158442+0000","objects_scrubbed":0,"log_size":3315,"log_dups_size":0,"ondisk_log_size":3315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:34.453416+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":49749,"num_write_kb":591259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"40'22","reported_seq":103,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:41.174504+0000","last_change":"2024-04-26T22:10:41.174543+0000","last_active":"2024-04-26T22:10:41.174504+0000","last_peered":"2024-04-26T22:10:41.174504+0000","last_clean":"2024-04-26T22:10:41.174504+0000","last_became_active":"2024-04-26T21:59:56.650335+0000","last_became_peered":"2024-04-26T21:59:56.650335+0000","last_unstale":"2024-04-26T22:10:41.174504+0000","last_undegraded":"2024-04-26T22:10:41.174504+0000","last_fullsized":"2024-04-26T22:10:41.174504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'22","last_scrub_stamp":"2024-04-26T22:10:41.174418+0000","last_deep_scrub":"40'22","last_deep_scrub_stamp":"2024-04-26T22:10:41.174418+0000","last_clean_scrub_stamp":"2024-04-26T22:10:41.174418+0000","objects_scrubbed":2,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:34.469392+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":18,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"40'6620","reported_seq":6674,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.398992+0000","last_change":"2024-04-26T22:10:38.398992+0000","last_active":"2024-04-26T22:10:38.398992+0000","last_peered":"2024-04-26T22:10:38.398992+0000","last_clean":"2024-04-26T22:10:38.398992+0000","last_became_active":"2024-04-26T22:00:39.151025+0000","last_became_peered":"2024-04-26T22:00:39.151025+0000","last_unstale":"2024-04-26T22:10:38.398992+0000","last_undegraded":"2024-04-26T22:10:38.398992+0000","last_fullsized":"2024-04-26T22:10:38.398992+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6620","last_scrub_stamp":"2024-04-26T22:10:38.398847+0000","last_deep_scrub":"40'6620","last_deep_scrub_stamp":"2024-04-26T22:10:38.398847+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.398847+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:48.805412+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":100009,"num_write_kb":1224131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"40'43","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.765919+0000","last_change":"2024-04-26T22:10:27.766038+0000","last_active":"2024-04-26T22:10:27.765919+0000","last_peered":"2024-04-26T22:10:27.765919+0000","last_clean":"2024-04-26T22:10:27.765919+0000","last_became_active":"2024-04-26T22:00:05.816367+0000","last_became_peered":"2024-04-26T22:00:05.816367+0000","last_unstale":"2024-04-26T22:10:27.765919+0000","last_undegraded":"2024-04-26T22:10:27.765919+0000","last_fullsized":"2024-04-26T22:10:27.765919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'43","last_scrub_stamp":"2024-04-26T22:10:27.765838+0000","last_deep_scrub":"40'43","last_deep_scrub_stamp":"2024-04-26T22:10:27.765838+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.765838+0000","objects_scrubbed":2,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:18.805686+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":30,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"40'4367","reported_seq":4456,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.389651+0000","last_change":"2024-04-26T22:10:37.389677+0000","last_active":"2024-04-26T22:10:37.389651+0000","last_peered":"2024-04-26T22:10:37.389651+0000","last_clean":"2024-04-26T22:10:37.389651+0000","last_became_active":"2024-04-26T22:00:06.814545+0000","last_became_peered":"2024-04-26T22:00:06.814545+0000","last_unstale":"2024-04-26T22:10:37.389651+0000","last_undegraded":"2024-04-26T22:10:37.389651+0000","last_fullsized":"2024-04-26T22:10:37.389651+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4367","last_scrub_stamp":"2024-04-26T22:10:37.389553+0000","last_deep_scrub":"40'4367","last_deep_scrub_stamp":"2024-04-26T22:10:37.389553+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.389553+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:09.719845+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65356,"num_write_kb":790948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"40'4740","reported_seq":4807,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.342228+0000","last_change":"2024-04-26T22:10:33.342228+0000","last_active":"2024-04-26T22:10:33.342228+0000","last_peered":"2024-04-26T22:10:33.342228+0000","last_clean":"2024-04-26T22:10:33.342228+0000","last_became_active":"2024-04-26T21:59:57.638073+0000","last_became_peered":"2024-04-26T21:59:57.638073+0000","last_unstale":"2024-04-26T22:10:33.342228+0000","last_undegraded":"2024-04-26T22:10:33.342228+0000","last_fullsized":"2024-04-26T22:10:33.342228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4740","last_scrub_stamp":"2024-04-26T22:10:33.342090+0000","last_deep_scrub":"40'4740","last_deep_scrub_stamp":"2024-04-26T22:10:33.342090+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.342090+0000","objects_scrubbed":0,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:59.003154+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":71515,"num_write_kb":926848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"40'209","reported_seq":292,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.183050+0000","last_change":"2024-04-26T22:10:36.183083+0000","last_active":"2024-04-26T22:10:36.183050+0000","last_peered":"2024-04-26T22:10:36.183050+0000","last_clean":"2024-04-26T22:10:36.183050+0000","last_became_active":"2024-04-26T21:59:56.644889+0000","last_became_peered":"2024-04-26T21:59:56.644889+0000","last_unstale":"2024-04-26T22:10:36.183050+0000","last_undegraded":"2024-04-26T22:10:36.183050+0000","last_fullsized":"2024-04-26T22:10:36.183050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'209","last_scrub_stamp":"2024-04-26T22:10:36.182952+0000","last_deep_scrub":"40'209","last_deep_scrub_stamp":"2024-04-26T22:10:36.182952+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.182952+0000","objects_scrubbed":3,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:39.012653+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":29169,"num_write":192,"num_write_kb":8355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"40'41","reported_seq":114,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.883143+0000","last_change":"2024-04-26T22:10:35.883186+0000","last_active":"2024-04-26T22:10:35.883143+0000","last_peered":"2024-04-26T22:10:35.883143+0000","last_clean":"2024-04-26T22:10:35.883143+0000","last_became_active":"2024-04-26T22:00:05.815481+0000","last_became_peered":"2024-04-26T22:00:05.815481+0000","last_unstale":"2024-04-26T22:10:35.883143+0000","last_undegraded":"2024-04-26T22:10:35.883143+0000","last_fullsized":"2024-04-26T22:10:35.883143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'41","last_scrub_stamp":"2024-04-26T22:10:35.883051+0000","last_deep_scrub":"40'41","last_deep_scrub_stamp":"2024-04-26T22:10:35.883051+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.883051+0000","objects_scrubbed":2,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:09.957036+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":13991,"num_write":36,"num_write_kb":4124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'3984","reported_seq":4070,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.486129+0000","last_change":"2024-04-26T22:10:22.486153+0000","last_active":"2024-04-26T22:10:22.486129+0000","last_peered":"2024-04-26T22:10:22.486129+0000","last_clean":"2024-04-26T22:10:22.486129+0000","last_became_active":"2024-04-26T22:00:06.814856+0000","last_became_peered":"2024-04-26T22:00:06.814856+0000","last_unstale":"2024-04-26T22:10:22.486129+0000","last_undegraded":"2024-04-26T22:10:22.486129+0000","last_fullsized":"2024-04-26T22:10:22.486129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3984","last_scrub_stamp":"2024-04-26T22:10:22.486045+0000","last_deep_scrub":"40'3984","last_deep_scrub_stamp":"2024-04-26T22:10:22.486045+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.486045+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:18.725892+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59797,"num_write_kb":746756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"40'4602","reported_seq":4670,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.012894+0000","last_change":"2024-04-26T22:10:28.012925+0000","last_active":"2024-04-26T22:10:28.012894+0000","last_peered":"2024-04-26T22:10:28.012894+0000","last_clean":"2024-04-26T22:10:28.012894+0000","last_became_active":"2024-04-26T21:59:57.658767+0000","last_became_peered":"2024-04-26T21:59:57.658767+0000","last_unstale":"2024-04-26T22:10:28.012894+0000","last_undegraded":"2024-04-26T22:10:28.012894+0000","last_fullsized":"2024-04-26T22:10:28.012894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4602","last_scrub_stamp":"2024-04-26T22:10:28.012760+0000","last_deep_scrub":"40'4602","last_deep_scrub_stamp":"2024-04-26T22:10:28.012760+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.012760+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:08.470714+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69468,"num_write_kb":843448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"40'25","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.309321+0000","last_change":"2024-04-26T22:10:33.309321+0000","last_active":"2024-04-26T22:10:33.309321+0000","last_peered":"2024-04-26T22:10:33.309321+0000","last_clean":"2024-04-26T22:10:33.309321+0000","last_became_active":"2024-04-26T21:59:56.648758+0000","last_became_peered":"2024-04-26T21:59:56.648758+0000","last_unstale":"2024-04-26T22:10:33.309321+0000","last_undegraded":"2024-04-26T22:10:33.309321+0000","last_fullsized":"2024-04-26T22:10:33.309321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'25","last_scrub_stamp":"2024-04-26T22:10:33.309216+0000","last_deep_scrub":"40'25","last_deep_scrub_stamp":"2024-04-26T22:10:33.309216+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.309216+0000","objects_scrubbed":1,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:32:45.314332+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":20,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"40'3","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.792564+0000","last_change":"2024-04-26T22:10:25.792588+0000","last_active":"2024-04-26T22:10:25.792564+0000","last_peered":"2024-04-26T22:10:25.792564+0000","last_clean":"2024-04-26T22:10:25.792564+0000","last_became_active":"2024-04-26T22:00:05.816407+0000","last_became_peered":"2024-04-26T22:00:05.816407+0000","last_unstale":"2024-04-26T22:10:25.792564+0000","last_undegraded":"2024-04-26T22:10:25.792564+0000","last_fullsized":"2024-04-26T22:10:25.792564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:25.792472+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:25.792472+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.792472+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:00.793620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"40'3976","reported_seq":4062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:43.904399+0000","last_change":"2024-04-26T22:10:43.904428+0000","last_active":"2024-04-26T22:10:43.904399+0000","last_peered":"2024-04-26T22:10:43.904399+0000","last_clean":"2024-04-26T22:10:43.904399+0000","last_became_active":"2024-04-26T22:00:06.824876+0000","last_became_peered":"2024-04-26T22:00:06.824876+0000","last_unstale":"2024-04-26T22:10:43.904399+0000","last_undegraded":"2024-04-26T22:10:43.904399+0000","last_fullsized":"2024-04-26T22:10:43.904399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3976","last_scrub_stamp":"2024-04-26T22:10:43.904284+0000","last_deep_scrub":"40'3976","last_deep_scrub_stamp":"2024-04-26T22:10:43.904284+0000","last_clean_scrub_stamp":"2024-04-26T22:10:43.904284+0000","objects_scrubbed":0,"log_size":3976,"log_dups_size":0,"ondisk_log_size":3976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:12.674696+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":59917,"num_write_kb":706723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"40'5423","reported_seq":5490,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:41.056290+0000","last_change":"2024-04-26T22:10:41.056290+0000","last_active":"2024-04-26T22:10:41.056290+0000","last_peered":"2024-04-26T22:10:41.056290+0000","last_clean":"2024-04-26T22:10:41.056290+0000","last_became_active":"2024-04-26T21:59:57.647821+0000","last_became_peered":"2024-04-26T21:59:57.647821+0000","last_unstale":"2024-04-26T22:10:41.056290+0000","last_undegraded":"2024-04-26T22:10:41.056290+0000","last_fullsized":"2024-04-26T22:10:41.056290+0000","mapping_epoch":18,"log_start":"28'200","ondisk_log_start":"28'200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5423","last_scrub_stamp":"2024-04-26T22:10:41.056155+0000","last_deep_scrub":"40'5423","last_deep_scrub_stamp":"2024-04-26T22:10:41.056155+0000","last_clean_scrub_stamp":"2024-04-26T22:10:41.056155+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:36.139659+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":81931,"num_write_kb":1035269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"40'44","reported_seq":114,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.369244+0000","last_change":"2024-04-26T22:10:30.369244+0000","last_active":"2024-04-26T22:10:30.369244+0000","last_peered":"2024-04-26T22:10:30.369244+0000","last_clean":"2024-04-26T22:10:30.369244+0000","last_became_active":"2024-04-26T21:59:56.645008+0000","last_became_peered":"2024-04-26T21:59:56.645008+0000","last_unstale":"2024-04-26T22:10:30.369244+0000","last_undegraded":"2024-04-26T22:10:30.369244+0000","last_fullsized":"2024-04-26T22:10:30.369244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'44","last_scrub_stamp":"2024-04-26T22:10:30.368976+0000","last_deep_scrub":"40'44","last_deep_scrub_stamp":"2024-04-26T22:10:30.368976+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.368976+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:03.853474+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363115,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":324,"num_write":38,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"40'87","reported_seq":157,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.446504+0000","last_change":"2024-04-26T22:10:27.446567+0000","last_active":"2024-04-26T22:10:27.446504+0000","last_peered":"2024-04-26T22:10:27.446504+0000","last_clean":"2024-04-26T22:10:27.446504+0000","last_became_active":"2024-04-26T22:00:05.825136+0000","last_became_peered":"2024-04-26T22:00:05.825136+0000","last_unstale":"2024-04-26T22:10:27.446504+0000","last_undegraded":"2024-04-26T22:10:27.446504+0000","last_fullsized":"2024-04-26T22:10:27.446504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'87","last_scrub_stamp":"2024-04-26T22:10:27.446409+0000","last_deep_scrub":"40'87","last_deep_scrub_stamp":"2024-04-26T22:10:27.446409+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.446409+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:53.107473+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":6841,"num_write":81,"num_write_kb":4168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"40'4751","reported_seq":4838,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.249776+0000","last_change":"2024-04-26T22:10:31.249803+0000","last_active":"2024-04-26T22:10:31.249776+0000","last_peered":"2024-04-26T22:10:31.249776+0000","last_clean":"2024-04-26T22:10:31.249776+0000","last_became_active":"2024-04-26T22:00:06.823040+0000","last_became_peered":"2024-04-26T22:00:06.823040+0000","last_unstale":"2024-04-26T22:10:31.249776+0000","last_undegraded":"2024-04-26T22:10:31.249776+0000","last_fullsized":"2024-04-26T22:10:31.249776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4751","last_scrub_stamp":"2024-04-26T22:10:31.249660+0000","last_deep_scrub":"40'4751","last_deep_scrub_stamp":"2024-04-26T22:10:31.249660+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.249660+0000","objects_scrubbed":0,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:21.922150+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":71494,"num_write_kb":858502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"40'5174","reported_seq":5243,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.837126+0000","last_change":"2024-04-26T22:10:32.837126+0000","last_active":"2024-04-26T22:10:32.837126+0000","last_peered":"2024-04-26T22:10:32.837126+0000","last_clean":"2024-04-26T22:10:32.837126+0000","last_became_active":"2024-04-26T21:59:57.648372+0000","last_became_peered":"2024-04-26T21:59:57.648372+0000","last_unstale":"2024-04-26T22:10:32.837126+0000","last_undegraded":"2024-04-26T22:10:32.837126+0000","last_fullsized":"2024-04-26T22:10:32.837126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5174","last_scrub_stamp":"2024-04-26T22:10:32.836923+0000","last_deep_scrub":"40'5174","last_deep_scrub_stamp":"2024-04-26T22:10:32.836923+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.836923+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:42.665499+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":78246,"num_write_kb":941647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"40'9","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.213391+0000","last_change":"2024-04-26T22:10:40.213391+0000","last_active":"2024-04-26T22:10:40.213391+0000","last_peered":"2024-04-26T22:10:40.213391+0000","last_clean":"2024-04-26T22:10:40.213391+0000","last_became_active":"2024-04-26T21:59:56.649590+0000","last_became_peered":"2024-04-26T21:59:56.649590+0000","last_unstale":"2024-04-26T22:10:40.213391+0000","last_undegraded":"2024-04-26T22:10:40.213391+0000","last_fullsized":"2024-04-26T22:10:40.213391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'9","last_scrub_stamp":"2024-04-26T22:10:40.213271+0000","last_deep_scrub":"40'9","last_deep_scrub_stamp":"2024-04-26T22:10:40.213271+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.213271+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:04.089425+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":4,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"40'39","reported_seq":116,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.808525+0000","last_change":"2024-04-26T22:10:22.808582+0000","last_active":"2024-04-26T22:10:22.808525+0000","last_peered":"2024-04-26T22:10:22.808525+0000","last_clean":"2024-04-26T22:10:22.808525+0000","last_became_active":"2024-04-26T22:00:05.820095+0000","last_became_peered":"2024-04-26T22:00:05.820095+0000","last_unstale":"2024-04-26T22:10:22.808525+0000","last_undegraded":"2024-04-26T22:10:22.808525+0000","last_fullsized":"2024-04-26T22:10:22.808525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'39","last_scrub_stamp":"2024-04-26T22:10:22.808431+0000","last_deep_scrub":"40'39","last_deep_scrub_stamp":"2024-04-26T22:10:22.808431+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.808431+0000","objects_scrubbed":2,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:44.877319+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":31,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"40'3913","reported_seq":4008,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.020278+0000","last_change":"2024-04-26T22:10:35.020307+0000","last_active":"2024-04-26T22:10:35.020278+0000","last_peered":"2024-04-26T22:10:35.020278+0000","last_clean":"2024-04-26T22:10:35.020278+0000","last_became_active":"2024-04-26T22:00:06.813594+0000","last_became_peered":"2024-04-26T22:00:06.813594+0000","last_unstale":"2024-04-26T22:10:35.020278+0000","last_undegraded":"2024-04-26T22:10:35.020278+0000","last_fullsized":"2024-04-26T22:10:35.020278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3913","last_scrub_stamp":"2024-04-26T22:10:35.020128+0000","last_deep_scrub":"40'3913","last_deep_scrub_stamp":"2024-04-26T22:10:35.020128+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.020128+0000","objects_scrubbed":0,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:37.849829+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":58791,"num_write_kb":740377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'4599","reported_seq":4667,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.164144+0000","last_change":"2024-04-26T22:10:29.164144+0000","last_active":"2024-04-26T22:10:29.164144+0000","last_peered":"2024-04-26T22:10:29.164144+0000","last_clean":"2024-04-26T22:10:29.164144+0000","last_became_active":"2024-04-26T21:59:57.643023+0000","last_became_peered":"2024-04-26T21:59:57.643023+0000","last_unstale":"2024-04-26T22:10:29.164144+0000","last_undegraded":"2024-04-26T22:10:29.164144+0000","last_fullsized":"2024-04-26T22:10:29.164144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4599","last_scrub_stamp":"2024-04-26T22:10:29.163964+0000","last_deep_scrub":"40'4599","last_deep_scrub_stamp":"2024-04-26T22:10:29.163964+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.163964+0000","objects_scrubbed":0,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:04.165113+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":69140,"num_write_kb":847801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"40'9","reported_seq":89,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.364630+0000","last_change":"2024-04-26T22:10:30.364630+0000","last_active":"2024-04-26T22:10:30.364630+0000","last_peered":"2024-04-26T22:10:30.364630+0000","last_clean":"2024-04-26T22:10:30.364630+0000","last_became_active":"2024-04-26T21:59:56.646750+0000","last_became_peered":"2024-04-26T21:59:56.646750+0000","last_unstale":"2024-04-26T22:10:30.364630+0000","last_undegraded":"2024-04-26T22:10:30.364630+0000","last_fullsized":"2024-04-26T22:10:30.364630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'9","last_scrub_stamp":"2024-04-26T22:10:30.364522+0000","last_deep_scrub":"40'9","last_deep_scrub_stamp":"2024-04-26T22:10:30.364522+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.364522+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:57.094328+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"40'23","reported_seq":110,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.810252+0000","last_change":"2024-04-26T22:10:34.810290+0000","last_active":"2024-04-26T22:10:34.810252+0000","last_peered":"2024-04-26T22:10:34.810252+0000","last_clean":"2024-04-26T22:10:34.810252+0000","last_became_active":"2024-04-26T22:00:05.814761+0000","last_became_peered":"2024-04-26T22:00:05.814761+0000","last_unstale":"2024-04-26T22:10:34.810252+0000","last_undegraded":"2024-04-26T22:10:34.810252+0000","last_fullsized":"2024-04-26T22:10:34.810252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'23","last_scrub_stamp":"2024-04-26T22:10:34.810145+0000","last_deep_scrub":"40'23","last_deep_scrub_stamp":"2024-04-26T22:10:34.810145+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.810145+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:57.149020+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":98,"num_write":20,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"40'3929","reported_seq":4008,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:41.893859+0000","last_change":"2024-04-26T22:10:41.893886+0000","last_active":"2024-04-26T22:10:41.893859+0000","last_peered":"2024-04-26T22:10:41.893859+0000","last_clean":"2024-04-26T22:10:41.893859+0000","last_became_active":"2024-04-26T22:00:06.819584+0000","last_became_peered":"2024-04-26T22:00:06.819584+0000","last_unstale":"2024-04-26T22:10:41.893859+0000","last_undegraded":"2024-04-26T22:10:41.893859+0000","last_fullsized":"2024-04-26T22:10:41.893859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3929","last_scrub_stamp":"2024-04-26T22:10:41.893750+0000","last_deep_scrub":"40'3929","last_deep_scrub_stamp":"2024-04-26T22:10:41.893750+0000","last_clean_scrub_stamp":"2024-04-26T22:10:41.893750+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:31.195922+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":59023,"num_write_kb":718389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'5311","reported_seq":5378,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.992216+0000","last_change":"2024-04-26T22:10:33.992216+0000","last_active":"2024-04-26T22:10:33.992216+0000","last_peered":"2024-04-26T22:10:33.992216+0000","last_clean":"2024-04-26T22:10:33.992216+0000","last_became_active":"2024-04-26T21:59:57.656033+0000","last_became_peered":"2024-04-26T21:59:57.656033+0000","last_unstale":"2024-04-26T22:10:33.992216+0000","last_undegraded":"2024-04-26T22:10:33.992216+0000","last_fullsized":"2024-04-26T22:10:33.992216+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5311","last_scrub_stamp":"2024-04-26T22:10:33.992048+0000","last_deep_scrub":"40'5311","last_deep_scrub_stamp":"2024-04-26T22:10:33.992048+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.992048+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:52.407619+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80379,"num_write_kb":1004221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"40'218","reported_seq":292,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.350054+0000","last_change":"2024-04-26T22:10:32.350054+0000","last_active":"2024-04-26T22:10:32.350054+0000","last_peered":"2024-04-26T22:10:32.350054+0000","last_clean":"2024-04-26T22:10:32.350054+0000","last_became_active":"2024-04-26T21:59:56.648020+0000","last_became_peered":"2024-04-26T21:59:56.648020+0000","last_unstale":"2024-04-26T22:10:32.350054+0000","last_undegraded":"2024-04-26T22:10:32.350054+0000","last_fullsized":"2024-04-26T22:10:32.350054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'218","last_scrub_stamp":"2024-04-26T22:10:32.349915+0000","last_deep_scrub":"40'218","last_deep_scrub_stamp":"2024-04-26T22:10:32.349915+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.349915+0000","objects_scrubbed":1,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:12.807870+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":61,"num_write":416,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"40'24","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.950133+0000","last_change":"2024-04-26T22:10:22.950185+0000","last_active":"2024-04-26T22:10:22.950133+0000","last_peered":"2024-04-26T22:10:22.950133+0000","last_clean":"2024-04-26T22:10:22.950133+0000","last_became_active":"2024-04-26T22:00:05.816207+0000","last_became_peered":"2024-04-26T22:00:05.816207+0000","last_unstale":"2024-04-26T22:10:22.950133+0000","last_undegraded":"2024-04-26T22:10:22.950133+0000","last_fullsized":"2024-04-26T22:10:22.950133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'24","last_scrub_stamp":"2024-04-26T22:10:22.950054+0000","last_deep_scrub":"40'24","last_deep_scrub_stamp":"2024-04-26T22:10:22.950054+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.950054+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:48.611118+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":15,"num_write_kb":8199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"40'4334","reported_seq":4419,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.373617+0000","last_change":"2024-04-26T22:10:38.373663+0000","last_active":"2024-04-26T22:10:38.373617+0000","last_peered":"2024-04-26T22:10:38.373617+0000","last_clean":"2024-04-26T22:10:38.373617+0000","last_became_active":"2024-04-26T22:00:06.814610+0000","last_became_peered":"2024-04-26T22:00:06.814610+0000","last_unstale":"2024-04-26T22:10:38.373617+0000","last_undegraded":"2024-04-26T22:10:38.373617+0000","last_fullsized":"2024-04-26T22:10:38.373617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4334","last_scrub_stamp":"2024-04-26T22:10:38.373498+0000","last_deep_scrub":"40'4334","last_deep_scrub_stamp":"2024-04-26T22:10:38.373498+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.373498+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:37.285923+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":65365,"num_write_kb":782662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"40'5496","reported_seq":5566,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.426140+0000","last_change":"2024-04-26T22:10:24.426140+0000","last_active":"2024-04-26T22:10:24.426140+0000","last_peered":"2024-04-26T22:10:24.426140+0000","last_clean":"2024-04-26T22:10:24.426140+0000","last_became_active":"2024-04-26T21:59:57.650560+0000","last_became_peered":"2024-04-26T21:59:57.650560+0000","last_unstale":"2024-04-26T22:10:24.426140+0000","last_undegraded":"2024-04-26T22:10:24.426140+0000","last_fullsized":"2024-04-26T22:10:24.426140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5496","last_scrub_stamp":"2024-04-26T22:10:24.425962+0000","last_deep_scrub":"40'5496","last_deep_scrub_stamp":"2024-04-26T22:10:24.425962+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.425962+0000","objects_scrubbed":0,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:59.427110+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":82892,"num_write_kb":1050745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"40'36","reported_seq":106,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.219085+0000","last_change":"2024-04-26T22:10:39.219085+0000","last_active":"2024-04-26T22:10:39.219085+0000","last_peered":"2024-04-26T22:10:39.219085+0000","last_clean":"2024-04-26T22:10:39.219085+0000","last_became_active":"2024-04-26T21:59:56.649474+0000","last_became_peered":"2024-04-26T21:59:56.649474+0000","last_unstale":"2024-04-26T22:10:39.219085+0000","last_undegraded":"2024-04-26T22:10:39.219085+0000","last_fullsized":"2024-04-26T22:10:39.219085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'36","last_scrub_stamp":"2024-04-26T22:10:39.218959+0000","last_deep_scrub":"40'36","last_deep_scrub_stamp":"2024-04-26T22:10:39.218959+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.218959+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:07.989371+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":29,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"28'1","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.240571+0000","last_change":"2024-04-26T22:10:36.240630+0000","last_active":"2024-04-26T22:10:36.240571+0000","last_peered":"2024-04-26T22:10:36.240571+0000","last_clean":"2024-04-26T22:10:36.240571+0000","last_became_active":"2024-04-26T22:00:05.816941+0000","last_became_peered":"2024-04-26T22:00:05.816941+0000","last_unstale":"2024-04-26T22:10:36.240571+0000","last_undegraded":"2024-04-26T22:10:36.240571+0000","last_fullsized":"2024-04-26T22:10:36.240571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'1","last_scrub_stamp":"2024-04-26T22:10:36.240429+0000","last_deep_scrub":"28'1","last_deep_scrub_stamp":"2024-04-26T22:10:36.240429+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.240429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:46.646994+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"40'3789","reported_seq":3872,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.302299+0000","last_change":"2024-04-26T22:10:37.302344+0000","last_active":"2024-04-26T22:10:37.302299+0000","last_peered":"2024-04-26T22:10:37.302299+0000","last_clean":"2024-04-26T22:10:37.302299+0000","last_became_active":"2024-04-26T22:00:06.812882+0000","last_became_peered":"2024-04-26T22:00:06.812882+0000","last_unstale":"2024-04-26T22:10:37.302299+0000","last_undegraded":"2024-04-26T22:10:37.302299+0000","last_fullsized":"2024-04-26T22:10:37.302299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3789","last_scrub_stamp":"2024-04-26T22:10:37.302131+0000","last_deep_scrub":"40'3789","last_deep_scrub_stamp":"2024-04-26T22:10:37.302131+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.302131+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:36.214557+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":56913,"num_write_kb":707433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"40'4759","reported_seq":4827,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.169618+0000","last_change":"2024-04-26T22:10:38.169647+0000","last_active":"2024-04-26T22:10:38.169618+0000","last_peered":"2024-04-26T22:10:38.169618+0000","last_clean":"2024-04-26T22:10:38.169618+0000","last_became_active":"2024-04-26T21:59:57.645113+0000","last_became_peered":"2024-04-26T21:59:57.645113+0000","last_unstale":"2024-04-26T22:10:38.169618+0000","last_undegraded":"2024-04-26T22:10:38.169618+0000","last_fullsized":"2024-04-26T22:10:38.169618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4759","last_scrub_stamp":"2024-04-26T22:10:38.169516+0000","last_deep_scrub":"40'4759","last_deep_scrub_stamp":"2024-04-26T22:10:38.169516+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.169516+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:56.916728+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71808,"num_write_kb":921006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"40'7","reported_seq":75,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.354144+0000","last_change":"2024-04-26T22:10:31.354168+0000","last_active":"2024-04-26T22:10:31.354144+0000","last_peered":"2024-04-26T22:10:31.354144+0000","last_clean":"2024-04-26T22:10:31.354144+0000","last_became_active":"2024-04-26T21:59:56.645844+0000","last_became_peered":"2024-04-26T21:59:56.645844+0000","last_unstale":"2024-04-26T22:10:31.354144+0000","last_undegraded":"2024-04-26T22:10:31.354144+0000","last_fullsized":"2024-04-26T22:10:31.354144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'7","last_scrub_stamp":"2024-04-26T22:10:31.354057+0000","last_deep_scrub":"40'7","last_deep_scrub_stamp":"2024-04-26T22:10:31.354057+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.354057+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:41.760623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"40'185","reported_seq":253,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.486350+0000","last_change":"2024-04-26T22:10:21.486350+0000","last_active":"2024-04-26T22:10:21.486350+0000","last_peered":"2024-04-26T22:10:21.486350+0000","last_clean":"2024-04-26T22:10:21.486350+0000","last_became_active":"2024-04-26T22:00:05.818249+0000","last_became_peered":"2024-04-26T22:00:05.818249+0000","last_unstale":"2024-04-26T22:10:21.486350+0000","last_undegraded":"2024-04-26T22:10:21.486350+0000","last_fullsized":"2024-04-26T22:10:21.486350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'185","last_scrub_stamp":"2024-04-26T22:10:21.486206+0000","last_deep_scrub":"40'185","last_deep_scrub_stamp":"2024-04-26T22:10:21.486206+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.486206+0000","objects_scrubbed":2,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:43.555095+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":27847,"num_write":173,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"40'3541","reported_seq":3620,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.309309+0000","last_change":"2024-04-26T22:10:39.309309+0000","last_active":"2024-04-26T22:10:39.309309+0000","last_peered":"2024-04-26T22:10:39.309309+0000","last_clean":"2024-04-26T22:10:39.309309+0000","last_became_active":"2024-04-26T22:00:06.817848+0000","last_became_peered":"2024-04-26T22:00:06.817848+0000","last_unstale":"2024-04-26T22:10:39.309309+0000","last_undegraded":"2024-04-26T22:10:39.309309+0000","last_fullsized":"2024-04-26T22:10:39.309309+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3541","last_scrub_stamp":"2024-04-26T22:10:39.309160+0000","last_deep_scrub":"40'3541","last_deep_scrub_stamp":"2024-04-26T22:10:39.309160+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.309160+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:01.648035+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":53073,"num_write_kb":661602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"40'5106","reported_seq":5174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.815261+0000","last_change":"2024-04-26T22:10:30.815288+0000","last_active":"2024-04-26T22:10:30.815261+0000","last_peered":"2024-04-26T22:10:30.815261+0000","last_clean":"2024-04-26T22:10:30.815261+0000","last_became_active":"2024-04-26T21:59:57.639295+0000","last_became_peered":"2024-04-26T21:59:57.639295+0000","last_unstale":"2024-04-26T22:10:30.815261+0000","last_undegraded":"2024-04-26T22:10:30.815261+0000","last_fullsized":"2024-04-26T22:10:30.815261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5106","last_scrub_stamp":"2024-04-26T22:10:30.815149+0000","last_deep_scrub":"40'5106","last_deep_scrub_stamp":"2024-04-26T22:10:30.815149+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.815149+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:40.838350+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":77293,"num_write_kb":982493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"40'223","reported_seq":298,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.260885+0000","last_change":"2024-04-26T22:10:36.260916+0000","last_active":"2024-04-26T22:10:36.260885+0000","last_peered":"2024-04-26T22:10:36.260885+0000","last_clean":"2024-04-26T22:10:36.260885+0000","last_became_active":"2024-04-26T21:59:56.649913+0000","last_became_peered":"2024-04-26T21:59:56.649913+0000","last_unstale":"2024-04-26T22:10:36.260885+0000","last_undegraded":"2024-04-26T22:10:36.260885+0000","last_fullsized":"2024-04-26T22:10:36.260885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'223","last_scrub_stamp":"2024-04-26T22:10:36.260804+0000","last_deep_scrub":"40'223","last_deep_scrub_stamp":"2024-04-26T22:10:36.260804+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.260804+0000","objects_scrubbed":2,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:08.428051+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":213,"num_write_kb":8367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"40'5","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:20.475608+0000","last_change":"2024-04-26T22:10:20.475608+0000","last_active":"2024-04-26T22:10:20.475608+0000","last_peered":"2024-04-26T22:10:20.475608+0000","last_clean":"2024-04-26T22:10:20.475608+0000","last_became_active":"2024-04-26T22:00:05.815171+0000","last_became_peered":"2024-04-26T22:00:05.815171+0000","last_unstale":"2024-04-26T22:10:20.475608+0000","last_undegraded":"2024-04-26T22:10:20.475608+0000","last_fullsized":"2024-04-26T22:10:20.475608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:20.475457+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:20.475457+0000","last_clean_scrub_stamp":"2024-04-26T22:10:20.475457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:02.356378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"40'3541","reported_seq":3624,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.431964+0000","last_change":"2024-04-26T22:10:34.431964+0000","last_active":"2024-04-26T22:10:34.431964+0000","last_peered":"2024-04-26T22:10:34.431964+0000","last_clean":"2024-04-26T22:10:34.431964+0000","last_became_active":"2024-04-26T22:00:06.824054+0000","last_became_peered":"2024-04-26T22:00:06.824054+0000","last_unstale":"2024-04-26T22:10:34.431964+0000","last_undegraded":"2024-04-26T22:10:34.431964+0000","last_fullsized":"2024-04-26T22:10:34.431964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3541","last_scrub_stamp":"2024-04-26T22:10:34.431799+0000","last_deep_scrub":"40'3541","last_deep_scrub_stamp":"2024-04-26T22:10:34.431799+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.431799+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:29:00.092863+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":52805,"num_write_kb":629973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"40'5114","reported_seq":5183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.456582+0000","last_change":"2024-04-26T22:10:26.456626+0000","last_active":"2024-04-26T22:10:26.456582+0000","last_peered":"2024-04-26T22:10:26.456582+0000","last_clean":"2024-04-26T22:10:26.456582+0000","last_became_active":"2024-04-26T21:59:57.649759+0000","last_became_peered":"2024-04-26T21:59:57.649759+0000","last_unstale":"2024-04-26T22:10:26.456582+0000","last_undegraded":"2024-04-26T22:10:26.456582+0000","last_fullsized":"2024-04-26T22:10:26.456582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5114","last_scrub_stamp":"2024-04-26T22:10:26.456451+0000","last_deep_scrub":"40'5114","last_deep_scrub_stamp":"2024-04-26T22:10:26.456451+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.456451+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:17.496298+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":77095,"num_write_kb":967489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"40'19","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.557467+0000","last_change":"2024-04-26T22:10:24.557508+0000","last_active":"2024-04-26T22:10:24.557467+0000","last_peered":"2024-04-26T22:10:24.557467+0000","last_clean":"2024-04-26T22:10:24.557467+0000","last_became_active":"2024-04-26T21:59:56.641368+0000","last_became_peered":"2024-04-26T21:59:56.641368+0000","last_unstale":"2024-04-26T22:10:24.557467+0000","last_undegraded":"2024-04-26T22:10:24.557467+0000","last_fullsized":"2024-04-26T22:10:24.557467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'19","last_scrub_stamp":"2024-04-26T22:10:24.557372+0000","last_deep_scrub":"40'19","last_deep_scrub_stamp":"2024-04-26T22:10:24.557372+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.557372+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:31.018707+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88181,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":134,"num_write":15,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"40'6","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.409016+0000","last_change":"2024-04-26T22:10:25.409016+0000","last_active":"2024-04-26T22:10:25.409016+0000","last_peered":"2024-04-26T22:10:25.409016+0000","last_clean":"2024-04-26T22:10:25.409016+0000","last_became_active":"2024-04-26T22:00:05.822254+0000","last_became_peered":"2024-04-26T22:00:05.822254+0000","last_unstale":"2024-04-26T22:10:25.409016+0000","last_undegraded":"2024-04-26T22:10:25.409016+0000","last_fullsized":"2024-04-26T22:10:25.409016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:25.408850+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:25.408850+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.408850+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:11.494130+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'4139","reported_seq":4226,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:45.039370+0000","last_change":"2024-04-26T22:10:45.039370+0000","last_active":"2024-04-26T22:10:45.039370+0000","last_peered":"2024-04-26T22:10:45.039370+0000","last_clean":"2024-04-26T22:10:45.039370+0000","last_became_active":"2024-04-26T22:00:06.831604+0000","last_became_peered":"2024-04-26T22:00:06.831604+0000","last_unstale":"2024-04-26T22:10:45.039370+0000","last_undegraded":"2024-04-26T22:10:45.039370+0000","last_fullsized":"2024-04-26T22:10:45.039370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4139","last_scrub_stamp":"2024-04-26T22:10:45.039211+0000","last_deep_scrub":"40'4139","last_deep_scrub_stamp":"2024-04-26T22:10:45.039211+0000","last_clean_scrub_stamp":"2024-04-26T22:10:45.039211+0000","objects_scrubbed":0,"log_size":4139,"log_dups_size":0,"ondisk_log_size":4139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:23.333060+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":62199,"num_write_kb":728313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"40'5239","reported_seq":5307,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.132755+0000","last_change":"2024-04-26T22:10:27.132813+0000","last_active":"2024-04-26T22:10:27.132755+0000","last_peered":"2024-04-26T22:10:27.132755+0000","last_clean":"2024-04-26T22:10:27.132755+0000","last_became_active":"2024-04-26T21:59:57.642219+0000","last_became_peered":"2024-04-26T21:59:57.642219+0000","last_unstale":"2024-04-26T22:10:27.132755+0000","last_undegraded":"2024-04-26T22:10:27.132755+0000","last_fullsized":"2024-04-26T22:10:27.132755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5239","last_scrub_stamp":"2024-04-26T22:10:27.132630+0000","last_deep_scrub":"40'5239","last_deep_scrub_stamp":"2024-04-26T22:10:27.132630+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.132630+0000","objects_scrubbed":0,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:12.277463+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":79195,"num_write_kb":994121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"40'32","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.198699+0000","last_change":"2024-04-26T22:10:38.198739+0000","last_active":"2024-04-26T22:10:38.198699+0000","last_peered":"2024-04-26T22:10:38.198699+0000","last_clean":"2024-04-26T22:10:38.198699+0000","last_became_active":"2024-04-26T21:59:56.647821+0000","last_became_peered":"2024-04-26T21:59:56.647821+0000","last_unstale":"2024-04-26T22:10:38.198699+0000","last_undegraded":"2024-04-26T22:10:38.198699+0000","last_fullsized":"2024-04-26T22:10:38.198699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'32","last_scrub_stamp":"2024-04-26T22:10:38.198618+0000","last_deep_scrub":"40'32","last_deep_scrub_stamp":"2024-04-26T22:10:38.198618+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.198618+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:23.794950+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":8201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"40'212","reported_seq":279,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.174067+0000","last_change":"2024-04-26T22:10:34.174067+0000","last_active":"2024-04-26T22:10:34.174067+0000","last_peered":"2024-04-26T22:10:34.174067+0000","last_clean":"2024-04-26T22:10:34.174067+0000","last_became_active":"2024-04-26T22:00:05.815352+0000","last_became_peered":"2024-04-26T22:00:05.815352+0000","last_unstale":"2024-04-26T22:10:34.174067+0000","last_undegraded":"2024-04-26T22:10:34.174067+0000","last_fullsized":"2024-04-26T22:10:34.174067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'212","last_scrub_stamp":"2024-04-26T22:10:34.173950+0000","last_deep_scrub":"40'212","last_deep_scrub_stamp":"2024-04-26T22:10:34.173950+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.173950+0000","objects_scrubbed":1,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.197151+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":74,"num_write":426,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"40'4145","reported_seq":4231,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.269577+0000","last_change":"2024-04-26T22:10:29.269577+0000","last_active":"2024-04-26T22:10:29.269577+0000","last_peered":"2024-04-26T22:10:29.269577+0000","last_clean":"2024-04-26T22:10:29.269577+0000","last_became_active":"2024-04-26T22:00:06.816843+0000","last_became_peered":"2024-04-26T22:00:06.816843+0000","last_unstale":"2024-04-26T22:10:29.269577+0000","last_undegraded":"2024-04-26T22:10:29.269577+0000","last_fullsized":"2024-04-26T22:10:29.269577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4145","last_scrub_stamp":"2024-04-26T22:10:29.269445+0000","last_deep_scrub":"40'4145","last_deep_scrub_stamp":"2024-04-26T22:10:29.269445+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.269445+0000","objects_scrubbed":0,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:09.727400+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":62372,"num_write_kb":775201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"40'5748","reported_seq":5815,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.011933+0000","last_change":"2024-04-26T22:10:32.011960+0000","last_active":"2024-04-26T22:10:32.011933+0000","last_peered":"2024-04-26T22:10:32.011933+0000","last_clean":"2024-04-26T22:10:32.011933+0000","last_became_active":"2024-04-26T21:59:57.656862+0000","last_became_peered":"2024-04-26T21:59:57.656862+0000","last_unstale":"2024-04-26T22:10:32.011933+0000","last_undegraded":"2024-04-26T22:10:32.011933+0000","last_fullsized":"2024-04-26T22:10:32.011933+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5748","last_scrub_stamp":"2024-04-26T22:10:32.011809+0000","last_deep_scrub":"40'5748","last_deep_scrub_stamp":"2024-04-26T22:10:32.011809+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.011809+0000","objects_scrubbed":0,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:17.844177+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":87080,"num_write_kb":1065746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"39'3","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.237247+0000","last_change":"2024-04-26T22:10:33.237288+0000","last_active":"2024-04-26T22:10:33.237247+0000","last_peered":"2024-04-26T22:10:33.237247+0000","last_clean":"2024-04-26T22:10:33.237247+0000","last_became_active":"2024-04-26T21:59:56.640395+0000","last_became_peered":"2024-04-26T21:59:56.640395+0000","last_unstale":"2024-04-26T22:10:33.237247+0000","last_undegraded":"2024-04-26T22:10:33.237247+0000","last_fullsized":"2024-04-26T22:10:33.237247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3","last_scrub_stamp":"2024-04-26T22:10:33.237154+0000","last_deep_scrub":"39'3","last_deep_scrub_stamp":"2024-04-26T22:10:33.237154+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.237154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:09.709068+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"40'119","reported_seq":226,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.205515+0000","last_change":"2024-04-26T22:10:35.205563+0000","last_active":"2024-04-26T22:10:35.205515+0000","last_peered":"2024-04-26T22:10:35.205515+0000","last_clean":"2024-04-26T22:10:35.205515+0000","last_became_active":"2024-04-26T21:59:46.372186+0000","last_became_peered":"2024-04-26T21:59:46.372186+0000","last_unstale":"2024-04-26T22:10:35.205515+0000","last_undegraded":"2024-04-26T22:10:35.205515+0000","last_fullsized":"2024-04-26T22:10:35.205515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'119","last_scrub_stamp":"2024-04-26T22:10:35.205248+0000","last_deep_scrub":"40'119","last_deep_scrub_stamp":"2024-04-26T22:10:35.205248+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.205248+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:53.620818+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"40'6","reported_seq":66,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.314475+0000","last_change":"2024-04-26T22:10:37.314501+0000","last_active":"2024-04-26T22:10:37.314475+0000","last_peered":"2024-04-26T22:10:37.314475+0000","last_clean":"2024-04-26T22:10:37.314475+0000","last_became_active":"2024-04-26T22:00:05.823254+0000","last_became_peered":"2024-04-26T22:00:05.823254+0000","last_unstale":"2024-04-26T22:10:37.314475+0000","last_undegraded":"2024-04-26T22:10:37.314475+0000","last_fullsized":"2024-04-26T22:10:37.314475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:37.314391+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:37.314391+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.314391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:11.459169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"40'3379","reported_seq":3469,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.455658+0000","last_change":"2024-04-26T22:10:30.455687+0000","last_active":"2024-04-26T22:10:30.455658+0000","last_peered":"2024-04-26T22:10:30.455658+0000","last_clean":"2024-04-26T22:10:30.455658+0000","last_became_active":"2024-04-26T22:00:06.814125+0000","last_became_peered":"2024-04-26T22:00:06.814125+0000","last_unstale":"2024-04-26T22:10:30.455658+0000","last_undegraded":"2024-04-26T22:10:30.455658+0000","last_fullsized":"2024-04-26T22:10:30.455658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3379","last_scrub_stamp":"2024-04-26T22:10:30.455550+0000","last_deep_scrub":"40'3379","last_deep_scrub_stamp":"2024-04-26T22:10:30.455550+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.455550+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:57.185356+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":50792,"num_write_kb":610820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"40'5433","reported_seq":5503,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.215101+0000","last_change":"2024-04-26T22:10:30.215131+0000","last_active":"2024-04-26T22:10:30.215101+0000","last_peered":"2024-04-26T22:10:30.215101+0000","last_clean":"2024-04-26T22:10:30.215101+0000","last_became_active":"2024-04-26T21:59:57.655671+0000","last_became_peered":"2024-04-26T21:59:57.655671+0000","last_unstale":"2024-04-26T22:10:30.215101+0000","last_undegraded":"2024-04-26T22:10:30.215101+0000","last_fullsized":"2024-04-26T22:10:30.215101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5433","last_scrub_stamp":"2024-04-26T22:10:30.214972+0000","last_deep_scrub":"40'5433","last_deep_scrub_stamp":"2024-04-26T22:10:30.214972+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.214972+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:52.553847+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":81848,"num_write_kb":1014508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"40'3","reported_seq":70,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.564717+0000","last_change":"2024-04-26T22:10:26.564744+0000","last_active":"2024-04-26T22:10:26.564717+0000","last_peered":"2024-04-26T22:10:26.564717+0000","last_clean":"2024-04-26T22:10:26.564717+0000","last_became_active":"2024-04-26T21:59:56.646436+0000","last_became_peered":"2024-04-26T21:59:56.646436+0000","last_unstale":"2024-04-26T22:10:26.564717+0000","last_undegraded":"2024-04-26T22:10:26.564717+0000","last_fullsized":"2024-04-26T22:10:26.564717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:26.564626+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:26.564626+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.564626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:56.616015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"40'202","reported_seq":263,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.285126+0000","last_change":"2024-04-26T22:10:36.285126+0000","last_active":"2024-04-26T22:10:36.285126+0000","last_peered":"2024-04-26T22:10:36.285126+0000","last_clean":"2024-04-26T22:10:36.285126+0000","last_became_active":"2024-04-26T22:00:05.822639+0000","last_became_peered":"2024-04-26T22:00:05.822639+0000","last_unstale":"2024-04-26T22:10:36.285126+0000","last_undegraded":"2024-04-26T22:10:36.285126+0000","last_fullsized":"2024-04-26T22:10:36.285126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'202","last_scrub_stamp":"2024-04-26T22:10:36.285007+0000","last_deep_scrub":"40'202","last_deep_scrub_stamp":"2024-04-26T22:10:36.285007+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.285007+0000","objects_scrubbed":1,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:09.769505+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":196,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"40'3780","reported_seq":3866,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.399702+0000","last_change":"2024-04-26T22:10:33.399727+0000","last_active":"2024-04-26T22:10:33.399702+0000","last_peered":"2024-04-26T22:10:33.399702+0000","last_clean":"2024-04-26T22:10:33.399702+0000","last_became_active":"2024-04-26T22:00:06.823588+0000","last_became_peered":"2024-04-26T22:00:06.823588+0000","last_unstale":"2024-04-26T22:10:33.399702+0000","last_undegraded":"2024-04-26T22:10:33.399702+0000","last_fullsized":"2024-04-26T22:10:33.399702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3780","last_scrub_stamp":"2024-04-26T22:10:33.399366+0000","last_deep_scrub":"40'3780","last_deep_scrub_stamp":"2024-04-26T22:10:33.399366+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.399366+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:24.439213+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":56837,"num_write_kb":699720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"40'5733","reported_seq":5803,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.254368+0000","last_change":"2024-04-26T22:10:32.254411+0000","last_active":"2024-04-26T22:10:32.254368+0000","last_peered":"2024-04-26T22:10:32.254368+0000","last_clean":"2024-04-26T22:10:32.254368+0000","last_became_active":"2024-04-26T21:59:57.648761+0000","last_became_peered":"2024-04-26T21:59:57.648761+0000","last_unstale":"2024-04-26T22:10:32.254368+0000","last_undegraded":"2024-04-26T22:10:32.254368+0000","last_fullsized":"2024-04-26T22:10:32.254368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5733","last_scrub_stamp":"2024-04-26T22:10:32.254230+0000","last_deep_scrub":"40'5733","last_deep_scrub_stamp":"2024-04-26T22:10:32.254230+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.254230+0000","objects_scrubbed":0,"log_size":5733,"log_dups_size":0,"ondisk_log_size":5733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:36.887990+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":86585,"num_write_kb":1068440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"40'21","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.386012+0000","last_change":"2024-04-26T22:10:29.386047+0000","last_active":"2024-04-26T22:10:29.386012+0000","last_peered":"2024-04-26T22:10:29.386012+0000","last_clean":"2024-04-26T22:10:29.386012+0000","last_became_active":"2024-04-26T21:59:56.643887+0000","last_became_peered":"2024-04-26T21:59:56.643887+0000","last_unstale":"2024-04-26T22:10:29.386012+0000","last_undegraded":"2024-04-26T22:10:29.386012+0000","last_fullsized":"2024-04-26T22:10:29.386012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:29.385929+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:29.385929+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.385929+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:39.409130+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16390,"num_write":17,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"40'382","reported_seq":458,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.224869+0000","last_change":"2024-04-26T22:10:35.224908+0000","last_active":"2024-04-26T22:10:35.224869+0000","last_peered":"2024-04-26T22:10:35.224869+0000","last_clean":"2024-04-26T22:10:35.224869+0000","last_became_active":"2024-04-26T22:00:05.816085+0000","last_became_peered":"2024-04-26T22:00:05.816085+0000","last_unstale":"2024-04-26T22:10:35.224869+0000","last_undegraded":"2024-04-26T22:10:35.224869+0000","last_fullsized":"2024-04-26T22:10:35.224869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'382","last_scrub_stamp":"2024-04-26T22:10:35.224766+0000","last_deep_scrub":"40'382","last_deep_scrub_stamp":"2024-04-26T22:10:35.224766+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.224766+0000","objects_scrubbed":3,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:51.709281+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":25852,"num_write":364,"num_write_kb":8383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"40'3909","reported_seq":3996,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:45.140282+0000","last_change":"2024-04-26T22:10:45.140310+0000","last_active":"2024-04-26T22:10:45.140282+0000","last_peered":"2024-04-26T22:10:45.140282+0000","last_clean":"2024-04-26T22:10:45.140282+0000","last_became_active":"2024-04-26T22:00:06.820762+0000","last_became_peered":"2024-04-26T22:00:06.820762+0000","last_unstale":"2024-04-26T22:10:45.140282+0000","last_undegraded":"2024-04-26T22:10:45.140282+0000","last_fullsized":"2024-04-26T22:10:45.140282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3909","last_scrub_stamp":"2024-04-26T22:10:45.140170+0000","last_deep_scrub":"40'3909","last_deep_scrub_stamp":"2024-04-26T22:10:45.140170+0000","last_clean_scrub_stamp":"2024-04-26T22:10:45.140170+0000","objects_scrubbed":0,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:23.426690+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":58819,"num_write_kb":699051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"40'6248","reported_seq":6315,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.199932+0000","last_change":"2024-04-26T22:10:24.199962+0000","last_active":"2024-04-26T22:10:24.199932+0000","last_peered":"2024-04-26T22:10:24.199932+0000","last_clean":"2024-04-26T22:10:24.199932+0000","last_became_active":"2024-04-26T21:59:57.658196+0000","last_became_peered":"2024-04-26T21:59:57.658196+0000","last_unstale":"2024-04-26T22:10:24.199932+0000","last_undegraded":"2024-04-26T22:10:24.199932+0000","last_fullsized":"2024-04-26T22:10:24.199932+0000","mapping_epoch":18,"log_start":"28'400","ondisk_log_start":"28'400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6248","last_scrub_stamp":"2024-04-26T22:10:24.199807+0000","last_deep_scrub":"40'6248","last_deep_scrub_stamp":"2024-04-26T22:10:24.199807+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.199807+0000","objects_scrubbed":0,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:49.860871+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":94569,"num_write_kb":1160134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"40'113","reported_seq":194,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.541477+0000","last_change":"2024-04-26T22:10:25.541510+0000","last_active":"2024-04-26T22:10:25.541477+0000","last_peered":"2024-04-26T22:10:25.541477+0000","last_clean":"2024-04-26T22:10:25.541477+0000","last_became_active":"2024-04-26T21:59:56.642025+0000","last_became_peered":"2024-04-26T21:59:56.642025+0000","last_unstale":"2024-04-26T22:10:25.541477+0000","last_undegraded":"2024-04-26T22:10:25.541477+0000","last_fullsized":"2024-04-26T22:10:25.541477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'113","last_scrub_stamp":"2024-04-26T22:10:25.541375+0000","last_deep_scrub":"40'113","last_deep_scrub_stamp":"2024-04-26T22:10:25.541375+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.541375+0000","objects_scrubbed":2,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:18.885415+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":12321,"num_write":106,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"40'14","reported_seq":79,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.252568+0000","last_change":"2024-04-26T22:10:23.252568+0000","last_active":"2024-04-26T22:10:23.252568+0000","last_peered":"2024-04-26T22:10:23.252568+0000","last_clean":"2024-04-26T22:10:23.252568+0000","last_became_active":"2024-04-26T22:00:05.820729+0000","last_became_peered":"2024-04-26T22:00:05.820729+0000","last_unstale":"2024-04-26T22:10:23.252568+0000","last_undegraded":"2024-04-26T22:10:23.252568+0000","last_fullsized":"2024-04-26T22:10:23.252568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'14","last_scrub_stamp":"2024-04-26T22:10:23.252456+0000","last_deep_scrub":"40'14","last_deep_scrub_stamp":"2024-04-26T22:10:23.252456+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.252456+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:38.945444+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16384,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"40'4399","reported_seq":4482,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.441280+0000","last_change":"2024-04-26T22:10:31.441307+0000","last_active":"2024-04-26T22:10:31.441280+0000","last_peered":"2024-04-26T22:10:31.441280+0000","last_clean":"2024-04-26T22:10:31.441280+0000","last_became_active":"2024-04-26T22:00:06.815328+0000","last_became_peered":"2024-04-26T22:00:06.815328+0000","last_unstale":"2024-04-26T22:10:31.441280+0000","last_undegraded":"2024-04-26T22:10:31.441280+0000","last_fullsized":"2024-04-26T22:10:31.441280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4399","last_scrub_stamp":"2024-04-26T22:10:31.441157+0000","last_deep_scrub":"40'4399","last_deep_scrub_stamp":"2024-04-26T22:10:31.441157+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.441157+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:06.442305+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":66147,"num_write_kb":822005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"40'5224","reported_seq":5291,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.212449+0000","last_change":"2024-04-26T22:10:23.212477+0000","last_active":"2024-04-26T22:10:23.212449+0000","last_peered":"2024-04-26T22:10:23.212449+0000","last_clean":"2024-04-26T22:10:23.212449+0000","last_became_active":"2024-04-26T21:59:57.658351+0000","last_became_peered":"2024-04-26T21:59:57.658351+0000","last_unstale":"2024-04-26T22:10:23.212449+0000","last_undegraded":"2024-04-26T22:10:23.212449+0000","last_fullsized":"2024-04-26T22:10:23.212449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5224","last_scrub_stamp":"2024-04-26T22:10:23.212329+0000","last_deep_scrub":"40'5224","last_deep_scrub_stamp":"2024-04-26T22:10:23.212329+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.212329+0000","objects_scrubbed":0,"log_size":5224,"log_dups_size":0,"ondisk_log_size":5224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:14.252177+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":78957,"num_write_kb":961695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"40'18","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.803568+0000","last_change":"2024-04-26T22:10:28.803600+0000","last_active":"2024-04-26T22:10:28.803568+0000","last_peered":"2024-04-26T22:10:28.803568+0000","last_clean":"2024-04-26T22:10:28.803568+0000","last_became_active":"2024-04-26T21:59:56.646257+0000","last_became_peered":"2024-04-26T21:59:56.646257+0000","last_unstale":"2024-04-26T22:10:28.803568+0000","last_undegraded":"2024-04-26T22:10:28.803568+0000","last_fullsized":"2024-04-26T22:10:28.803568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'18","last_scrub_stamp":"2024-04-26T22:10:28.803483+0000","last_deep_scrub":"40'18","last_deep_scrub_stamp":"2024-04-26T22:10:28.803483+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.803483+0000","objects_scrubbed":2,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:54.464547+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":12294,"num_write":13,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"40'3","reported_seq":63,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.753439+0000","last_change":"2024-04-26T22:10:26.753439+0000","last_active":"2024-04-26T22:10:26.753439+0000","last_peered":"2024-04-26T22:10:26.753439+0000","last_clean":"2024-04-26T22:10:26.753439+0000","last_became_active":"2024-04-26T22:00:05.822525+0000","last_became_peered":"2024-04-26T22:00:05.822525+0000","last_unstale":"2024-04-26T22:10:26.753439+0000","last_undegraded":"2024-04-26T22:10:26.753439+0000","last_fullsized":"2024-04-26T22:10:26.753439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:26.753319+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:26.753319+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.753319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:12.838598+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"40'3562","reported_seq":3647,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:52.067133+0000","last_change":"2024-04-26T22:10:52.067156+0000","last_active":"2024-04-26T22:10:52.067133+0000","last_peered":"2024-04-26T22:10:52.067133+0000","last_clean":"2024-04-26T22:10:52.067133+0000","last_became_active":"2024-04-26T22:00:06.827001+0000","last_became_peered":"2024-04-26T22:00:06.827001+0000","last_unstale":"2024-04-26T22:10:52.067133+0000","last_undegraded":"2024-04-26T22:10:52.067133+0000","last_fullsized":"2024-04-26T22:10:52.067133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3562","last_scrub_stamp":"2024-04-26T22:10:52.067021+0000","last_deep_scrub":"40'3562","last_deep_scrub_stamp":"2024-04-26T22:10:52.067021+0000","last_clean_scrub_stamp":"2024-04-26T22:10:52.067021+0000","objects_scrubbed":0,"log_size":3562,"log_dups_size":0,"ondisk_log_size":3562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:54.544301+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":53350,"num_write_kb":651982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"40'5336","reported_seq":5391,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.370312+0000","last_change":"2024-04-26T22:10:37.370338+0000","last_active":"2024-04-26T22:10:37.370312+0000","last_peered":"2024-04-26T22:10:37.370312+0000","last_clean":"2024-04-26T22:10:37.370312+0000","last_became_active":"2024-04-26T22:00:39.150992+0000","last_became_peered":"2024-04-26T22:00:39.150992+0000","last_unstale":"2024-04-26T22:10:37.370312+0000","last_undegraded":"2024-04-26T22:10:37.370312+0000","last_fullsized":"2024-04-26T22:10:37.370312+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5336","last_scrub_stamp":"2024-04-26T22:10:37.370208+0000","last_deep_scrub":"40'5336","last_deep_scrub_stamp":"2024-04-26T22:10:37.370208+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.370208+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:10.854706+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":80461,"num_write_kb":1019956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"40'293","reported_seq":398,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.535350+0000","last_change":"2024-04-26T22:10:23.535380+0000","last_active":"2024-04-26T22:10:23.535350+0000","last_peered":"2024-04-26T22:10:23.535350+0000","last_clean":"2024-04-26T22:10:23.535350+0000","last_became_active":"2024-04-26T21:59:56.639421+0000","last_became_peered":"2024-04-26T21:59:56.639421+0000","last_unstale":"2024-04-26T22:10:23.535350+0000","last_undegraded":"2024-04-26T22:10:23.535350+0000","last_fullsized":"2024-04-26T22:10:23.535350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'293","last_scrub_stamp":"2024-04-26T22:10:23.535274+0000","last_deep_scrub":"40'293","last_deep_scrub_stamp":"2024-04-26T22:10:23.535274+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.535274+0000","objects_scrubbed":6,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:00.794998+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":12387,"num_write":284,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"40'4","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.154205+0000","last_change":"2024-04-26T22:10:23.154226+0000","last_active":"2024-04-26T22:10:23.154205+0000","last_peered":"2024-04-26T22:10:23.154205+0000","last_clean":"2024-04-26T22:10:23.154205+0000","last_became_active":"2024-04-26T22:00:05.810341+0000","last_became_peered":"2024-04-26T22:00:05.810341+0000","last_unstale":"2024-04-26T22:10:23.154205+0000","last_undegraded":"2024-04-26T22:10:23.154205+0000","last_fullsized":"2024-04-26T22:10:23.154205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4","last_scrub_stamp":"2024-04-26T22:10:23.154129+0000","last_deep_scrub":"40'4","last_deep_scrub_stamp":"2024-04-26T22:10:23.154129+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.154129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:16.498170+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"40'4185","reported_seq":4279,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:47.109724+0000","last_change":"2024-04-26T22:10:47.109774+0000","last_active":"2024-04-26T22:10:47.109724+0000","last_peered":"2024-04-26T22:10:47.109724+0000","last_clean":"2024-04-26T22:10:47.109724+0000","last_became_active":"2024-04-26T22:00:06.832390+0000","last_became_peered":"2024-04-26T22:00:06.832390+0000","last_unstale":"2024-04-26T22:10:47.109724+0000","last_undegraded":"2024-04-26T22:10:47.109724+0000","last_fullsized":"2024-04-26T22:10:47.109724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4185","last_scrub_stamp":"2024-04-26T22:10:47.109616+0000","last_deep_scrub":"40'4185","last_deep_scrub_stamp":"2024-04-26T22:10:47.109616+0000","last_clean_scrub_stamp":"2024-04-26T22:10:47.109616+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:35.488789+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":0,"num_write":62914,"num_write_kb":762533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'5750","reported_seq":5819,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.343589+0000","last_change":"2024-04-26T22:10:32.343618+0000","last_active":"2024-04-26T22:10:32.343589+0000","last_peered":"2024-04-26T22:10:32.343589+0000","last_clean":"2024-04-26T22:10:32.343589+0000","last_became_active":"2024-04-26T21:59:57.636878+0000","last_became_peered":"2024-04-26T21:59:57.636878+0000","last_unstale":"2024-04-26T22:10:32.343589+0000","last_undegraded":"2024-04-26T22:10:32.343589+0000","last_fullsized":"2024-04-26T22:10:32.343589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5750","last_scrub_stamp":"2024-04-26T22:10:32.343474+0000","last_deep_scrub":"40'5750","last_deep_scrub_stamp":"2024-04-26T22:10:32.343474+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.343474+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:43.677224+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":86794,"num_write_kb":1073964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"40'6","reported_seq":73,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.354569+0000","last_change":"2024-04-26T22:10:28.354613+0000","last_active":"2024-04-26T22:10:28.354569+0000","last_peered":"2024-04-26T22:10:28.354569+0000","last_clean":"2024-04-26T22:10:28.354569+0000","last_became_active":"2024-04-26T21:59:56.644621+0000","last_became_peered":"2024-04-26T21:59:56.644621+0000","last_unstale":"2024-04-26T22:10:28.354569+0000","last_undegraded":"2024-04-26T22:10:28.354569+0000","last_fullsized":"2024-04-26T22:10:28.354569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:28.354466+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:28.354466+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.354466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:10.649352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"40'3","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.967991+0000","last_change":"2024-04-26T22:10:21.968014+0000","last_active":"2024-04-26T22:10:21.967991+0000","last_peered":"2024-04-26T22:10:21.967991+0000","last_clean":"2024-04-26T22:10:21.967991+0000","last_became_active":"2024-04-26T22:00:05.815052+0000","last_became_peered":"2024-04-26T22:00:05.815052+0000","last_unstale":"2024-04-26T22:10:21.967991+0000","last_undegraded":"2024-04-26T22:10:21.967991+0000","last_fullsized":"2024-04-26T22:10:21.967991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3","last_scrub_stamp":"2024-04-26T22:10:21.967918+0000","last_deep_scrub":"40'3","last_deep_scrub_stamp":"2024-04-26T22:10:21.967918+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.967918+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:13.007765+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"40'3717","reported_seq":3795,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:44.056101+0000","last_change":"2024-04-26T22:10:44.056156+0000","last_active":"2024-04-26T22:10:44.056101+0000","last_peered":"2024-04-26T22:10:44.056101+0000","last_clean":"2024-04-26T22:10:44.056101+0000","last_became_active":"2024-04-26T22:00:06.831619+0000","last_became_peered":"2024-04-26T22:00:06.831619+0000","last_unstale":"2024-04-26T22:10:44.056101+0000","last_undegraded":"2024-04-26T22:10:44.056101+0000","last_fullsized":"2024-04-26T22:10:44.056101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3717","last_scrub_stamp":"2024-04-26T22:10:44.055973+0000","last_deep_scrub":"40'3717","last_deep_scrub_stamp":"2024-04-26T22:10:44.055973+0000","last_clean_scrub_stamp":"2024-04-26T22:10:44.055973+0000","objects_scrubbed":0,"log_size":3717,"log_dups_size":0,"ondisk_log_size":3717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:46.532479+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":55831,"num_write_kb":673382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"40'6101","reported_seq":6170,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.165399+0000","last_change":"2024-04-26T22:10:25.165431+0000","last_active":"2024-04-26T22:10:25.165399+0000","last_peered":"2024-04-26T22:10:25.165399+0000","last_clean":"2024-04-26T22:10:25.165399+0000","last_became_active":"2024-04-26T21:59:57.658876+0000","last_became_peered":"2024-04-26T21:59:57.658876+0000","last_unstale":"2024-04-26T22:10:25.165399+0000","last_undegraded":"2024-04-26T22:10:25.165399+0000","last_fullsized":"2024-04-26T22:10:25.165399+0000","mapping_epoch":18,"log_start":"28'300","ondisk_log_start":"28'300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6101","last_scrub_stamp":"2024-04-26T22:10:25.165276+0000","last_deep_scrub":"40'6101","last_deep_scrub_stamp":"2024-04-26T22:10:25.165276+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.165276+0000","objects_scrubbed":0,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:47.066166+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":92157,"num_write_kb":1133252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"40'5","reported_seq":72,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.207081+0000","last_change":"2024-04-26T22:10:37.207109+0000","last_active":"2024-04-26T22:10:37.207081+0000","last_peered":"2024-04-26T22:10:37.207081+0000","last_clean":"2024-04-26T22:10:37.207081+0000","last_became_active":"2024-04-26T21:59:56.651110+0000","last_became_peered":"2024-04-26T21:59:56.651110+0000","last_unstale":"2024-04-26T22:10:37.207081+0000","last_undegraded":"2024-04-26T22:10:37.207081+0000","last_fullsized":"2024-04-26T22:10:37.207081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:37.206986+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:37.206986+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.206986+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:46.662405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"40'377","reported_seq":479,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.994206+0000","last_change":"2024-04-26T22:10:26.994244+0000","last_active":"2024-04-26T22:10:26.994206+0000","last_peered":"2024-04-26T22:10:26.994206+0000","last_clean":"2024-04-26T22:10:26.994206+0000","last_became_active":"2024-04-26T22:00:05.820147+0000","last_became_peered":"2024-04-26T22:00:05.820147+0000","last_unstale":"2024-04-26T22:10:26.994206+0000","last_undegraded":"2024-04-26T22:10:26.994206+0000","last_fullsized":"2024-04-26T22:10:26.994206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'377","last_scrub_stamp":"2024-04-26T22:10:26.994121+0000","last_deep_scrub":"40'377","last_deep_scrub_stamp":"2024-04-26T22:10:26.994121+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.994121+0000","objects_scrubbed":5,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:36.822697+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":75,"num_read_kb":31157,"num_write":363,"num_write_kb":8442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"40'5464","reported_seq":5554,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:49.020719+0000","last_change":"2024-04-26T22:10:49.020746+0000","last_active":"2024-04-26T22:10:49.020719+0000","last_peered":"2024-04-26T22:10:49.020719+0000","last_clean":"2024-04-26T22:10:49.020719+0000","last_became_active":"2024-04-26T22:00:06.819498+0000","last_became_peered":"2024-04-26T22:00:06.819498+0000","last_unstale":"2024-04-26T22:10:49.020719+0000","last_undegraded":"2024-04-26T22:10:49.020719+0000","last_fullsized":"2024-04-26T22:10:49.020719+0000","mapping_epoch":22,"log_start":"31'200","ondisk_log_start":"31'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5464","last_scrub_stamp":"2024-04-26T22:10:49.020612+0000","last_deep_scrub":"40'5464","last_deep_scrub_stamp":"2024-04-26T22:10:49.020612+0000","last_clean_scrub_stamp":"2024-04-26T22:10:49.020612+0000","objects_scrubbed":0,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:19.828024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":82133,"num_write_kb":980051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"40'5077","reported_seq":5142,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.164714+0000","last_change":"2024-04-26T22:10:21.164737+0000","last_active":"2024-04-26T22:10:21.164714+0000","last_peered":"2024-04-26T22:10:21.164714+0000","last_clean":"2024-04-26T22:10:21.164714+0000","last_became_active":"2024-04-26T21:59:57.651543+0000","last_became_peered":"2024-04-26T21:59:57.651543+0000","last_unstale":"2024-04-26T22:10:21.164714+0000","last_undegraded":"2024-04-26T22:10:21.164714+0000","last_fullsized":"2024-04-26T22:10:21.164714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5077","last_scrub_stamp":"2024-04-26T22:10:21.164618+0000","last_deep_scrub":"40'5077","last_deep_scrub_stamp":"2024-04-26T22:10:21.164618+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.164618+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:58.424342+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":76618,"num_write_kb":958363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"40'23","reported_seq":109,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.119062+0000","last_change":"2024-04-26T22:10:25.119109+0000","last_active":"2024-04-26T22:10:25.119062+0000","last_peered":"2024-04-26T22:10:25.119062+0000","last_clean":"2024-04-26T22:10:25.119062+0000","last_became_active":"2024-04-26T21:59:56.641416+0000","last_became_peered":"2024-04-26T21:59:56.641416+0000","last_unstale":"2024-04-26T22:10:25.119062+0000","last_undegraded":"2024-04-26T22:10:25.119062+0000","last_fullsized":"2024-04-26T22:10:25.119062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'23","last_scrub_stamp":"2024-04-26T22:10:25.118971+0000","last_deep_scrub":"40'23","last_deep_scrub_stamp":"2024-04-26T22:10:25.118971+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.118971+0000","objects_scrubbed":4,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:38.211661+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":32,"num_read_kb":32816,"num_write":13,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"40'16","reported_seq":99,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.253601+0000","last_change":"2024-04-26T22:10:25.253631+0000","last_active":"2024-04-26T22:10:25.253601+0000","last_peered":"2024-04-26T22:10:25.253601+0000","last_clean":"2024-04-26T22:10:25.253601+0000","last_became_active":"2024-04-26T22:00:05.821135+0000","last_became_peered":"2024-04-26T22:00:05.821135+0000","last_unstale":"2024-04-26T22:10:25.253601+0000","last_undegraded":"2024-04-26T22:10:25.253601+0000","last_fullsized":"2024-04-26T22:10:25.253601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'16","last_scrub_stamp":"2024-04-26T22:10:25.253519+0000","last_deep_scrub":"40'16","last_deep_scrub_stamp":"2024-04-26T22:10:25.253519+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.253519+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:21.547699+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"40'4714","reported_seq":4806,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.998967+0000","last_change":"2024-04-26T22:10:32.999009+0000","last_active":"2024-04-26T22:10:32.998967+0000","last_peered":"2024-04-26T22:10:32.998967+0000","last_clean":"2024-04-26T22:10:32.998967+0000","last_became_active":"2024-04-26T22:00:06.812824+0000","last_became_peered":"2024-04-26T22:00:06.812824+0000","last_unstale":"2024-04-26T22:10:32.998967+0000","last_undegraded":"2024-04-26T22:10:32.998967+0000","last_fullsized":"2024-04-26T22:10:32.998967+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4714","last_scrub_stamp":"2024-04-26T22:10:32.998840+0000","last_deep_scrub":"40'4714","last_deep_scrub_stamp":"2024-04-26T22:10:32.998840+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.998840+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:05.329132+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":70866,"num_write_kb":843772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"40'5114","reported_seq":5179,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.217373+0000","last_change":"2024-04-26T22:10:28.217402+0000","last_active":"2024-04-26T22:10:28.217373+0000","last_peered":"2024-04-26T22:10:28.217373+0000","last_clean":"2024-04-26T22:10:28.217373+0000","last_became_active":"2024-04-26T21:59:57.656238+0000","last_became_peered":"2024-04-26T21:59:57.656238+0000","last_unstale":"2024-04-26T22:10:28.217373+0000","last_undegraded":"2024-04-26T22:10:28.217373+0000","last_fullsized":"2024-04-26T22:10:28.217373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5114","last_scrub_stamp":"2024-04-26T22:10:28.217250+0000","last_deep_scrub":"40'5114","last_deep_scrub_stamp":"2024-04-26T22:10:28.217250+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.217250+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:38.045826+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77180,"num_write_kb":958350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"40'829","reported_seq":931,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.400024+0000","last_change":"2024-04-26T22:10:27.400067+0000","last_active":"2024-04-26T22:10:27.400024+0000","last_peered":"2024-04-26T22:10:27.400024+0000","last_clean":"2024-04-26T22:10:27.400024+0000","last_became_active":"2024-04-26T21:59:56.643789+0000","last_became_peered":"2024-04-26T21:59:56.643789+0000","last_unstale":"2024-04-26T22:10:27.400024+0000","last_undegraded":"2024-04-26T22:10:27.400024+0000","last_fullsized":"2024-04-26T22:10:27.400024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'829","last_scrub_stamp":"2024-04-26T22:10:27.399913+0000","last_deep_scrub":"40'829","last_deep_scrub_stamp":"2024-04-26T22:10:27.399913+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.399913+0000","objects_scrubbed":6,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:54:01.351892+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":36914,"num_write":813,"num_write_kb":13003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"40'5","reported_seq":64,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.444749+0000","last_change":"2024-04-26T22:10:23.444787+0000","last_active":"2024-04-26T22:10:23.444749+0000","last_peered":"2024-04-26T22:10:23.444749+0000","last_clean":"2024-04-26T22:10:23.444749+0000","last_became_active":"2024-04-26T22:00:05.820572+0000","last_became_peered":"2024-04-26T22:00:05.820572+0000","last_unstale":"2024-04-26T22:10:23.444749+0000","last_undegraded":"2024-04-26T22:10:23.444749+0000","last_fullsized":"2024-04-26T22:10:23.444749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5","last_scrub_stamp":"2024-04-26T22:10:23.444637+0000","last_deep_scrub":"40'5","last_deep_scrub_stamp":"2024-04-26T22:10:23.444637+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.444637+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:50.174442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"40'3745","reported_seq":3832,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:44.367269+0000","last_change":"2024-04-26T22:10:44.367294+0000","last_active":"2024-04-26T22:10:44.367269+0000","last_peered":"2024-04-26T22:10:44.367269+0000","last_clean":"2024-04-26T22:10:44.367269+0000","last_became_active":"2024-04-26T22:00:06.826259+0000","last_became_peered":"2024-04-26T22:00:06.826259+0000","last_unstale":"2024-04-26T22:10:44.367269+0000","last_undegraded":"2024-04-26T22:10:44.367269+0000","last_fullsized":"2024-04-26T22:10:44.367269+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3745","last_scrub_stamp":"2024-04-26T22:10:44.367178+0000","last_deep_scrub":"40'3745","last_deep_scrub_stamp":"2024-04-26T22:10:44.367178+0000","last_clean_scrub_stamp":"2024-04-26T22:10:44.367178+0000","objects_scrubbed":0,"log_size":3745,"log_dups_size":0,"ondisk_log_size":3745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:37.662152+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56184,"num_write_kb":680067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"40'7267","reported_seq":7335,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.796950+0000","last_change":"2024-04-26T22:10:33.797021+0000","last_active":"2024-04-26T22:10:33.796950+0000","last_peered":"2024-04-26T22:10:33.796950+0000","last_clean":"2024-04-26T22:10:33.796950+0000","last_became_active":"2024-04-26T21:59:57.649677+0000","last_became_peered":"2024-04-26T21:59:57.649677+0000","last_unstale":"2024-04-26T22:10:33.796950+0000","last_undegraded":"2024-04-26T22:10:33.796950+0000","last_fullsized":"2024-04-26T22:10:33.796950+0000","mapping_epoch":18,"log_start":"30'1800","ondisk_log_start":"30'1800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'7267","last_scrub_stamp":"2024-04-26T22:10:33.796788+0000","last_deep_scrub":"40'7267","last_deep_scrub_stamp":"2024-04-26T22:10:33.796788+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.796788+0000","objects_scrubbed":0,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:14.254743+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":109924,"num_write_kb":1323267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"40'372","reported_seq":458,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.286226+0000","last_change":"2024-04-26T22:10:34.286266+0000","last_active":"2024-04-26T22:10:34.286226+0000","last_peered":"2024-04-26T22:10:34.286226+0000","last_clean":"2024-04-26T22:10:34.286226+0000","last_became_active":"2024-04-26T21:59:56.648973+0000","last_became_peered":"2024-04-26T21:59:56.648973+0000","last_unstale":"2024-04-26T22:10:34.286226+0000","last_undegraded":"2024-04-26T22:10:34.286226+0000","last_fullsized":"2024-04-26T22:10:34.286226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'372","last_scrub_stamp":"2024-04-26T22:10:34.286145+0000","last_deep_scrub":"40'372","last_deep_scrub_stamp":"2024-04-26T22:10:34.286145+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.286145+0000","objects_scrubbed":3,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:24.958635+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4138,"num_write":367,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"40'235","reported_seq":320,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.790617+0000","last_change":"2024-04-26T22:10:24.790685+0000","last_active":"2024-04-26T22:10:24.790617+0000","last_peered":"2024-04-26T22:10:24.790617+0000","last_clean":"2024-04-26T22:10:24.790617+0000","last_became_active":"2024-04-26T22:00:05.820915+0000","last_became_peered":"2024-04-26T22:00:05.820915+0000","last_unstale":"2024-04-26T22:10:24.790617+0000","last_undegraded":"2024-04-26T22:10:24.790617+0000","last_fullsized":"2024-04-26T22:10:24.790617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'235","last_scrub_stamp":"2024-04-26T22:10:24.790516+0000","last_deep_scrub":"40'235","last_deep_scrub_stamp":"2024-04-26T22:10:24.790516+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.790516+0000","objects_scrubbed":4,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:51.520322+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":33257,"num_write":223,"num_write_kb":8323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"40'4111","reported_seq":4197,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.367556+0000","last_change":"2024-04-26T22:10:39.367581+0000","last_active":"2024-04-26T22:10:39.367556+0000","last_peered":"2024-04-26T22:10:39.367556+0000","last_clean":"2024-04-26T22:10:39.367556+0000","last_became_active":"2024-04-26T22:00:06.815946+0000","last_became_peered":"2024-04-26T22:00:06.815946+0000","last_unstale":"2024-04-26T22:10:39.367556+0000","last_undegraded":"2024-04-26T22:10:39.367556+0000","last_fullsized":"2024-04-26T22:10:39.367556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4111","last_scrub_stamp":"2024-04-26T22:10:39.367450+0000","last_deep_scrub":"40'4111","last_deep_scrub_stamp":"2024-04-26T22:10:39.367450+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.367450+0000","objects_scrubbed":0,"log_size":4111,"log_dups_size":0,"ondisk_log_size":4111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:56.087982+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61736,"num_write_kb":766472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"40'4628","reported_seq":4697,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.106830+0000","last_change":"2024-04-26T22:10:31.106858+0000","last_active":"2024-04-26T22:10:31.106830+0000","last_peered":"2024-04-26T22:10:31.106830+0000","last_clean":"2024-04-26T22:10:31.106830+0000","last_became_active":"2024-04-26T21:59:57.644588+0000","last_became_peered":"2024-04-26T21:59:57.644588+0000","last_unstale":"2024-04-26T22:10:31.106830+0000","last_undegraded":"2024-04-26T22:10:31.106830+0000","last_fullsized":"2024-04-26T22:10:31.106830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4628","last_scrub_stamp":"2024-04-26T22:10:31.106732+0000","last_deep_scrub":"40'4628","last_deep_scrub_stamp":"2024-04-26T22:10:31.106732+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.106732+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:02:22.146579+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":69686,"num_write_kb":866678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"40'103","reported_seq":172,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.305006+0000","last_change":"2024-04-26T22:10:35.305061+0000","last_active":"2024-04-26T22:10:35.305006+0000","last_peered":"2024-04-26T22:10:35.305006+0000","last_clean":"2024-04-26T22:10:35.305006+0000","last_became_active":"2024-04-26T21:59:56.646932+0000","last_became_peered":"2024-04-26T21:59:56.646932+0000","last_unstale":"2024-04-26T22:10:35.305006+0000","last_undegraded":"2024-04-26T22:10:35.305006+0000","last_fullsized":"2024-04-26T22:10:35.305006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'103","last_scrub_stamp":"2024-04-26T22:10:35.304882+0000","last_deep_scrub":"40'103","last_deep_scrub_stamp":"2024-04-26T22:10:35.304882+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.304882+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.441615+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":98,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"40'16","reported_seq":118,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.011647+0000","last_change":"2024-04-26T22:10:29.011685+0000","last_active":"2024-04-26T22:10:29.011647+0000","last_peered":"2024-04-26T22:10:29.011647+0000","last_clean":"2024-04-26T22:10:29.011647+0000","last_became_active":"2024-04-26T22:00:05.820182+0000","last_became_peered":"2024-04-26T22:00:05.820182+0000","last_unstale":"2024-04-26T22:10:29.011647+0000","last_undegraded":"2024-04-26T22:10:29.011647+0000","last_fullsized":"2024-04-26T22:10:29.011647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'16","last_scrub_stamp":"2024-04-26T22:10:29.011558+0000","last_deep_scrub":"40'16","last_deep_scrub_stamp":"2024-04-26T22:10:29.011558+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.011558+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:51.350434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'4365","reported_seq":4462,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:43.144791+0000","last_change":"2024-04-26T22:10:43.144821+0000","last_active":"2024-04-26T22:10:43.144791+0000","last_peered":"2024-04-26T22:10:43.144791+0000","last_clean":"2024-04-26T22:10:43.144791+0000","last_became_active":"2024-04-26T22:00:06.819182+0000","last_became_peered":"2024-04-26T22:00:06.819182+0000","last_unstale":"2024-04-26T22:10:43.144791+0000","last_undegraded":"2024-04-26T22:10:43.144791+0000","last_fullsized":"2024-04-26T22:10:43.144791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4365","last_scrub_stamp":"2024-04-26T22:10:43.144681+0000","last_deep_scrub":"40'4365","last_deep_scrub_stamp":"2024-04-26T22:10:43.144681+0000","last_clean_scrub_stamp":"2024-04-26T22:10:43.144681+0000","objects_scrubbed":0,"log_size":4365,"log_dups_size":0,"ondisk_log_size":4365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:57.885355+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":65682,"num_write_kb":806692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"40'5549","reported_seq":5622,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.230468+0000","last_change":"2024-04-26T22:10:28.230494+0000","last_active":"2024-04-26T22:10:28.230468+0000","last_peered":"2024-04-26T22:10:28.230468+0000","last_clean":"2024-04-26T22:10:28.230468+0000","last_became_active":"2024-04-26T21:59:57.652347+0000","last_became_peered":"2024-04-26T21:59:57.652347+0000","last_unstale":"2024-04-26T22:10:28.230468+0000","last_undegraded":"2024-04-26T22:10:28.230468+0000","last_fullsized":"2024-04-26T22:10:28.230468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5549","last_scrub_stamp":"2024-04-26T22:10:28.230358+0000","last_deep_scrub":"40'5549","last_deep_scrub_stamp":"2024-04-26T22:10:28.230358+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.230358+0000","objects_scrubbed":0,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:02.375137+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":83744,"num_write_kb":1000289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"40'4","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.348286+0000","last_change":"2024-04-26T22:10:31.348313+0000","last_active":"2024-04-26T22:10:31.348286+0000","last_peered":"2024-04-26T22:10:31.348286+0000","last_clean":"2024-04-26T22:10:31.348286+0000","last_became_active":"2024-04-26T21:59:56.648344+0000","last_became_peered":"2024-04-26T21:59:56.648344+0000","last_unstale":"2024-04-26T22:10:31.348286+0000","last_undegraded":"2024-04-26T22:10:31.348286+0000","last_fullsized":"2024-04-26T22:10:31.348286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4","last_scrub_stamp":"2024-04-26T22:10:31.348178+0000","last_deep_scrub":"40'4","last_deep_scrub_stamp":"2024-04-26T22:10:31.348178+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.348178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.937782+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'6","reported_seq":65,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.951899+0000","last_change":"2024-04-26T22:10:24.951929+0000","last_active":"2024-04-26T22:10:24.951899+0000","last_peered":"2024-04-26T22:10:24.951899+0000","last_clean":"2024-04-26T22:10:24.951899+0000","last_became_active":"2024-04-26T22:00:05.815875+0000","last_became_peered":"2024-04-26T22:00:05.815875+0000","last_unstale":"2024-04-26T22:10:24.951899+0000","last_undegraded":"2024-04-26T22:10:24.951899+0000","last_fullsized":"2024-04-26T22:10:24.951899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6","last_scrub_stamp":"2024-04-26T22:10:24.951797+0000","last_deep_scrub":"40'6","last_deep_scrub_stamp":"2024-04-26T22:10:24.951797+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.951797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:34.974998+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4980","reported_seq":5072,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:41.322025+0000","last_change":"2024-04-26T22:10:41.322088+0000","last_active":"2024-04-26T22:10:41.322025+0000","last_peered":"2024-04-26T22:10:41.322025+0000","last_clean":"2024-04-26T22:10:41.322025+0000","last_became_active":"2024-04-26T22:00:06.820987+0000","last_became_peered":"2024-04-26T22:00:06.820987+0000","last_unstale":"2024-04-26T22:10:41.322025+0000","last_undegraded":"2024-04-26T22:10:41.322025+0000","last_fullsized":"2024-04-26T22:10:41.322025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4980","last_scrub_stamp":"2024-04-26T22:10:41.321872+0000","last_deep_scrub":"40'4980","last_deep_scrub_stamp":"2024-04-26T22:10:41.321872+0000","last_clean_scrub_stamp":"2024-04-26T22:10:41.321872+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:45.955632+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":74941,"num_write_kb":925745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"40'17","reported_seq":81,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.140744+0000","last_change":"2024-04-26T22:10:33.140770+0000","last_active":"2024-04-26T22:10:33.140744+0000","last_peered":"2024-04-26T22:10:33.140744+0000","last_clean":"2024-04-26T22:10:33.140744+0000","last_became_active":"2024-04-26T22:00:05.812738+0000","last_became_peered":"2024-04-26T22:00:05.812738+0000","last_unstale":"2024-04-26T22:10:33.140744+0000","last_undegraded":"2024-04-26T22:10:33.140744+0000","last_fullsized":"2024-04-26T22:10:33.140744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'17","last_scrub_stamp":"2024-04-26T22:10:33.140667+0000","last_deep_scrub":"40'17","last_deep_scrub_stamp":"2024-04-26T22:10:33.140667+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.140667+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:29.434847+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":8,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"40'6345","reported_seq":6415,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.408724+0000","last_change":"2024-04-26T22:10:28.408756+0000","last_active":"2024-04-26T22:10:28.408724+0000","last_peered":"2024-04-26T22:10:28.408724+0000","last_clean":"2024-04-26T22:10:28.408724+0000","last_became_active":"2024-04-26T21:59:57.645934+0000","last_became_peered":"2024-04-26T21:59:57.645934+0000","last_unstale":"2024-04-26T22:10:28.408724+0000","last_undegraded":"2024-04-26T22:10:28.408724+0000","last_fullsized":"2024-04-26T22:10:28.408724+0000","mapping_epoch":18,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6345","last_scrub_stamp":"2024-04-26T22:10:28.408608+0000","last_deep_scrub":"40'6345","last_deep_scrub_stamp":"2024-04-26T22:10:28.408608+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.408608+0000","objects_scrubbed":0,"log_size":5745,"log_dups_size":0,"ondisk_log_size":5745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:24.702788+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":95977,"num_write_kb":1176758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"40'21","reported_seq":96,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.263535+0000","last_change":"2024-04-26T22:10:34.263586+0000","last_active":"2024-04-26T22:10:34.263535+0000","last_peered":"2024-04-26T22:10:34.263535+0000","last_clean":"2024-04-26T22:10:34.263535+0000","last_became_active":"2024-04-26T21:59:56.644963+0000","last_became_peered":"2024-04-26T21:59:56.644963+0000","last_unstale":"2024-04-26T22:10:34.263535+0000","last_undegraded":"2024-04-26T22:10:34.263535+0000","last_fullsized":"2024-04-26T22:10:34.263535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'21","last_scrub_stamp":"2024-04-26T22:10:34.263431+0000","last_deep_scrub":"40'21","last_deep_scrub_stamp":"2024-04-26T22:10:34.263431+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.263431+0000","objects_scrubbed":2,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:06.593723+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":28672,"num_write":16,"num_write_kb":8206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"40'4260","reported_seq":4339,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.279258+0000","last_change":"2024-04-26T22:10:38.279283+0000","last_active":"2024-04-26T22:10:38.279258+0000","last_peered":"2024-04-26T22:10:38.279258+0000","last_clean":"2024-04-26T22:10:38.279258+0000","last_became_active":"2024-04-26T22:00:06.813978+0000","last_became_peered":"2024-04-26T22:00:06.813978+0000","last_unstale":"2024-04-26T22:10:38.279258+0000","last_undegraded":"2024-04-26T22:10:38.279258+0000","last_fullsized":"2024-04-26T22:10:38.279258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4260","last_scrub_stamp":"2024-04-26T22:10:38.279159+0000","last_deep_scrub":"40'4260","last_deep_scrub_stamp":"2024-04-26T22:10:38.279159+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.279159+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:54.999691+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":64151,"num_write_kb":776804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"40'28","reported_seq":109,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.318402+0000","last_change":"2024-04-26T22:10:35.318445+0000","last_active":"2024-04-26T22:10:35.318402+0000","last_peered":"2024-04-26T22:10:35.318402+0000","last_clean":"2024-04-26T22:10:35.318402+0000","last_became_active":"2024-04-26T22:00:05.819384+0000","last_became_peered":"2024-04-26T22:00:05.819384+0000","last_unstale":"2024-04-26T22:10:35.318402+0000","last_undegraded":"2024-04-26T22:10:35.318402+0000","last_fullsized":"2024-04-26T22:10:35.318402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'28","last_scrub_stamp":"2024-04-26T22:10:35.318315+0000","last_deep_scrub":"40'28","last_deep_scrub_stamp":"2024-04-26T22:10:35.318315+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.318315+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:45.341516+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20512,"num_write":20,"num_write_kb":4102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"40'5323","reported_seq":5391,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.010733+0000","last_change":"2024-04-26T22:10:30.010762+0000","last_active":"2024-04-26T22:10:30.010733+0000","last_peered":"2024-04-26T22:10:30.010733+0000","last_clean":"2024-04-26T22:10:30.010733+0000","last_became_active":"2024-04-26T21:59:57.658190+0000","last_became_peered":"2024-04-26T21:59:57.658190+0000","last_unstale":"2024-04-26T22:10:30.010733+0000","last_undegraded":"2024-04-26T22:10:30.010733+0000","last_fullsized":"2024-04-26T22:10:30.010733+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5323","last_scrub_stamp":"2024-04-26T22:10:30.010619+0000","last_deep_scrub":"40'5323","last_deep_scrub_stamp":"2024-04-26T22:10:30.010619+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.010619+0000","objects_scrubbed":0,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:04.084603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":80432,"num_write_kb":1004371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"40'47","reported_seq":116,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.065136+0000","last_change":"2024-04-26T22:10:37.065172+0000","last_active":"2024-04-26T22:10:37.065136+0000","last_peered":"2024-04-26T22:10:37.065136+0000","last_clean":"2024-04-26T22:10:37.065136+0000","last_became_active":"2024-04-26T21:59:56.643708+0000","last_became_peered":"2024-04-26T21:59:56.643708+0000","last_unstale":"2024-04-26T22:10:37.065136+0000","last_undegraded":"2024-04-26T22:10:37.065136+0000","last_fullsized":"2024-04-26T22:10:37.065136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'47","last_scrub_stamp":"2024-04-26T22:10:37.065038+0000","last_deep_scrub":"40'47","last_deep_scrub_stamp":"2024-04-26T22:10:37.065038+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.065038+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:22.661370+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":41,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'4905","reported_seq":5000,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.404486+0000","last_change":"2024-04-26T22:10:40.404515+0000","last_active":"2024-04-26T22:10:40.404486+0000","last_peered":"2024-04-26T22:10:40.404486+0000","last_clean":"2024-04-26T22:10:40.404486+0000","last_became_active":"2024-04-26T22:00:06.823123+0000","last_became_peered":"2024-04-26T22:00:06.823123+0000","last_unstale":"2024-04-26T22:10:40.404486+0000","last_undegraded":"2024-04-26T22:10:40.404486+0000","last_fullsized":"2024-04-26T22:10:40.404486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4905","last_scrub_stamp":"2024-04-26T22:10:40.404383+0000","last_deep_scrub":"40'4905","last_deep_scrub_stamp":"2024-04-26T22:10:40.404383+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.404383+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:29.706555+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":74018,"num_write_kb":905043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"40'148","reported_seq":225,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.396878+0000","last_change":"2024-04-26T22:10:29.396913+0000","last_active":"2024-04-26T22:10:29.396878+0000","last_peered":"2024-04-26T22:10:29.396878+0000","last_clean":"2024-04-26T22:10:29.396878+0000","last_became_active":"2024-04-26T21:59:56.644744+0000","last_became_peered":"2024-04-26T21:59:56.644744+0000","last_unstale":"2024-04-26T22:10:29.396878+0000","last_undegraded":"2024-04-26T22:10:29.396878+0000","last_fullsized":"2024-04-26T22:10:29.396878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'148","last_scrub_stamp":"2024-04-26T22:10:29.396793+0000","last_deep_scrub":"40'148","last_deep_scrub_stamp":"2024-04-26T22:10:29.396793+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.396793+0000","objects_scrubbed":3,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:31.794704+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":36864,"num_write":136,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"40'4250","reported_seq":4316,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.169415+0000","last_change":"2024-04-26T22:10:28.169480+0000","last_active":"2024-04-26T22:10:28.169415+0000","last_peered":"2024-04-26T22:10:28.169415+0000","last_clean":"2024-04-26T22:10:28.169415+0000","last_became_active":"2024-04-26T21:59:57.642390+0000","last_became_peered":"2024-04-26T21:59:57.642390+0000","last_unstale":"2024-04-26T22:10:28.169415+0000","last_undegraded":"2024-04-26T22:10:28.169415+0000","last_fullsized":"2024-04-26T22:10:28.169415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4250","last_scrub_stamp":"2024-04-26T22:10:28.169303+0000","last_deep_scrub":"40'4250","last_deep_scrub_stamp":"2024-04-26T22:10:28.169303+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.169303+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:09.367910+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":64229,"num_write_kb":785523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"40'40","reported_seq":117,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.507144+0000","last_change":"2024-04-26T22:10:29.507176+0000","last_active":"2024-04-26T22:10:29.507144+0000","last_peered":"2024-04-26T22:10:29.507144+0000","last_clean":"2024-04-26T22:10:29.507144+0000","last_became_active":"2024-04-26T22:00:05.811782+0000","last_became_peered":"2024-04-26T22:00:05.811782+0000","last_unstale":"2024-04-26T22:10:29.507144+0000","last_undegraded":"2024-04-26T22:10:29.507144+0000","last_fullsized":"2024-04-26T22:10:29.507144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'40","last_scrub_stamp":"2024-04-26T22:10:29.507051+0000","last_deep_scrub":"40'40","last_deep_scrub_stamp":"2024-04-26T22:10:29.507051+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.507051+0000","objects_scrubbed":2,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.651885+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":8240,"num_write":35,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2966","reported_seq":3035,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.882227+0000","last_change":"2024-04-26T22:10:39.882255+0000","last_active":"2024-04-26T22:10:39.882227+0000","last_peered":"2024-04-26T22:10:39.882227+0000","last_clean":"2024-04-26T22:10:39.882227+0000","last_became_active":"2024-04-26T22:00:06.816855+0000","last_became_peered":"2024-04-26T22:00:06.816855+0000","last_unstale":"2024-04-26T22:10:39.882227+0000","last_undegraded":"2024-04-26T22:10:39.882227+0000","last_fullsized":"2024-04-26T22:10:39.882227+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'2966","last_scrub_stamp":"2024-04-26T22:10:39.882100+0000","last_deep_scrub":"40'2966","last_deep_scrub_stamp":"2024-04-26T22:10:39.882100+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.882100+0000","objects_scrubbed":0,"log_size":2966,"log_dups_size":0,"ondisk_log_size":2966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:58.297671+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":0,"num_write":44534,"num_write_kb":522679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"40'1256","reported_seq":1329,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.034755+0000","last_change":"2024-04-26T22:10:38.034811+0000","last_active":"2024-04-26T22:10:38.034755+0000","last_peered":"2024-04-26T22:10:38.034755+0000","last_clean":"2024-04-26T22:10:38.034755+0000","last_became_active":"2024-04-26T21:59:56.645134+0000","last_became_peered":"2024-04-26T21:59:56.645134+0000","last_unstale":"2024-04-26T22:10:38.034755+0000","last_undegraded":"2024-04-26T22:10:38.034755+0000","last_fullsized":"2024-04-26T22:10:38.034755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'1256","last_scrub_stamp":"2024-04-26T22:10:38.034645+0000","last_deep_scrub":"40'1256","last_deep_scrub_stamp":"2024-04-26T22:10:38.034645+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.034645+0000","objects_scrubbed":1,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:06.805058+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":20480,"num_write":1252,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"40'4813","reported_seq":4881,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.809356+0000","last_change":"2024-04-26T22:10:31.809402+0000","last_active":"2024-04-26T22:10:31.809356+0000","last_peered":"2024-04-26T22:10:31.809356+0000","last_clean":"2024-04-26T22:10:31.809356+0000","last_became_active":"2024-04-26T21:59:57.642924+0000","last_became_peered":"2024-04-26T21:59:57.642924+0000","last_unstale":"2024-04-26T22:10:31.809356+0000","last_undegraded":"2024-04-26T22:10:31.809356+0000","last_fullsized":"2024-04-26T22:10:31.809356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4813","last_scrub_stamp":"2024-04-26T22:10:31.809248+0000","last_deep_scrub":"40'4813","last_deep_scrub_stamp":"2024-04-26T22:10:31.809248+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.809248+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:05.293746+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":72672,"num_write_kb":866290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"40'90","reported_seq":179,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.039273+0000","last_change":"2024-04-26T22:10:31.039314+0000","last_active":"2024-04-26T22:10:31.039273+0000","last_peered":"2024-04-26T22:10:31.039273+0000","last_clean":"2024-04-26T22:10:31.039273+0000","last_became_active":"2024-04-26T22:00:05.820468+0000","last_became_peered":"2024-04-26T22:00:05.820468+0000","last_unstale":"2024-04-26T22:10:31.039273+0000","last_undegraded":"2024-04-26T22:10:31.039273+0000","last_fullsized":"2024-04-26T22:10:31.039273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'90","last_scrub_stamp":"2024-04-26T22:10:31.039155+0000","last_deep_scrub":"40'90","last_deep_scrub_stamp":"2024-04-26T22:10:31.039155+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.039155+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:06.856478+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":62,"num_read_kb":49200,"num_write":70,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"40'3892","reported_seq":3973,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:44.952568+0000","last_change":"2024-04-26T22:10:44.952593+0000","last_active":"2024-04-26T22:10:44.952568+0000","last_peered":"2024-04-26T22:10:44.952568+0000","last_clean":"2024-04-26T22:10:44.952568+0000","last_became_active":"2024-04-26T22:00:06.813834+0000","last_became_peered":"2024-04-26T22:00:06.813834+0000","last_unstale":"2024-04-26T22:10:44.952568+0000","last_undegraded":"2024-04-26T22:10:44.952568+0000","last_fullsized":"2024-04-26T22:10:44.952568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3892","last_scrub_stamp":"2024-04-26T22:10:44.952469+0000","last_deep_scrub":"40'3892","last_deep_scrub_stamp":"2024-04-26T22:10:44.952469+0000","last_clean_scrub_stamp":"2024-04-26T22:10:44.952469+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:23.246319+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":58573,"num_write_kb":703698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"40'6654","reported_seq":6722,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.435185+0000","last_change":"2024-04-26T22:10:26.435215+0000","last_active":"2024-04-26T22:10:26.435185+0000","last_peered":"2024-04-26T22:10:26.435185+0000","last_clean":"2024-04-26T22:10:26.435185+0000","last_became_active":"2024-04-26T21:59:57.639432+0000","last_became_peered":"2024-04-26T21:59:57.639432+0000","last_unstale":"2024-04-26T22:10:26.435185+0000","last_undegraded":"2024-04-26T22:10:26.435185+0000","last_fullsized":"2024-04-26T22:10:26.435185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6654","last_scrub_stamp":"2024-04-26T22:10:26.435070+0000","last_deep_scrub":"40'6654","last_deep_scrub_stamp":"2024-04-26T22:10:26.435070+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.435070+0000","objects_scrubbed":0,"log_size":6654,"log_dups_size":0,"ondisk_log_size":6654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:33.327046+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":100552,"num_write_kb":1253937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"40'4775","reported_seq":4869,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:41.405701+0000","last_change":"2024-04-26T22:10:41.405701+0000","last_active":"2024-04-26T22:10:41.405701+0000","last_peered":"2024-04-26T22:10:41.405701+0000","last_clean":"2024-04-26T22:10:41.405701+0000","last_became_active":"2024-04-26T22:00:06.824211+0000","last_became_peered":"2024-04-26T22:00:06.824211+0000","last_unstale":"2024-04-26T22:10:41.405701+0000","last_undegraded":"2024-04-26T22:10:41.405701+0000","last_fullsized":"2024-04-26T22:10:41.405701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4775","last_scrub_stamp":"2024-04-26T22:10:41.405522+0000","last_deep_scrub":"40'4775","last_deep_scrub_stamp":"2024-04-26T22:10:41.405522+0000","last_clean_scrub_stamp":"2024-04-26T22:10:41.405522+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:27.001853+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":71741,"num_write_kb":892635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"40'5861","reported_seq":5927,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:28.562664+0000","last_change":"2024-04-26T22:10:28.562664+0000","last_active":"2024-04-26T22:10:28.562664+0000","last_peered":"2024-04-26T22:10:28.562664+0000","last_clean":"2024-04-26T22:10:28.562664+0000","last_became_active":"2024-04-26T21:59:57.653449+0000","last_became_peered":"2024-04-26T21:59:57.653449+0000","last_unstale":"2024-04-26T22:10:28.562664+0000","last_undegraded":"2024-04-26T22:10:28.562664+0000","last_fullsized":"2024-04-26T22:10:28.562664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5861","last_scrub_stamp":"2024-04-26T22:10:28.562516+0000","last_deep_scrub":"40'5861","last_deep_scrub_stamp":"2024-04-26T22:10:28.562516+0000","last_clean_scrub_stamp":"2024-04-26T22:10:28.562516+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:50.631404+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88543,"num_write_kb":1086952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"40'3712","reported_seq":3789,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:42.134375+0000","last_change":"2024-04-26T22:10:42.134375+0000","last_active":"2024-04-26T22:10:42.134375+0000","last_peered":"2024-04-26T22:10:42.134375+0000","last_clean":"2024-04-26T22:10:42.134375+0000","last_became_active":"2024-04-26T22:00:06.827798+0000","last_became_peered":"2024-04-26T22:00:06.827798+0000","last_unstale":"2024-04-26T22:10:42.134375+0000","last_undegraded":"2024-04-26T22:10:42.134375+0000","last_fullsized":"2024-04-26T22:10:42.134375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3712","last_scrub_stamp":"2024-04-26T22:10:42.134244+0000","last_deep_scrub":"40'3712","last_deep_scrub_stamp":"2024-04-26T22:10:42.134244+0000","last_clean_scrub_stamp":"2024-04-26T22:10:42.134244+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:34.341644+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":0,"num_write":55802,"num_write_kb":677006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"40'6511","reported_seq":6584,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:48.125779+0000","last_change":"2024-04-26T22:10:48.125805+0000","last_active":"2024-04-26T22:10:48.125779+0000","last_peered":"2024-04-26T22:10:48.125779+0000","last_clean":"2024-04-26T22:10:48.125779+0000","last_became_active":"2024-04-26T22:00:39.200592+0000","last_became_peered":"2024-04-26T22:00:39.200592+0000","last_unstale":"2024-04-26T22:10:48.125779+0000","last_undegraded":"2024-04-26T22:10:48.125779+0000","last_fullsized":"2024-04-26T22:10:48.125779+0000","mapping_epoch":25,"log_start":"28'600","ondisk_log_start":"28'600","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6511","last_scrub_stamp":"2024-04-26T22:10:48.125669+0000","last_deep_scrub":"40'6511","last_deep_scrub_stamp":"2024-04-26T22:10:48.125669+0000","last_clean_scrub_stamp":"2024-04-26T22:10:48.125669+0000","objects_scrubbed":0,"log_size":5911,"log_dups_size":0,"ondisk_log_size":5911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:18.933081+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":98396,"num_write_kb":1218831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"40'4288","reported_seq":4374,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:42.369583+0000","last_change":"2024-04-26T22:10:42.369583+0000","last_active":"2024-04-26T22:10:42.369583+0000","last_peered":"2024-04-26T22:10:42.369583+0000","last_clean":"2024-04-26T22:10:42.369583+0000","last_became_active":"2024-04-26T22:00:06.823957+0000","last_became_peered":"2024-04-26T22:00:06.823957+0000","last_unstale":"2024-04-26T22:10:42.369583+0000","last_undegraded":"2024-04-26T22:10:42.369583+0000","last_fullsized":"2024-04-26T22:10:42.369583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4288","last_scrub_stamp":"2024-04-26T22:10:42.369451+0000","last_deep_scrub":"40'4288","last_deep_scrub_stamp":"2024-04-26T22:10:42.369451+0000","last_clean_scrub_stamp":"2024-04-26T22:10:42.369451+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:11.139863+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":64735,"num_write_kb":800999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"40'4650","reported_seq":4717,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.121706+0000","last_change":"2024-04-26T22:10:32.121706+0000","last_active":"2024-04-26T22:10:32.121706+0000","last_peered":"2024-04-26T22:10:32.121706+0000","last_clean":"2024-04-26T22:10:32.121706+0000","last_became_active":"2024-04-26T21:59:57.645338+0000","last_became_peered":"2024-04-26T21:59:57.645338+0000","last_unstale":"2024-04-26T22:10:32.121706+0000","last_undegraded":"2024-04-26T22:10:32.121706+0000","last_fullsized":"2024-04-26T22:10:32.121706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4650","last_scrub_stamp":"2024-04-26T22:10:32.121543+0000","last_deep_scrub":"40'4650","last_deep_scrub_stamp":"2024-04-26T22:10:32.121543+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.121543+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:57.782607+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":70263,"num_write_kb":869775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"40'5113","reported_seq":5204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:46.132942+0000","last_change":"2024-04-26T22:10:46.132966+0000","last_active":"2024-04-26T22:10:46.132942+0000","last_peered":"2024-04-26T22:10:46.132942+0000","last_clean":"2024-04-26T22:10:46.132942+0000","last_became_active":"2024-04-26T22:00:06.820657+0000","last_became_peered":"2024-04-26T22:00:06.820657+0000","last_unstale":"2024-04-26T22:10:46.132942+0000","last_undegraded":"2024-04-26T22:10:46.132942+0000","last_fullsized":"2024-04-26T22:10:46.132942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5113","last_scrub_stamp":"2024-04-26T22:10:46.132829+0000","last_deep_scrub":"40'5113","last_deep_scrub_stamp":"2024-04-26T22:10:46.132829+0000","last_clean_scrub_stamp":"2024-04-26T22:10:46.132829+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:24.426678+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":0,"num_write":76970,"num_write_kb":914783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"40'4980","reported_seq":5048,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.029163+0000","last_change":"2024-04-26T22:10:40.029163+0000","last_active":"2024-04-26T22:10:40.029163+0000","last_peered":"2024-04-26T22:10:40.029163+0000","last_clean":"2024-04-26T22:10:40.029163+0000","last_became_active":"2024-04-26T21:59:57.646697+0000","last_became_peered":"2024-04-26T21:59:57.646697+0000","last_unstale":"2024-04-26T22:10:40.029163+0000","last_undegraded":"2024-04-26T22:10:40.029163+0000","last_fullsized":"2024-04-26T22:10:40.029163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4980","last_scrub_stamp":"2024-04-26T22:10:40.029039+0000","last_deep_scrub":"40'4980","last_deep_scrub_stamp":"2024-04-26T22:10:40.029039+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.029039+0000","objects_scrubbed":0,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:33.324013+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":75382,"num_write_kb":914602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"40'3556","reported_seq":3638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:37.874865+0000","last_change":"2024-04-26T22:10:37.874865+0000","last_active":"2024-04-26T22:10:37.874865+0000","last_peered":"2024-04-26T22:10:37.874865+0000","last_clean":"2024-04-26T22:10:37.874865+0000","last_became_active":"2024-04-26T22:00:06.815449+0000","last_became_peered":"2024-04-26T22:00:06.815449+0000","last_unstale":"2024-04-26T22:10:37.874865+0000","last_undegraded":"2024-04-26T22:10:37.874865+0000","last_fullsized":"2024-04-26T22:10:37.874865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3556","last_scrub_stamp":"2024-04-26T22:10:37.874735+0000","last_deep_scrub":"40'3556","last_deep_scrub_stamp":"2024-04-26T22:10:37.874735+0000","last_clean_scrub_stamp":"2024-04-26T22:10:37.874735+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:23.707103+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":53572,"num_write_kb":653683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"40'5738","reported_seq":5805,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.325023+0000","last_change":"2024-04-26T22:10:35.325023+0000","last_active":"2024-04-26T22:10:35.325023+0000","last_peered":"2024-04-26T22:10:35.325023+0000","last_clean":"2024-04-26T22:10:35.325023+0000","last_became_active":"2024-04-26T21:59:57.644524+0000","last_became_peered":"2024-04-26T21:59:57.644524+0000","last_unstale":"2024-04-26T22:10:35.325023+0000","last_undegraded":"2024-04-26T22:10:35.325023+0000","last_fullsized":"2024-04-26T22:10:35.325023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5738","last_scrub_stamp":"2024-04-26T22:10:35.324891+0000","last_deep_scrub":"40'5738","last_deep_scrub_stamp":"2024-04-26T22:10:35.324891+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.324891+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:39.958651+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":86675,"num_write_kb":1103247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"40'5159","reported_seq":5248,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:33.257786+0000","last_change":"2024-04-26T22:10:33.257786+0000","last_active":"2024-04-26T22:10:33.257786+0000","last_peered":"2024-04-26T22:10:33.257786+0000","last_clean":"2024-04-26T22:10:33.257786+0000","last_became_active":"2024-04-26T22:00:06.818396+0000","last_became_peered":"2024-04-26T22:00:06.818396+0000","last_unstale":"2024-04-26T22:10:33.257786+0000","last_undegraded":"2024-04-26T22:10:33.257786+0000","last_fullsized":"2024-04-26T22:10:33.257786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5159","last_scrub_stamp":"2024-04-26T22:10:33.257606+0000","last_deep_scrub":"40'5159","last_deep_scrub_stamp":"2024-04-26T22:10:33.257606+0000","last_clean_scrub_stamp":"2024-04-26T22:10:33.257606+0000","objects_scrubbed":0,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:19.089974+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":77704,"num_write_kb":942965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"40'4895","reported_seq":4977,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.446854+0000","last_change":"2024-04-26T22:10:25.446854+0000","last_active":"2024-04-26T22:10:25.446854+0000","last_peered":"2024-04-26T22:10:25.446854+0000","last_clean":"2024-04-26T22:10:25.446854+0000","last_became_active":"2024-04-26T21:59:57.649392+0000","last_became_peered":"2024-04-26T21:59:57.649392+0000","last_unstale":"2024-04-26T22:10:25.446854+0000","last_undegraded":"2024-04-26T22:10:25.446854+0000","last_fullsized":"2024-04-26T22:10:25.446854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4895","last_scrub_stamp":"2024-04-26T22:10:25.446699+0000","last_deep_scrub":"40'4895","last_deep_scrub_stamp":"2024-04-26T22:10:25.446699+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.446699+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:18.790740+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":363,"num_write":74201,"num_write_kb":908910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"40'4370","reported_seq":4449,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.382324+0000","last_change":"2024-04-26T22:10:40.382362+0000","last_active":"2024-04-26T22:10:40.382324+0000","last_peered":"2024-04-26T22:10:40.382324+0000","last_clean":"2024-04-26T22:10:40.382324+0000","last_became_active":"2024-04-26T22:00:06.816749+0000","last_became_peered":"2024-04-26T22:00:06.816749+0000","last_unstale":"2024-04-26T22:10:40.382324+0000","last_undegraded":"2024-04-26T22:10:40.382324+0000","last_fullsized":"2024-04-26T22:10:40.382324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4370","last_scrub_stamp":"2024-04-26T22:10:40.382222+0000","last_deep_scrub":"40'4370","last_deep_scrub_stamp":"2024-04-26T22:10:40.382222+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.382222+0000","objects_scrubbed":0,"log_size":4370,"log_dups_size":0,"ondisk_log_size":4370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:31.054712+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":65733,"num_write_kb":804896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"40'6465","reported_seq":6535,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:25.949476+0000","last_change":"2024-04-26T22:10:25.949476+0000","last_active":"2024-04-26T22:10:25.949476+0000","last_peered":"2024-04-26T22:10:25.949476+0000","last_clean":"2024-04-26T22:10:25.949476+0000","last_became_active":"2024-04-26T21:59:57.660097+0000","last_became_peered":"2024-04-26T21:59:57.660097+0000","last_unstale":"2024-04-26T22:10:25.949476+0000","last_undegraded":"2024-04-26T22:10:25.949476+0000","last_fullsized":"2024-04-26T22:10:25.949476+0000","mapping_epoch":18,"log_start":"30'1200","ondisk_log_start":"30'1200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6465","last_scrub_stamp":"2024-04-26T22:10:25.949356+0000","last_deep_scrub":"40'6465","last_deep_scrub_stamp":"2024-04-26T22:10:25.949356+0000","last_clean_scrub_stamp":"2024-04-26T22:10:25.949356+0000","objects_scrubbed":0,"log_size":5265,"log_dups_size":0,"ondisk_log_size":5265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:59.433854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":97620,"num_write_kb":1192207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"40'3334","reported_seq":3415,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:47.970882+0000","last_change":"2024-04-26T22:10:47.970911+0000","last_active":"2024-04-26T22:10:47.970882+0000","last_peered":"2024-04-26T22:10:47.970882+0000","last_clean":"2024-04-26T22:10:47.970882+0000","last_became_active":"2024-04-26T22:00:06.815457+0000","last_became_peered":"2024-04-26T22:00:06.815457+0000","last_unstale":"2024-04-26T22:10:47.970882+0000","last_undegraded":"2024-04-26T22:10:47.970882+0000","last_fullsized":"2024-04-26T22:10:47.970882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3334","last_scrub_stamp":"2024-04-26T22:10:47.970754+0000","last_deep_scrub":"40'3334","last_deep_scrub_stamp":"2024-04-26T22:10:47.970754+0000","last_clean_scrub_stamp":"2024-04-26T22:10:47.970754+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:53.475011+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":49724,"num_write_kb":588291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"40'4839","reported_seq":4906,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.219051+0000","last_change":"2024-04-26T22:10:30.219051+0000","last_active":"2024-04-26T22:10:30.219051+0000","last_peered":"2024-04-26T22:10:30.219051+0000","last_clean":"2024-04-26T22:10:30.219051+0000","last_became_active":"2024-04-26T21:59:57.651591+0000","last_became_peered":"2024-04-26T21:59:57.651591+0000","last_unstale":"2024-04-26T22:10:30.219051+0000","last_undegraded":"2024-04-26T22:10:30.219051+0000","last_fullsized":"2024-04-26T22:10:30.219051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4839","last_scrub_stamp":"2024-04-26T22:10:30.218860+0000","last_deep_scrub":"40'4839","last_deep_scrub_stamp":"2024-04-26T22:10:30.218860+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.218860+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:46.939392+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":73090,"num_write_kb":888426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"40'5009","reported_seq":5088,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:49.122936+0000","last_change":"2024-04-26T22:10:49.122967+0000","last_active":"2024-04-26T22:10:49.122936+0000","last_peered":"2024-04-26T22:10:49.122936+0000","last_clean":"2024-04-26T22:10:49.122936+0000","last_became_active":"2024-04-26T22:00:06.824624+0000","last_became_peered":"2024-04-26T22:00:06.824624+0000","last_unstale":"2024-04-26T22:10:49.122936+0000","last_undegraded":"2024-04-26T22:10:49.122936+0000","last_fullsized":"2024-04-26T22:10:49.122936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5009","last_scrub_stamp":"2024-04-26T22:10:49.122812+0000","last_deep_scrub":"40'5009","last_deep_scrub_stamp":"2024-04-26T22:10:49.122812+0000","last_clean_scrub_stamp":"2024-04-26T22:10:49.122812+0000","objects_scrubbed":0,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:54.627069+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":0,"num_write":75285,"num_write_kb":916871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"40'5333","reported_seq":5399,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:43.005728+0000","last_change":"2024-04-26T22:10:43.005756+0000","last_active":"2024-04-26T22:10:43.005728+0000","last_peered":"2024-04-26T22:10:43.005728+0000","last_clean":"2024-04-26T22:10:43.005728+0000","last_became_active":"2024-04-26T21:59:57.652430+0000","last_became_peered":"2024-04-26T21:59:57.652430+0000","last_unstale":"2024-04-26T22:10:43.005728+0000","last_undegraded":"2024-04-26T22:10:43.005728+0000","last_fullsized":"2024-04-26T22:10:43.005728+0000","mapping_epoch":18,"log_start":"28'100","ondisk_log_start":"28'100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5333","last_scrub_stamp":"2024-04-26T22:10:43.005591+0000","last_deep_scrub":"40'5333","last_deep_scrub_stamp":"2024-04-26T22:10:43.005591+0000","last_clean_scrub_stamp":"2024-04-26T22:10:43.005591+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:20.085120+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":80596,"num_write_kb":994453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"40'4157","reported_seq":4240,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:35.386617+0000","last_change":"2024-04-26T22:10:35.386643+0000","last_active":"2024-04-26T22:10:35.386617+0000","last_peered":"2024-04-26T22:10:35.386617+0000","last_clean":"2024-04-26T22:10:35.386617+0000","last_became_active":"2024-04-26T22:00:06.824091+0000","last_became_peered":"2024-04-26T22:00:06.824091+0000","last_unstale":"2024-04-26T22:10:35.386617+0000","last_undegraded":"2024-04-26T22:10:35.386617+0000","last_fullsized":"2024-04-26T22:10:35.386617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4157","last_scrub_stamp":"2024-04-26T22:10:35.386499+0000","last_deep_scrub":"40'4157","last_deep_scrub_stamp":"2024-04-26T22:10:35.386499+0000","last_clean_scrub_stamp":"2024-04-26T22:10:35.386499+0000","objects_scrubbed":0,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:31.680679+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":62491,"num_write_kb":760147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"40'5861","reported_seq":5913,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.426860+0000","last_change":"2024-04-26T22:10:36.426889+0000","last_active":"2024-04-26T22:10:36.426860+0000","last_peered":"2024-04-26T22:10:36.426860+0000","last_clean":"2024-04-26T22:10:36.426860+0000","last_became_active":"2024-04-26T22:00:38.946980+0000","last_became_peered":"2024-04-26T22:00:38.946980+0000","last_unstale":"2024-04-26T22:10:36.426860+0000","last_undegraded":"2024-04-26T22:10:36.426860+0000","last_fullsized":"2024-04-26T22:10:36.426860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5861","last_scrub_stamp":"2024-04-26T22:10:36.426732+0000","last_deep_scrub":"40'5861","last_deep_scrub_stamp":"2024-04-26T22:10:36.426732+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.426732+0000","objects_scrubbed":0,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:46.449932+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88698,"num_write_kb":1108811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"40'4156","reported_seq":4237,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:50.003596+0000","last_change":"2024-04-26T22:10:50.003643+0000","last_active":"2024-04-26T22:10:50.003596+0000","last_peered":"2024-04-26T22:10:50.003596+0000","last_clean":"2024-04-26T22:10:50.003596+0000","last_became_active":"2024-04-26T22:00:06.821614+0000","last_became_peered":"2024-04-26T22:00:06.821614+0000","last_unstale":"2024-04-26T22:10:50.003596+0000","last_undegraded":"2024-04-26T22:10:50.003596+0000","last_fullsized":"2024-04-26T22:10:50.003596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4156","last_scrub_stamp":"2024-04-26T22:10:50.003467+0000","last_deep_scrub":"40'4156","last_deep_scrub_stamp":"2024-04-26T22:10:50.003467+0000","last_clean_scrub_stamp":"2024-04-26T22:10:50.003467+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:49.301624+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62195,"num_write_kb":734457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"40'5569","reported_seq":5638,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.136600+0000","last_change":"2024-04-26T22:10:22.136624+0000","last_active":"2024-04-26T22:10:22.136600+0000","last_peered":"2024-04-26T22:10:22.136600+0000","last_clean":"2024-04-26T22:10:22.136600+0000","last_became_active":"2024-04-26T21:59:57.647532+0000","last_became_peered":"2024-04-26T21:59:57.647532+0000","last_unstale":"2024-04-26T22:10:22.136600+0000","last_undegraded":"2024-04-26T22:10:22.136600+0000","last_fullsized":"2024-04-26T22:10:22.136600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5569","last_scrub_stamp":"2024-04-26T22:10:22.136506+0000","last_deep_scrub":"40'5569","last_deep_scrub_stamp":"2024-04-26T22:10:22.136506+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.136506+0000","objects_scrubbed":0,"log_size":5569,"log_dups_size":0,"ondisk_log_size":5569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:22.727401+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":83899,"num_write_kb":1024385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"40'4460","reported_seq":4540,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.929257+0000","last_change":"2024-04-26T22:10:40.929285+0000","last_active":"2024-04-26T22:10:40.929257+0000","last_peered":"2024-04-26T22:10:40.929257+0000","last_clean":"2024-04-26T22:10:40.929257+0000","last_became_active":"2024-04-26T22:00:06.817959+0000","last_became_peered":"2024-04-26T22:00:06.817959+0000","last_unstale":"2024-04-26T22:10:40.929257+0000","last_undegraded":"2024-04-26T22:10:40.929257+0000","last_fullsized":"2024-04-26T22:10:40.929257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4460","last_scrub_stamp":"2024-04-26T22:10:40.929146+0000","last_deep_scrub":"40'4460","last_deep_scrub_stamp":"2024-04-26T22:10:40.929146+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.929146+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:43.758847+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":67080,"num_write_kb":827171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"40'5284","reported_seq":5351,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.333765+0000","last_change":"2024-04-26T22:10:34.333790+0000","last_active":"2024-04-26T22:10:34.333765+0000","last_peered":"2024-04-26T22:10:34.333765+0000","last_clean":"2024-04-26T22:10:34.333765+0000","last_became_active":"2024-04-26T21:59:57.637972+0000","last_became_peered":"2024-04-26T21:59:57.637972+0000","last_unstale":"2024-04-26T22:10:34.333765+0000","last_undegraded":"2024-04-26T22:10:34.333765+0000","last_fullsized":"2024-04-26T22:10:34.333765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5284","last_scrub_stamp":"2024-04-26T22:10:34.333648+0000","last_deep_scrub":"40'5284","last_deep_scrub_stamp":"2024-04-26T22:10:34.333648+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.333648+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:30.627828+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":79924,"num_write_kb":1000899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"40'4361","reported_seq":4450,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.884600+0000","last_change":"2024-04-26T22:10:38.884652+0000","last_active":"2024-04-26T22:10:38.884600+0000","last_peered":"2024-04-26T22:10:38.884600+0000","last_clean":"2024-04-26T22:10:38.884600+0000","last_became_active":"2024-04-26T22:00:06.815791+0000","last_became_peered":"2024-04-26T22:00:06.815791+0000","last_unstale":"2024-04-26T22:10:38.884600+0000","last_undegraded":"2024-04-26T22:10:38.884600+0000","last_fullsized":"2024-04-26T22:10:38.884600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4361","last_scrub_stamp":"2024-04-26T22:10:38.884458+0000","last_deep_scrub":"40'4361","last_deep_scrub_stamp":"2024-04-26T22:10:38.884458+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.884458+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:11.214750+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":65464,"num_write_kb":794873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"40'5458","reported_seq":5525,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.302249+0000","last_change":"2024-04-26T22:10:32.302278+0000","last_active":"2024-04-26T22:10:32.302249+0000","last_peered":"2024-04-26T22:10:32.302249+0000","last_clean":"2024-04-26T22:10:32.302249+0000","last_became_active":"2024-04-26T21:59:57.654774+0000","last_became_peered":"2024-04-26T21:59:57.654774+0000","last_unstale":"2024-04-26T22:10:32.302249+0000","last_undegraded":"2024-04-26T22:10:32.302249+0000","last_fullsized":"2024-04-26T22:10:32.302249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5458","last_scrub_stamp":"2024-04-26T22:10:32.302152+0000","last_deep_scrub":"40'5458","last_deep_scrub_stamp":"2024-04-26T22:10:32.302152+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.302152+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:08.119475+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":82585,"num_write_kb":1000929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"40'3624","reported_seq":3699,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.235149+0000","last_change":"2024-04-26T22:10:27.235175+0000","last_active":"2024-04-26T22:10:27.235149+0000","last_peered":"2024-04-26T22:10:27.235149+0000","last_clean":"2024-04-26T22:10:27.235149+0000","last_became_active":"2024-04-26T22:00:06.817425+0000","last_became_peered":"2024-04-26T22:00:06.817425+0000","last_unstale":"2024-04-26T22:10:27.235149+0000","last_undegraded":"2024-04-26T22:10:27.235149+0000","last_fullsized":"2024-04-26T22:10:27.235149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3624","last_scrub_stamp":"2024-04-26T22:10:27.235043+0000","last_deep_scrub":"40'3624","last_deep_scrub_stamp":"2024-04-26T22:10:27.235043+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.235043+0000","objects_scrubbed":0,"log_size":3624,"log_dups_size":0,"ondisk_log_size":3624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:00:00.719541+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":0,"num_write":54495,"num_write_kb":662506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"40'4431","reported_seq":4504,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.594748+0000","last_change":"2024-04-26T22:10:27.594797+0000","last_active":"2024-04-26T22:10:27.594748+0000","last_peered":"2024-04-26T22:10:27.594748+0000","last_clean":"2024-04-26T22:10:27.594748+0000","last_became_active":"2024-04-26T21:59:57.651515+0000","last_became_peered":"2024-04-26T21:59:57.651515+0000","last_unstale":"2024-04-26T22:10:27.594748+0000","last_undegraded":"2024-04-26T22:10:27.594748+0000","last_fullsized":"2024-04-26T22:10:27.594748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4431","last_scrub_stamp":"2024-04-26T22:10:27.594561+0000","last_deep_scrub":"40'4431","last_deep_scrub_stamp":"2024-04-26T22:10:27.594561+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.594561+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:40.687251+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":66582,"num_write_kb":814643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"40'4089","reported_seq":4174,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.319106+0000","last_change":"2024-04-26T22:10:40.319135+0000","last_active":"2024-04-26T22:10:40.319106+0000","last_peered":"2024-04-26T22:10:40.319106+0000","last_clean":"2024-04-26T22:10:40.319106+0000","last_became_active":"2024-04-26T22:00:06.818079+0000","last_became_peered":"2024-04-26T22:00:06.818079+0000","last_unstale":"2024-04-26T22:10:40.319106+0000","last_undegraded":"2024-04-26T22:10:40.319106+0000","last_fullsized":"2024-04-26T22:10:40.319106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4089","last_scrub_stamp":"2024-04-26T22:10:40.318986+0000","last_deep_scrub":"40'4089","last_deep_scrub_stamp":"2024-04-26T22:10:40.318986+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.318986+0000","objects_scrubbed":0,"log_size":4089,"log_dups_size":0,"ondisk_log_size":4089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:44.952746+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":61370,"num_write_kb":735117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"40'6428","reported_seq":6496,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.245094+0000","last_change":"2024-04-26T22:10:26.245119+0000","last_active":"2024-04-26T22:10:26.245094+0000","last_peered":"2024-04-26T22:10:26.245094+0000","last_clean":"2024-04-26T22:10:26.245094+0000","last_became_active":"2024-04-26T21:59:57.652291+0000","last_became_peered":"2024-04-26T21:59:57.652291+0000","last_unstale":"2024-04-26T22:10:26.245094+0000","last_undegraded":"2024-04-26T22:10:26.245094+0000","last_fullsized":"2024-04-26T22:10:26.245094+0000","mapping_epoch":18,"log_start":"28'500","ondisk_log_start":"28'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6428","last_scrub_stamp":"2024-04-26T22:10:26.244996+0000","last_deep_scrub":"40'6428","last_deep_scrub_stamp":"2024-04-26T22:10:26.244996+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.244996+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:36.268196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":97094,"num_write_kb":1219735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"40'4506","reported_seq":4597,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:38.384356+0000","last_change":"2024-04-26T22:10:38.384380+0000","last_active":"2024-04-26T22:10:38.384356+0000","last_peered":"2024-04-26T22:10:38.384356+0000","last_clean":"2024-04-26T22:10:38.384356+0000","last_became_active":"2024-04-26T22:00:06.817448+0000","last_became_peered":"2024-04-26T22:00:06.817448+0000","last_unstale":"2024-04-26T22:10:38.384356+0000","last_undegraded":"2024-04-26T22:10:38.384356+0000","last_fullsized":"2024-04-26T22:10:38.384356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4506","last_scrub_stamp":"2024-04-26T22:10:38.384255+0000","last_deep_scrub":"40'4506","last_deep_scrub_stamp":"2024-04-26T22:10:38.384255+0000","last_clean_scrub_stamp":"2024-04-26T22:10:38.384255+0000","objects_scrubbed":0,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:56.799825+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":0,"num_write":67596,"num_write_kb":862862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"40'4747","reported_seq":4814,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.266486+0000","last_change":"2024-04-26T22:10:24.266514+0000","last_active":"2024-04-26T22:10:24.266486+0000","last_peered":"2024-04-26T22:10:24.266486+0000","last_clean":"2024-04-26T22:10:24.266486+0000","last_became_active":"2024-04-26T21:59:57.658070+0000","last_became_peered":"2024-04-26T21:59:57.658070+0000","last_unstale":"2024-04-26T22:10:24.266486+0000","last_undegraded":"2024-04-26T22:10:24.266486+0000","last_fullsized":"2024-04-26T22:10:24.266486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4747","last_scrub_stamp":"2024-04-26T22:10:24.266388+0000","last_deep_scrub":"40'4747","last_deep_scrub_stamp":"2024-04-26T22:10:24.266388+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.266388+0000","objects_scrubbed":0,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:41.401313+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":71795,"num_write_kb":917779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"40'4901","reported_seq":4985,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:42.849302+0000","last_change":"2024-04-26T22:10:42.849331+0000","last_active":"2024-04-26T22:10:42.849302+0000","last_peered":"2024-04-26T22:10:42.849302+0000","last_clean":"2024-04-26T22:10:42.849302+0000","last_became_active":"2024-04-26T22:00:06.822128+0000","last_became_peered":"2024-04-26T22:00:06.822128+0000","last_unstale":"2024-04-26T22:10:42.849302+0000","last_undegraded":"2024-04-26T22:10:42.849302+0000","last_fullsized":"2024-04-26T22:10:42.849302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4901","last_scrub_stamp":"2024-04-26T22:10:42.849192+0000","last_deep_scrub":"40'4901","last_deep_scrub_stamp":"2024-04-26T22:10:42.849192+0000","last_clean_scrub_stamp":"2024-04-26T22:10:42.849192+0000","objects_scrubbed":0,"log_size":4901,"log_dups_size":0,"ondisk_log_size":4901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:28.445524+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":0,"num_write":73795,"num_write_kb":926135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"40'5581","reported_seq":5648,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.353780+0000","last_change":"2024-04-26T22:10:36.353808+0000","last_active":"2024-04-26T22:10:36.353780+0000","last_peered":"2024-04-26T22:10:36.353780+0000","last_clean":"2024-04-26T22:10:36.353780+0000","last_became_active":"2024-04-26T21:59:57.645304+0000","last_became_peered":"2024-04-26T21:59:57.645304+0000","last_unstale":"2024-04-26T22:10:36.353780+0000","last_undegraded":"2024-04-26T22:10:36.353780+0000","last_fullsized":"2024-04-26T22:10:36.353780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5581","last_scrub_stamp":"2024-04-26T22:10:36.353677+0000","last_deep_scrub":"40'5581","last_deep_scrub_stamp":"2024-04-26T22:10:36.353677+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.353677+0000","objects_scrubbed":0,"log_size":5581,"log_dups_size":0,"ondisk_log_size":5581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:22.186045+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":84297,"num_write_kb":1026829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"40'3600","reported_seq":3680,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:46.088155+0000","last_change":"2024-04-26T22:10:46.088186+0000","last_active":"2024-04-26T22:10:46.088155+0000","last_peered":"2024-04-26T22:10:46.088155+0000","last_clean":"2024-04-26T22:10:46.088155+0000","last_became_active":"2024-04-26T22:00:06.832399+0000","last_became_peered":"2024-04-26T22:00:06.832399+0000","last_unstale":"2024-04-26T22:10:46.088155+0000","last_undegraded":"2024-04-26T22:10:46.088155+0000","last_fullsized":"2024-04-26T22:10:46.088155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3600","last_scrub_stamp":"2024-04-26T22:10:46.088047+0000","last_deep_scrub":"40'3600","last_deep_scrub_stamp":"2024-04-26T22:10:46.088047+0000","last_clean_scrub_stamp":"2024-04-26T22:10:46.088047+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.633724+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":0,"num_write":54091,"num_write_kb":649095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"40'5612","reported_seq":5680,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:30.152606+0000","last_change":"2024-04-26T22:10:30.152659+0000","last_active":"2024-04-26T22:10:30.152606+0000","last_peered":"2024-04-26T22:10:30.152606+0000","last_clean":"2024-04-26T22:10:30.152606+0000","last_became_active":"2024-04-26T21:59:57.643665+0000","last_became_peered":"2024-04-26T21:59:57.643665+0000","last_unstale":"2024-04-26T22:10:30.152606+0000","last_undegraded":"2024-04-26T22:10:30.152606+0000","last_fullsized":"2024-04-26T22:10:30.152606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5612","last_scrub_stamp":"2024-04-26T22:10:30.152464+0000","last_deep_scrub":"40'5612","last_deep_scrub_stamp":"2024-04-26T22:10:30.152464+0000","last_clean_scrub_stamp":"2024-04-26T22:10:30.152464+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:16.237743+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":84765,"num_write_kb":1047612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"40'3974","reported_seq":4062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:47.097969+0000","last_change":"2024-04-26T22:10:47.098007+0000","last_active":"2024-04-26T22:10:47.097969+0000","last_peered":"2024-04-26T22:10:47.097969+0000","last_clean":"2024-04-26T22:10:47.097969+0000","last_became_active":"2024-04-26T22:00:06.822881+0000","last_became_peered":"2024-04-26T22:00:06.822881+0000","last_unstale":"2024-04-26T22:10:47.097969+0000","last_undegraded":"2024-04-26T22:10:47.097969+0000","last_fullsized":"2024-04-26T22:10:47.097969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3974","last_scrub_stamp":"2024-04-26T22:10:47.097869+0000","last_deep_scrub":"40'3974","last_deep_scrub_stamp":"2024-04-26T22:10:47.097869+0000","last_clean_scrub_stamp":"2024-04-26T22:10:47.097869+0000","objects_scrubbed":0,"log_size":3974,"log_dups_size":0,"ondisk_log_size":3974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:27.938847+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":59852,"num_write_kb":718261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"40'4819","reported_seq":4890,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:36.027511+0000","last_change":"2024-04-26T22:10:36.027563+0000","last_active":"2024-04-26T22:10:36.027511+0000","last_peered":"2024-04-26T22:10:36.027511+0000","last_clean":"2024-04-26T22:10:36.027511+0000","last_became_active":"2024-04-26T21:59:57.653193+0000","last_became_peered":"2024-04-26T21:59:57.653193+0000","last_unstale":"2024-04-26T22:10:36.027511+0000","last_undegraded":"2024-04-26T22:10:36.027511+0000","last_fullsized":"2024-04-26T22:10:36.027511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4819","last_scrub_stamp":"2024-04-26T22:10:36.027399+0000","last_deep_scrub":"40'4819","last_deep_scrub_stamp":"2024-04-26T22:10:36.027399+0000","last_clean_scrub_stamp":"2024-04-26T22:10:36.027399+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:25.329571+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":72748,"num_write_kb":932622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"40'4140","reported_seq":4229,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:46.959858+0000","last_change":"2024-04-26T22:10:46.959907+0000","last_active":"2024-04-26T22:10:46.959858+0000","last_peered":"2024-04-26T22:10:46.959858+0000","last_clean":"2024-04-26T22:10:46.959858+0000","last_became_active":"2024-04-26T22:00:06.815077+0000","last_became_peered":"2024-04-26T22:00:06.815077+0000","last_unstale":"2024-04-26T22:10:46.959858+0000","last_undegraded":"2024-04-26T22:10:46.959858+0000","last_fullsized":"2024-04-26T22:10:46.959858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4140","last_scrub_stamp":"2024-04-26T22:10:46.959708+0000","last_deep_scrub":"40'4140","last_deep_scrub_stamp":"2024-04-26T22:10:46.959708+0000","last_clean_scrub_stamp":"2024-04-26T22:10:46.959708+0000","objects_scrubbed":0,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:35.338882+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":62174,"num_write_kb":775596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"40'4503","reported_seq":4571,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.829753+0000","last_change":"2024-04-26T22:10:29.829807+0000","last_active":"2024-04-26T22:10:29.829753+0000","last_peered":"2024-04-26T22:10:29.829753+0000","last_clean":"2024-04-26T22:10:29.829753+0000","last_became_active":"2024-04-26T21:59:57.637788+0000","last_became_peered":"2024-04-26T21:59:57.637788+0000","last_unstale":"2024-04-26T22:10:29.829753+0000","last_undegraded":"2024-04-26T22:10:29.829753+0000","last_fullsized":"2024-04-26T22:10:29.829753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4503","last_scrub_stamp":"2024-04-26T22:10:29.829629+0000","last_deep_scrub":"40'4503","last_deep_scrub_stamp":"2024-04-26T22:10:29.829629+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.829629+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:26.123809+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":68209,"num_write_kb":848035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"40'3768","reported_seq":3857,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:21.462225+0000","last_change":"2024-04-26T22:10:21.462249+0000","last_active":"2024-04-26T22:10:21.462225+0000","last_peered":"2024-04-26T22:10:21.462225+0000","last_clean":"2024-04-26T22:10:21.462225+0000","last_became_active":"2024-04-26T22:00:06.815214+0000","last_became_peered":"2024-04-26T22:00:06.815214+0000","last_unstale":"2024-04-26T22:10:21.462225+0000","last_undegraded":"2024-04-26T22:10:21.462225+0000","last_fullsized":"2024-04-26T22:10:21.462225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3768","last_scrub_stamp":"2024-04-26T22:10:21.462125+0000","last_deep_scrub":"40'3768","last_deep_scrub_stamp":"2024-04-26T22:10:21.462125+0000","last_clean_scrub_stamp":"2024-04-26T22:10:21.462125+0000","objects_scrubbed":0,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:39.714111+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":56583,"num_write_kb":708038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"40'5303","reported_seq":5368,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:24.423568+0000","last_change":"2024-04-26T22:10:24.423594+0000","last_active":"2024-04-26T22:10:24.423568+0000","last_peered":"2024-04-26T22:10:24.423568+0000","last_clean":"2024-04-26T22:10:24.423568+0000","last_became_active":"2024-04-26T21:59:57.644953+0000","last_became_peered":"2024-04-26T21:59:57.644953+0000","last_unstale":"2024-04-26T22:10:24.423568+0000","last_undegraded":"2024-04-26T22:10:24.423568+0000","last_fullsized":"2024-04-26T22:10:24.423568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5303","last_scrub_stamp":"2024-04-26T22:10:24.423454+0000","last_deep_scrub":"40'5303","last_deep_scrub_stamp":"2024-04-26T22:10:24.423454+0000","last_clean_scrub_stamp":"2024-04-26T22:10:24.423454+0000","objects_scrubbed":0,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:25.014349+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80229,"num_write_kb":963753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"40'3667","reported_seq":3754,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:31.266169+0000","last_change":"2024-04-26T22:10:31.266196+0000","last_active":"2024-04-26T22:10:31.266169+0000","last_peered":"2024-04-26T22:10:31.266169+0000","last_clean":"2024-04-26T22:10:31.266169+0000","last_became_active":"2024-04-26T22:00:06.818374+0000","last_became_peered":"2024-04-26T22:00:06.818374+0000","last_unstale":"2024-04-26T22:10:31.266169+0000","last_undegraded":"2024-04-26T22:10:31.266169+0000","last_fullsized":"2024-04-26T22:10:31.266169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3667","last_scrub_stamp":"2024-04-26T22:10:31.266071+0000","last_deep_scrub":"40'3667","last_deep_scrub_stamp":"2024-04-26T22:10:31.266071+0000","last_clean_scrub_stamp":"2024-04-26T22:10:31.266071+0000","objects_scrubbed":0,"log_size":3667,"log_dups_size":0,"ondisk_log_size":3667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:05.340055+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":55047,"num_write_kb":687611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"40'6069","reported_seq":6137,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:23.459956+0000","last_change":"2024-04-26T22:10:23.459996+0000","last_active":"2024-04-26T22:10:23.459956+0000","last_peered":"2024-04-26T22:10:23.459956+0000","last_clean":"2024-04-26T22:10:23.459956+0000","last_became_active":"2024-04-26T21:59:57.647338+0000","last_became_peered":"2024-04-26T21:59:57.647338+0000","last_unstale":"2024-04-26T22:10:23.459956+0000","last_undegraded":"2024-04-26T22:10:23.459956+0000","last_fullsized":"2024-04-26T22:10:23.459956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6069","last_scrub_stamp":"2024-04-26T22:10:23.459828+0000","last_deep_scrub":"40'6069","last_deep_scrub_stamp":"2024-04-26T22:10:23.459828+0000","last_clean_scrub_stamp":"2024-04-26T22:10:23.459828+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:35.043456+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":91606,"num_write_kb":1169870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"40'4173","reported_seq":4261,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:29.232685+0000","last_change":"2024-04-26T22:10:29.232712+0000","last_active":"2024-04-26T22:10:29.232685+0000","last_peered":"2024-04-26T22:10:29.232685+0000","last_clean":"2024-04-26T22:10:29.232685+0000","last_became_active":"2024-04-26T22:00:06.821600+0000","last_became_peered":"2024-04-26T22:00:06.821600+0000","last_unstale":"2024-04-26T22:10:29.232685+0000","last_undegraded":"2024-04-26T22:10:29.232685+0000","last_fullsized":"2024-04-26T22:10:29.232685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4173","last_scrub_stamp":"2024-04-26T22:10:29.232579+0000","last_deep_scrub":"40'4173","last_deep_scrub_stamp":"2024-04-26T22:10:29.232579+0000","last_clean_scrub_stamp":"2024-04-26T22:10:29.232579+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:28.145004+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":0,"num_write":62684,"num_write_kb":755589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"40'6512","reported_seq":6583,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:50.130722+0000","last_change":"2024-04-26T22:10:50.130763+0000","last_active":"2024-04-26T22:10:50.130722+0000","last_peered":"2024-04-26T22:10:50.130722+0000","last_clean":"2024-04-26T22:10:50.130722+0000","last_became_active":"2024-04-26T22:00:39.200691+0000","last_became_peered":"2024-04-26T22:00:39.200691+0000","last_unstale":"2024-04-26T22:10:50.130722+0000","last_undegraded":"2024-04-26T22:10:50.130722+0000","last_fullsized":"2024-04-26T22:10:50.130722+0000","mapping_epoch":25,"log_start":"29'700","ondisk_log_start":"29'700","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6512","last_scrub_stamp":"2024-04-26T22:10:50.130591+0000","last_deep_scrub":"40'6512","last_deep_scrub_stamp":"2024-04-26T22:10:50.130591+0000","last_clean_scrub_stamp":"2024-04-26T22:10:50.130591+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:20.938003+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":98505,"num_write_kb":1266779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"40'5161","reported_seq":5257,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:45.917559+0000","last_change":"2024-04-26T22:10:45.917587+0000","last_active":"2024-04-26T22:10:45.917559+0000","last_peered":"2024-04-26T22:10:45.917559+0000","last_clean":"2024-04-26T22:10:45.917559+0000","last_became_active":"2024-04-26T22:00:06.814087+0000","last_became_peered":"2024-04-26T22:00:06.814087+0000","last_unstale":"2024-04-26T22:10:45.917559+0000","last_undegraded":"2024-04-26T22:10:45.917559+0000","last_fullsized":"2024-04-26T22:10:45.917559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5161","last_scrub_stamp":"2024-04-26T22:10:45.917448+0000","last_deep_scrub":"40'5161","last_deep_scrub_stamp":"2024-04-26T22:10:45.917448+0000","last_clean_scrub_stamp":"2024-04-26T22:10:45.917448+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:13.463125+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":77820,"num_write_kb":957388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"40'5846","reported_seq":5912,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:42.049812+0000","last_change":"2024-04-26T22:10:42.049884+0000","last_active":"2024-04-26T22:10:42.049812+0000","last_peered":"2024-04-26T22:10:42.049812+0000","last_clean":"2024-04-26T22:10:42.049812+0000","last_became_active":"2024-04-26T21:59:57.648298+0000","last_became_peered":"2024-04-26T21:59:57.648298+0000","last_unstale":"2024-04-26T22:10:42.049812+0000","last_undegraded":"2024-04-26T22:10:42.049812+0000","last_fullsized":"2024-04-26T22:10:42.049812+0000","mapping_epoch":18,"log_start":"29'600","ondisk_log_start":"29'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5846","last_scrub_stamp":"2024-04-26T22:10:42.049701+0000","last_deep_scrub":"40'5846","last_deep_scrub_stamp":"2024-04-26T22:10:42.049701+0000","last_clean_scrub_stamp":"2024-04-26T22:10:42.049701+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:53.853737+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":88593,"num_write_kb":1072922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"40'3484","reported_seq":3562,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:42.300191+0000","last_change":"2024-04-26T22:10:42.300236+0000","last_active":"2024-04-26T22:10:42.300191+0000","last_peered":"2024-04-26T22:10:42.300191+0000","last_clean":"2024-04-26T22:10:42.300191+0000","last_became_active":"2024-04-26T22:00:06.822259+0000","last_became_peered":"2024-04-26T22:00:06.822259+0000","last_unstale":"2024-04-26T22:10:42.300191+0000","last_undegraded":"2024-04-26T22:10:42.300191+0000","last_fullsized":"2024-04-26T22:10:42.300191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3484","last_scrub_stamp":"2024-04-26T22:10:42.300054+0000","last_deep_scrub":"40'3484","last_deep_scrub_stamp":"2024-04-26T22:10:42.300054+0000","last_clean_scrub_stamp":"2024-04-26T22:10:42.300054+0000","objects_scrubbed":0,"log_size":3484,"log_dups_size":0,"ondisk_log_size":3484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:18.771969+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":0,"num_write":52218,"num_write_kb":635931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"40'5213","reported_seq":5282,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:26.212704+0000","last_change":"2024-04-26T22:10:26.212754+0000","last_active":"2024-04-26T22:10:26.212704+0000","last_peered":"2024-04-26T22:10:26.212704+0000","last_clean":"2024-04-26T22:10:26.212704+0000","last_became_active":"2024-04-26T21:59:57.656495+0000","last_became_peered":"2024-04-26T21:59:57.656495+0000","last_unstale":"2024-04-26T22:10:26.212704+0000","last_undegraded":"2024-04-26T22:10:26.212704+0000","last_fullsized":"2024-04-26T22:10:26.212704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5213","last_scrub_stamp":"2024-04-26T22:10:26.212563+0000","last_deep_scrub":"40'5213","last_deep_scrub_stamp":"2024-04-26T22:10:26.212563+0000","last_clean_scrub_stamp":"2024-04-26T22:10:26.212563+0000","objects_scrubbed":0,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:59.766447+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":78898,"num_write_kb":987567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"40'3779","reported_seq":3867,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:39.377643+0000","last_change":"2024-04-26T22:10:39.377705+0000","last_active":"2024-04-26T22:10:39.377643+0000","last_peered":"2024-04-26T22:10:39.377643+0000","last_clean":"2024-04-26T22:10:39.377643+0000","last_became_active":"2024-04-26T22:00:06.820490+0000","last_became_peered":"2024-04-26T22:00:06.820490+0000","last_unstale":"2024-04-26T22:10:39.377643+0000","last_undegraded":"2024-04-26T22:10:39.377643+0000","last_fullsized":"2024-04-26T22:10:39.377643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3779","last_scrub_stamp":"2024-04-26T22:10:39.377496+0000","last_deep_scrub":"40'3779","last_deep_scrub_stamp":"2024-04-26T22:10:39.377496+0000","last_clean_scrub_stamp":"2024-04-26T22:10:39.377496+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:42.207197+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":56904,"num_write_kb":701117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"40'4635","reported_seq":4701,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.204396+0000","last_change":"2024-04-26T22:10:22.204423+0000","last_active":"2024-04-26T22:10:22.204396+0000","last_peered":"2024-04-26T22:10:22.204396+0000","last_clean":"2024-04-26T22:10:22.204396+0000","last_became_active":"2024-04-26T21:59:57.658132+0000","last_became_peered":"2024-04-26T21:59:57.658132+0000","last_unstale":"2024-04-26T22:10:22.204396+0000","last_undegraded":"2024-04-26T22:10:22.204396+0000","last_fullsized":"2024-04-26T22:10:22.204396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4635","last_scrub_stamp":"2024-04-26T22:10:22.204275+0000","last_deep_scrub":"40'4635","last_deep_scrub_stamp":"2024-04-26T22:10:22.204275+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.204275+0000","objects_scrubbed":0,"log_size":4635,"log_dups_size":0,"ondisk_log_size":4635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:08.289555+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":70184,"num_write_kb":869081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"40'4125","reported_seq":4207,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:43.090798+0000","last_change":"2024-04-26T22:10:43.090826+0000","last_active":"2024-04-26T22:10:43.090798+0000","last_peered":"2024-04-26T22:10:43.090798+0000","last_clean":"2024-04-26T22:10:43.090798+0000","last_became_active":"2024-04-26T22:00:06.831467+0000","last_became_peered":"2024-04-26T22:00:06.831467+0000","last_unstale":"2024-04-26T22:10:43.090798+0000","last_undegraded":"2024-04-26T22:10:43.090798+0000","last_fullsized":"2024-04-26T22:10:43.090798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4125","last_scrub_stamp":"2024-04-26T22:10:43.090695+0000","last_deep_scrub":"40'4125","last_deep_scrub_stamp":"2024-04-26T22:10:43.090695+0000","last_clean_scrub_stamp":"2024-04-26T22:10:43.090695+0000","objects_scrubbed":0,"log_size":4125,"log_dups_size":0,"ondisk_log_size":4125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:54.894730+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":62115,"num_write_kb":743778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"40'6598","reported_seq":6670,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:42.130655+0000","last_change":"2024-04-26T22:10:42.130686+0000","last_active":"2024-04-26T22:10:42.130655+0000","last_peered":"2024-04-26T22:10:42.130655+0000","last_clean":"2024-04-26T22:10:42.130655+0000","last_became_active":"2024-04-26T21:59:57.651581+0000","last_became_peered":"2024-04-26T21:59:57.651581+0000","last_unstale":"2024-04-26T22:10:42.130655+0000","last_undegraded":"2024-04-26T22:10:42.130655+0000","last_fullsized":"2024-04-26T22:10:42.130655+0000","mapping_epoch":18,"log_start":"29'800","ondisk_log_start":"29'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6598","last_scrub_stamp":"2024-04-26T22:10:42.130431+0000","last_deep_scrub":"40'6598","last_deep_scrub_stamp":"2024-04-26T22:10:42.130431+0000","last_clean_scrub_stamp":"2024-04-26T22:10:42.130431+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:37:37.213935+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":99507,"num_write_kb":1174132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"40'3584","reported_seq":3662,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:51.050087+0000","last_change":"2024-04-26T22:10:51.050114+0000","last_active":"2024-04-26T22:10:51.050087+0000","last_peered":"2024-04-26T22:10:51.050087+0000","last_clean":"2024-04-26T22:10:51.050087+0000","last_became_active":"2024-04-26T22:00:06.826818+0000","last_became_peered":"2024-04-26T22:00:06.826818+0000","last_unstale":"2024-04-26T22:10:51.050087+0000","last_undegraded":"2024-04-26T22:10:51.050087+0000","last_fullsized":"2024-04-26T22:10:51.050087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3584","last_scrub_stamp":"2024-04-26T22:10:51.049978+0000","last_deep_scrub":"40'3584","last_deep_scrub_stamp":"2024-04-26T22:10:51.049978+0000","last_clean_scrub_stamp":"2024-04-26T22:10:51.049978+0000","objects_scrubbed":0,"log_size":3584,"log_dups_size":0,"ondisk_log_size":3584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:46.494901+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":54000,"num_write_kb":647349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"40'4854","reported_seq":4920,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:40.199686+0000","last_change":"2024-04-26T22:10:40.199710+0000","last_active":"2024-04-26T22:10:40.199686+0000","last_peered":"2024-04-26T22:10:40.199686+0000","last_clean":"2024-04-26T22:10:40.199686+0000","last_became_active":"2024-04-26T21:59:57.648376+0000","last_became_peered":"2024-04-26T21:59:57.648376+0000","last_unstale":"2024-04-26T22:10:40.199686+0000","last_undegraded":"2024-04-26T22:10:40.199686+0000","last_fullsized":"2024-04-26T22:10:40.199686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4854","last_scrub_stamp":"2024-04-26T22:10:40.199586+0000","last_deep_scrub":"40'4854","last_deep_scrub_stamp":"2024-04-26T22:10:40.199586+0000","last_clean_scrub_stamp":"2024-04-26T22:10:40.199586+0000","objects_scrubbed":0,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:04.075740+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73549,"num_write_kb":940613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"40'4744","reported_seq":4842,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:32.378872+0000","last_change":"2024-04-26T22:10:32.378912+0000","last_active":"2024-04-26T22:10:32.378872+0000","last_peered":"2024-04-26T22:10:32.378872+0000","last_clean":"2024-04-26T22:10:32.378872+0000","last_became_active":"2024-04-26T22:00:06.817349+0000","last_became_peered":"2024-04-26T22:00:06.817349+0000","last_unstale":"2024-04-26T22:10:32.378872+0000","last_undegraded":"2024-04-26T22:10:32.378872+0000","last_fullsized":"2024-04-26T22:10:32.378872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4744","last_scrub_stamp":"2024-04-26T22:10:32.378741+0000","last_deep_scrub":"40'4744","last_deep_scrub_stamp":"2024-04-26T22:10:32.378741+0000","last_clean_scrub_stamp":"2024-04-26T22:10:32.378741+0000","objects_scrubbed":0,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:18.464020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":71239,"num_write_kb":843376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"40'5453","reported_seq":5521,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:20.152937+0000","last_change":"2024-04-26T22:10:20.152937+0000","last_active":"2024-04-26T22:10:20.152937+0000","last_peered":"2024-04-26T22:10:20.152937+0000","last_clean":"2024-04-26T22:10:20.152937+0000","last_became_active":"2024-04-26T21:59:57.648133+0000","last_became_peered":"2024-04-26T21:59:57.648133+0000","last_unstale":"2024-04-26T22:10:20.152937+0000","last_undegraded":"2024-04-26T22:10:20.152937+0000","last_fullsized":"2024-04-26T22:10:20.152937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'5453","last_scrub_stamp":"2024-04-26T22:10:20.152763+0000","last_deep_scrub":"40'5453","last_deep_scrub_stamp":"2024-04-26T22:10:20.152763+0000","last_clean_scrub_stamp":"2024-04-26T22:10:20.152763+0000","objects_scrubbed":0,"log_size":5453,"log_dups_size":0,"ondisk_log_size":5453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:16.392610+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":82428,"num_write_kb":1036078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"40'4470","reported_seq":4557,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:27.187484+0000","last_change":"2024-04-26T22:10:27.187511+0000","last_active":"2024-04-26T22:10:27.187484+0000","last_peered":"2024-04-26T22:10:27.187484+0000","last_clean":"2024-04-26T22:10:27.187484+0000","last_became_active":"2024-04-26T22:00:06.815125+0000","last_became_peered":"2024-04-26T22:00:06.815125+0000","last_unstale":"2024-04-26T22:10:27.187484+0000","last_undegraded":"2024-04-26T22:10:27.187484+0000","last_fullsized":"2024-04-26T22:10:27.187484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4470","last_scrub_stamp":"2024-04-26T22:10:27.187360+0000","last_deep_scrub":"40'4470","last_deep_scrub_stamp":"2024-04-26T22:10:27.187360+0000","last_clean_scrub_stamp":"2024-04-26T22:10:27.187360+0000","objects_scrubbed":0,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:43.671876+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":0,"num_write":67360,"num_write_kb":793318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"40'6816","reported_seq":6885,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:22.462659+0000","last_change":"2024-04-26T22:10:22.462659+0000","last_active":"2024-04-26T22:10:22.462659+0000","last_peered":"2024-04-26T22:10:22.462659+0000","last_clean":"2024-04-26T22:10:22.462659+0000","last_became_active":"2024-04-26T21:59:57.656894+0000","last_became_peered":"2024-04-26T21:59:57.656894+0000","last_unstale":"2024-04-26T22:10:22.462659+0000","last_undegraded":"2024-04-26T22:10:22.462659+0000","last_fullsized":"2024-04-26T22:10:22.462659+0000","mapping_epoch":18,"log_start":"29'1100","ondisk_log_start":"29'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'6816","last_scrub_stamp":"2024-04-26T22:10:22.462544+0000","last_deep_scrub":"40'6816","last_deep_scrub_stamp":"2024-04-26T22:10:22.462544+0000","last_clean_scrub_stamp":"2024-04-26T22:10:22.462544+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:24.860454+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":103074,"num_write_kb":1294412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"40'3980","reported_seq":4062,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:48.090569+0000","last_change":"2024-04-26T22:10:48.090595+0000","last_active":"2024-04-26T22:10:48.090569+0000","last_peered":"2024-04-26T22:10:48.090569+0000","last_clean":"2024-04-26T22:10:48.090569+0000","last_became_active":"2024-04-26T22:00:06.823947+0000","last_became_peered":"2024-04-26T22:00:06.823947+0000","last_unstale":"2024-04-26T22:10:48.090569+0000","last_undegraded":"2024-04-26T22:10:48.090569+0000","last_fullsized":"2024-04-26T22:10:48.090569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3980","last_scrub_stamp":"2024-04-26T22:10:48.090457+0000","last_deep_scrub":"40'3980","last_deep_scrub_stamp":"2024-04-26T22:10:48.090457+0000","last_clean_scrub_stamp":"2024-04-26T22:10:48.090457+0000","objects_scrubbed":0,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:36.469630+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":0,"num_write":59871,"num_write_kb":728175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"40'4761","reported_seq":4831,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T22:10:34.234316+0000","last_change":"2024-04-26T22:10:34.234342+0000","last_active":"2024-04-26T22:10:34.234316+0000","last_peered":"2024-04-26T22:10:34.234316+0000","last_clean":"2024-04-26T22:10:34.234316+0000","last_became_active":"2024-04-26T21:59:57.652943+0000","last_became_peered":"2024-04-26T21:59:57.652943+0000","last_unstale":"2024-04-26T22:10:34.234316+0000","last_undegraded":"2024-04-26T22:10:34.234316+0000","last_fullsized":"2024-04-26T22:10:34.234316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'4761","last_scrub_stamp":"2024-04-26T22:10:34.234178+0000","last_deep_scrub":"40'4761","last_deep_scrub_stamp":"2024-04-26T22:10:34.234178+0000","last_clean_scrub_stamp":"2024-04-26T22:10:34.234178+0000","objects_scrubbed":0,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:06.564470+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":72121,"num_write_kb":895283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1800,"num_read_kb":0,"num_write":3978586,"num_write_kb":48381196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":264479,"ondisk_log_size":264479,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":109332168,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":691,"num_read_kb":381190,"num_write":3546,"num_write_kb":109340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":218726400,"data_stored":218664336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3537,"ondisk_log_size":3537,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":190,"num_read_kb":363,"num_write":5266740,"num_write_kb":65270508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338431,"ondisk_log_size":338431,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":143059102,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":553,"num_read_kb":447893,"num_write":4271,"num_write_kb":143037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286179328,"data_stored":286118204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4289,"ondisk_log_size":4289,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529688,"kb_used_data":21200,"kb_used_omap":1,"kb_used_meta":508478,"kb_avail":93842152,"statfs":{"total":96636764160,"available":96094363648,"internally_reserved":0,"allocated":21708800,"data_stored":21470005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":520681925},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":852784,"kb_used_data":66920,"kb_used_omap":1,"kb_used_meta":785854,"kb_avail":93519056,"statfs":{"total":96636764160,"available":95763513344,"internally_reserved":0,"allocated":68526080,"data_stored":68282539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":804714954},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574352,"kb_used_data":74184,"kb_used_omap":1,"kb_used_meta":500158,"kb_avail":93797488,"statfs":{"total":96636764160,"available":96048627712,"internally_reserved":0,"allocated":75964416,"data_stored":75718280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":512162246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794020,"kb_used_data":45596,"kb_used_omap":1,"kb_used_meta":748414,"kb_avail":93577820,"statfs":{"total":96636764160,"available":95823687680,"internally_reserved":0,"allocated":46690304,"data_stored":46445706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":766376389},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882668,"kb_used_data":82468,"kb_used_omap":1,"kb_used_meta":800190,"kb_avail":93489172,"statfs":{"total":96636764160,"available":95732912128,"internally_reserved":0,"allocated":84447232,"data_stored":84193910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":819395014},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900648,"kb_used_data":78560,"kb_used_omap":1,"kb_used_meta":822078,"kb_avail":93471192,"statfs":{"total":96636764160,"available":95714500608,"internally_reserved":0,"allocated":80445440,"data_stored":80191557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":841808327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767416,"kb_used_data":86832,"kb_used_omap":1,"kb_used_meta":680574,"kb_avail":93604424,"statfs":{"total":96636764160,"available":95850930176,"internally_reserved":0,"allocated":88915968,"data_stored":88663873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":696908229},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740368,"kb_used_data":41992,"kb_used_omap":1,"kb_used_meta":698366,"kb_avail":93631472,"statfs":{"total":96636764160,"available":95878627328,"internally_reserved":0,"allocated":42999808,"data_stored":42753566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715127237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29372416,"data_stored":29360766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":38112011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12673024,"data_stored":12671093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33652736,"data_stored":33643285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46149632,"data_stored":46137470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29364224,"data_stored":29360218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":46145536,"data_stored":46138016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50700288,"data_stored":50695345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8585216,"data_stored":8579392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46342144,"data_stored":46328752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37847040,"data_stored":37836920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16875520,"data_stored":16865968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":29368320,"data_stored":29360744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16777306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:11:04.051 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.174+0000 7fa149ab0640 1 Processor -- start 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa149ab0640 1 -- start start 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa149ab0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa14413ffb0 0x7fa1440c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa149ab0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa149ab0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1440c8a30 con 0x7fa144073530 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa149ab0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa144072be0 con 0x7fa14413ffb0 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa149ab0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1440c9640 con 0x7fa14415a280 2024-04-26T22:11:04.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa143fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa144073530 0x7fa144158180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57636/0 (socket says 172.21.15.135:57636) 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.175+0000 7fa143fff640 1 -- 172.21.15.135:0/3064883000 learned_addr learned my addr 172.21.15.135:0/3064883000 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3064883000 >> v2:172.21.15.139:3300/0 conn(0x7fa14413ffb0 0x7fa1440c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa142ffd640 1 --2- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1427fc640 1 -- 172.21.15.135:0/3064883000 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368644863 0 0) 0x7fa1440c8a30 con 0x7fa144073530 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1427fc640 1 -- 172.21.15.135:0/3064883000 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa12c003520 con 0x7fa144073530 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1437fe640 1 -- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 msgr2=0x7fa144072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1437fe640 1 -- 172.21.15.135:0/3064883000 >> v1:172.21.15.135:6789/0 conn(0x7fa144073530 legacy=0x7fa144158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1427fc640 1 -- 172.21.15.135:0/3064883000 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3432372660 0 0) 0x7fa12c003520 con 0x7fa144073530 2024-04-26T22:11:04.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.176+0000 7fa1437fe640 1 -- 172.21.15.135:0/3064883000 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1440c8d10 con 0x7fa14413ffb0 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14413ffb0 0x7fa1440c82a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa1340097a0 tx=0x7fa13402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56ca50741da7fd27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa1427fc640 1 -- 172.21.15.135:0/3064883000 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134037070 con 0x7fa14413ffb0 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa1427fc640 1 -- 172.21.15.135:0/3064883000 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13402ab10 con 0x7fa14413ffb0 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa1427fc640 1 -- 172.21.15.135:0/3064883000 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0019f0 con 0x7fa14413ffb0 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 -- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14413ffb0 msgr2=0x7fa1440c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14413ffb0 0x7fa1440c82a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa1340097a0 tx=0x7fa13402a080 comp rx=0 tx=0).stop 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 -- 172.21.15.135:0/3064883000 shutdown_connections 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3064883000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14413ffb0 0x7fa1440c82a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 -- 172.21.15.135:0/3064883000 >> 172.21.15.135:0/3064883000 conn(0x7fa14406dc10 msgr2=0x7fa14406c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 -- 172.21.15.135:0/3064883000 shutdown_connections 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.177+0000 7fa149ab0640 1 -- 172.21.15.135:0/3064883000 wait complete. 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.178+0000 7fa149ab0640 1 Processor -- start 2024-04-26T22:11:04.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 -- start start 2024-04-26T22:11:04.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa144073530 0x7fa1440ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 0x7fa144137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440c92d0 con 0x7fa14413ffb0 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440c87e0 con 0x7fa144073530 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa149ab0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa144140390 con 0x7fa14415a280 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa142ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 0x7fa144137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa142ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 0x7fa144137500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39282/0 (socket says 172.21.15.135:39282) 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.179+0000 7fa142ffd640 1 -- 172.21.15.135:0/3517183415 learned_addr learned my addr 172.21.15.135:0/3517183415 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa144073530 0x7fa1440ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa143fff640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa142ffd640 1 -- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa142ffd640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa142ffd640 1 -- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa144073530 msgr2=0x7fa1440ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa142ffd640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa144073530 0x7fa1440ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa142ffd640 1 -- 172.21.15.135:0/3517183415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1441ecdc0 con 0x7fa14413ffb0 2024-04-26T22:11:04.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa144073530 0x7fa1440ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa142ffd640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 0x7fa144137500 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa13800e870 tx=0x7fa13800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138017070 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13800c940 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1441ed060 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13800cd30 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.180+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa144156020 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.181+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144076650 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.182+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa13801a070 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.182+0000 7fa12bfff640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa13006c920 0x7fa13006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.182+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fa13808c300 con 0x7fa14413ffb0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.182+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa13006c920 0x7fa13006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.182+0000 7fa1437fe640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa13006c920 0x7fa13006ede0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa134034000 tx=0x7fa13402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.186+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa138059e40 con 0x7fa14413ffb0 2024-04-26T22:11:04.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.362+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fa1440507a0 con 0x7fa14413ffb0 2024-04-26T22:11:04.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.365+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fa138059740 con 0x7fa14413ffb0 2024-04-26T22:11:04.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.367+0000 7fa12bfff640 1 -- 172.21.15.135:0/3517183415 <== mon.0 v2:172.21.15.135:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa13805daf0 con 0x7fa14413ffb0 2024-04-26T22:11:04.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa13006c920 msgr2=0x7fa13006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa13006c920 0x7fa13006ede0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa134034000 tx=0x7fa13402e000 comp rx=0 tx=0).stop 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 msgr2=0x7fa144137500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 0x7fa144137500 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa13800e870 tx=0x7fa13800ed30 comp rx=0 tx=0).stop 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 shutdown_connections 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa13006c920 0x7fa13006ede0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa14415a280 0x7fa144137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14413ffb0 0x7fa144137500 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 --2- 172.21.15.135:0/3517183415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa144073530 0x7fa1440ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 >> 172.21.15.135:0/3517183415 conn(0x7fa14406dc10 msgr2=0x7fa1440b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 shutdown_connections 2024-04-26T22:11:04.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.368+0000 7fa149ab0640 1 -- 172.21.15.135:0/3517183415 wait complete. 2024-04-26T22:11:04.381 INFO:tasks.cephfs.filesystem:Destroying file system a and related pools 2024-04-26T22:11:04.381 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:11:04.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.476+0000 7f405908c640 1 Processor -- start 2024-04-26T22:11:04.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.476+0000 7f405908c640 1 -- start start 2024-04-26T22:11:04.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405908c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f40540720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405908c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f40540c40e0 0x7f4054151e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405908c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4054154ba0 con 0x7f4054152360 2024-04-26T22:11:04.479 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405908c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4054154e80 con 0x7f40540c40e0 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405908c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4054155160 con 0x7f40540c3060 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f40540c40e0 0x7f4054151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f40540c40e0 0x7f4054151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39284/0 (socket says 172.21.15.135:39284) 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405359e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4054152360 0x7f40540c46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57640/0 (socket says 172.21.15.135:57640) 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 -- 172.21.15.135:0/2777123364 learned_addr learned my addr 172.21.15.135:0/2777123364 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 -- 172.21.15.135:0/2777123364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 msgr2=0x7f40540720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f4052d9d640 1 --2- 172.21.15.135:0/2777123364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f40540720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f405259c640 1 -- 172.21.15.135:0/2777123364 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 602454054 0 0) 0x7f4054154ba0 con 0x7f4054152360 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 --2- 172.21.15.135:0/2777123364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f40540720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 -- 172.21.15.135:0/2777123364 >> v1:172.21.15.135:6789/0 conn(0x7f4054152360 legacy=0x7f40540c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.477+0000 7f404bfff640 1 -- 172.21.15.135:0/2777123364 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4054155440 con 0x7f40540c40e0 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f404bfff640 1 --2- 172.21.15.135:0/2777123364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f4054151e20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f403c009870 tx=0x7f403c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c80ce8d26f5273 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.480 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405259c640 1 -- 172.21.15.135:0/2777123364 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c037070 con 0x7f40540c40e0 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405259c640 1 -- 172.21.15.135:0/2777123364 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f403c032440 con 0x7f40540c40e0 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405259c640 1 -- 172.21.15.135:0/2777123364 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c032830 con 0x7f40540c40e0 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 -- 172.21.15.135:0/2777123364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 msgr2=0x7f4054151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 --2- 172.21.15.135:0/2777123364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f4054151e20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f403c009870 tx=0x7f403c02a1c0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 -- 172.21.15.135:0/2777123364 shutdown_connections 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 --2- 172.21.15.135:0/2777123364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f4054151e20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 --2- 172.21.15.135:0/2777123364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f40540720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 -- 172.21.15.135:0/2777123364 >> 172.21.15.135:0/2777123364 conn(0x7f405406c310 msgr2=0x7f405406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 -- 172.21.15.135:0/2777123364 shutdown_connections 2024-04-26T22:11:04.481 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.478+0000 7f405908c640 1 -- 172.21.15.135:0/2777123364 wait complete. 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 Processor -- start 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 -- start start 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f405414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f40541eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4054152360 0x7f40541ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40540c3440 con 0x7f40540c40e0 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40540c3aa0 con 0x7f4054152360 2024-04-26T22:11:04.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.480+0000 7f405908c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40540c35c0 con 0x7f40540c3060 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f40541eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f40541eb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39290/0 (socket says 172.21.15.135:39290) 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 -- 172.21.15.135:0/3644156513 learned_addr learned my addr 172.21.15.135:0/3644156513 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 -- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 msgr2=0x7f405414daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f405414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 -- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4054152360 msgr2=0x7f40541ebc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f405359e640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4054152360 0x7f40541ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4054152360 0x7f40541ebc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 -- 172.21.15.135:0/3644156513 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4054149c20 con 0x7f40540c40e0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404bfff640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f40541eb6e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f403c035040 tx=0x7f403c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c037040 con 0x7f40540c40e0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f403c004230 con 0x7f40540c40e0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405414c460 con 0x7f40540c40e0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c030770 con 0x7f40540c40e0 2024-04-26T22:11:04.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.481+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405414ca20 con 0x7f40540c40e0 2024-04-26T22:11:04.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.482+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40540507a0 con 0x7f40540c40e0 2024-04-26T22:11:04.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.483+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f403c02ad10 con 0x7f40540c40e0 2024-04-26T22:11:04.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.483+0000 7f404b7fe640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f403406c920 0x7f403406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.483+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f403c0ad080 con 0x7f40540c40e0 2024-04-26T22:11:04.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.483+0000 7f4052d9d640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f403406c920 0x7f403406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.483+0000 7f4052d9d640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f403406c920 0x7f403406ede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f404400a810 tx=0x7f4044008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.488 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.487+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f403c07acd0 con 0x7f40540c40e0 2024-04-26T22:11:04.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.697+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f405414bcb0 con 0x7f40540c40e0 2024-04-26T22:11:04.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.697+0000 7f404b7fe640 1 -- 172.21.15.135:0/3644156513 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6500 (secure 0 0 0) 0x7f403c07a5d0 con 0x7f40540c40e0 2024-04-26T22:11:04.699 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:04.699 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1333118834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1333118834},{"type":"v1","addr":"172.21.15.135:6837","nonce":1333118834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:11:04.699 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 81 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.699+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f403406c920 msgr2=0x7f403406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.699+0000 7f405908c640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f403406c920 0x7f403406ede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f404400a810 tx=0x7f4044008040 comp rx=0 tx=0).stop 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.699+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 msgr2=0x7f40541eb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.699+0000 7f405908c640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f40541eb6e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f403c035040 tx=0x7f403c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 shutdown_connections 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f403406c920 0x7f403406ede0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4054152360 0x7f40541ebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40540c40e0 0x7f40541eb6e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 --2- 172.21.15.135:0/3644156513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40540c3060 0x7f405414daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 >> 172.21.15.135:0/3644156513 conn(0x7f405406c310 msgr2=0x7f405414ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 shutdown_connections 2024-04-26T22:11:04.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.700+0000 7f405908c640 1 -- 172.21.15.135:0/3644156513 wait complete. 2024-04-26T22:11:04.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:11:04.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.808+0000 7f4f9a52b640 1 Processor -- start 2024-04-26T22:11:04.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.808+0000 7f4f9a52b640 1 -- start start 2024-04-26T22:11:04.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f9a52b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94157680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f9a52b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4f940ac610 0x7f4f9406f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f9a52b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9415b0b0 con 0x7f4f941540f0 2024-04-26T22:11:04.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f9a52b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4f941571f0 con 0x7f4f940ac610 2024-04-26T22:11:04.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f9a52b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f94157bc0 con 0x7f4f9415ac70 2024-04-26T22:11:04.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f93fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4f941540f0 0x7f4f940c4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:60318/0 (socket says 172.21.15.135:60318) 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f93fff640 1 -- 172.21.15.135:0/841697779 learned_addr learned my addr 172.21.15.135:0/841697779 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f937fe640 1 --2- 172.21.15.135:0/841697779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f98ac8640 1 --2- 172.21.15.135:0/841697779 >> v2:172.21.15.139:3300/0 conn(0x7f4f940ac610 0x7f4f9406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.809+0000 7f4f92ffd640 1 -- 172.21.15.135:0/841697779 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91268498 0 0) 0x7f4f9415b0b0 con 0x7f4f941540f0 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f92ffd640 1 -- 172.21.15.135:0/841697779 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7c003520 con 0x7f4f941540f0 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f937fe640 1 -- 172.21.15.135:0/841697779 >> v2:172.21.15.139:3300/0 conn(0x7f4f940ac610 msgr2=0x7f4f9406f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f937fe640 1 --2- 172.21.15.135:0/841697779 >> v2:172.21.15.139:3300/0 conn(0x7f4f940ac610 0x7f4f9406f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f937fe640 1 -- 172.21.15.135:0/841697779 >> v1:172.21.15.139:6789/0 conn(0x7f4f941540f0 legacy=0x7f4f940c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f937fe640 1 -- 172.21.15.135:0/841697779 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f94156bc0 con 0x7f4f9415ac70 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f98ac8640 1 --2- 172.21.15.135:0/841697779 >> v2:172.21.15.139:3300/0 conn(0x7f4f940ac610 0x7f4f9406f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f937fe640 1 --2- 172.21.15.135:0/841697779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94157680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4f880097a0 tx=0x7f4f8802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0ea121b6b0c467 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f92ffd640 1 -- 172.21.15.135:0/841697779 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88037070 con 0x7f4f9415ac70 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.810+0000 7f4f92ffd640 1 -- 172.21.15.135:0/841697779 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4f8802ae50 con 0x7f4f9415ac70 2024-04-26T22:11:04.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f92ffd640 1 -- 172.21.15.135:0/841697779 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88032640 con 0x7f4f9415ac70 2024-04-26T22:11:04.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 -- 172.21.15.135:0/841697779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 msgr2=0x7f4f94157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/841697779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94157680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4f880097a0 tx=0x7f4f8802a100 comp rx=0 tx=0).stop 2024-04-26T22:11:04.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 -- 172.21.15.135:0/841697779 shutdown_connections 2024-04-26T22:11:04.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/841697779 >> v2:172.21.15.139:3300/0 conn(0x7f4f940ac610 0x7f4f9406f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/841697779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94157680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 -- 172.21.15.135:0/841697779 >> 172.21.15.135:0/841697779 conn(0x7f4f9406c310 msgr2=0x7f4f9406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 -- 172.21.15.135:0/841697779 shutdown_connections 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.811+0000 7f4f9a52b640 1 -- 172.21.15.135:0/841697779 wait complete. 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.812+0000 7f4f9a52b640 1 Processor -- start 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.812+0000 7f4f9a52b640 1 -- start start 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f9a52b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 0x7f4f941e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f9a52b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f941540f0 0x7f4f94159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f9a52b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f9a52b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94156e80 con 0x7f4f940ac610 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f9a52b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f940c5630 con 0x7f4f941540f0 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f9a52b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f941544d0 con 0x7f4f9415ac70 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 0x7f4f941e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 0x7f4f941e0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39302/0 (socket says 172.21.15.135:39302) 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 -- 172.21.15.135:0/2583211554 learned_addr learned my addr 172.21.15.135:0/2583211554 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:04.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f937fe640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f941540f0 0x7f4f94159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f98ac8640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 -- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 msgr2=0x7f4f94150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 -- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f941540f0 msgr2=0x7f4f94159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f941540f0 0x7f4f94159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.813+0000 7f4f93fff640 1 -- 172.21.15.135:0/2583211554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f941e0290 con 0x7f4f940ac610 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.814+0000 7f4f93fff640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 0x7f4f941e0fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c430 tx=0x7f4f8400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.814+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84017070 con 0x7f4f940ac610 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.814+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4f84005ce0 con 0x7f4f940ac610 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.814+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f941e2cf0 con 0x7f4f940ac610 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.814+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84007310 con 0x7f4f940ac610 2024-04-26T22:11:04.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.814+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941e32b0 con 0x7f4f940ac610 2024-04-26T22:11:04.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.815+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f940ad810 con 0x7f4f940ac610 2024-04-26T22:11:04.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.815+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f8401a070 con 0x7f4f940ac610 2024-04-26T22:11:04.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.816+0000 7f4f90ff9640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f8006c920 0x7f4f8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:04.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.816+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f4f8408b7c0 con 0x7f4f940ac610 2024-04-26T22:11:04.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.816+0000 7f4f937fe640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f8006c920 0x7f4f8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:04.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.816+0000 7f4f937fe640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f8006c920 0x7f4f8006ede0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f88034000 tx=0x7f4f8802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:04.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:04.820+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f940ad810 con 0x7f4f940ac610 2024-04-26T22:11:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.030+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f9406e390 con 0x7f4f940ac610 2024-04-26T22:11:05.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.030+0000 7f4f90ff9640 1 -- 172.21.15.135:0/2583211554 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6500 (secure 0 0 0) 0x7f4f9406e390 con 0x7f4f940ac610 2024-04-26T22:11:05.032 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:05.032 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1333118834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1333118834},{"type":"v1","addr":"172.21.15.135:6837","nonce":1333118834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:59:56.985345+0000","modified":"2024-04-26T22:10:01.932108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5653":{"gid":5653,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":14,"addr":"172.21.15.139:6833/112034263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":112034263},{"type":"v1","addr":"172.21.15.139:6833","nonce":112034263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5653,"qdb_cluster":[5653]},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:11:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 81 2024-04-26T22:11:05.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f8006c920 msgr2=0x7f4f8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f8006c920 0x7f4f8006ede0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f88034000 tx=0x7f4f8802e000 comp rx=0 tx=0).stop 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 msgr2=0x7f4f941e0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 0x7f4f941e0fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c430 tx=0x7f4f8400c830 comp rx=0 tx=0).stop 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 shutdown_connections 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4f8006c920 0x7f4f8006ede0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f9415ac70 0x7f4f94150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f941540f0 0x7f4f94159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 --2- 172.21.15.135:0/2583211554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f940ac610 0x7f4f941e0fe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 >> 172.21.15.135:0/2583211554 conn(0x7f4f9406c310 msgr2=0x7f4f940c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 shutdown_connections 2024-04-26T22:11:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.033+0000 7f4f9a52b640 1 -- 172.21.15.135:0/2583211554 wait complete. 2024-04-26T22:11:05.046 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:11:05.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.143+0000 7f6021e6b640 1 Processor -- start 2024-04-26T22:11:05.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.143+0000 7f6021e6b640 1 -- start start 2024-04-26T22:11:05.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f6021e6b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f601c0c5140 0x7f601c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f6021e6b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c0c5520 0x7f601c0ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f6021e6b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f601c06e8e0 0x7f601c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f6021e6b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f601c15a190 con 0x7f601c0c5140 2024-04-26T22:11:05.208 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f6021e6b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f601c15a470 con 0x7f601c06e8e0 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f6021e6b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f601c06f0c0 con 0x7f601c0c5520 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f601c0c5140 0x7f601c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f601c0c5140 0x7f601c156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39304/0 (socket says 172.21.15.135:39304) 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 -- 172.21.15.135:0/3730568410 learned_addr learned my addr 172.21.15.135:0/3730568410 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601affd640 1 --2- 172.21.15.135:0/3730568410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c0c5520 0x7f601c0ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601bfff640 1 --2- 172.21.15.135:0/3730568410 >> v2:172.21.15.139:3300/0 conn(0x7f601c06e8e0 0x7f601c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 -- 172.21.15.135:0/3730568410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c0c5520 msgr2=0x7f601c0ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 --2- 172.21.15.135:0/3730568410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c0c5520 0x7f601c0ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 -- 172.21.15.135:0/3730568410 >> v2:172.21.15.139:3300/0 conn(0x7f601c06e8e0 msgr2=0x7f601c15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 --2- 172.21.15.135:0/3730568410 >> v2:172.21.15.139:3300/0 conn(0x7f601c06e8e0 0x7f601c15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.144+0000 7f601b7fe640 1 -- 172.21.15.135:0/3730568410 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c06f3a0 con 0x7f601c0c5140 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f601b7fe640 1 --2- 172.21.15.135:0/3730568410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5140 0x7f601c156c00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f601c06cef0 tx=0x7f600c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5647fc35c97379 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.209 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f601a7fc640 1 -- 172.21.15.135:0/3730568410 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c037070 con 0x7f601c0c5140 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f601a7fc640 1 -- 172.21.15.135:0/3730568410 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f600c032440 con 0x7f601c0c5140 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f601a7fc640 1 -- 172.21.15.135:0/3730568410 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c032830 con 0x7f601c0c5140 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 -- 172.21.15.135:0/3730568410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5140 msgr2=0x7f601c156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 --2- 172.21.15.135:0/3730568410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5140 0x7f601c156c00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f601c06cef0 tx=0x7f600c02a140 comp rx=0 tx=0).stop 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 -- 172.21.15.135:0/3730568410 shutdown_connections 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 --2- 172.21.15.135:0/3730568410 >> v2:172.21.15.139:3300/0 conn(0x7f601c06e8e0 0x7f601c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 --2- 172.21.15.135:0/3730568410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c0c5520 0x7f601c0ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 --2- 172.21.15.135:0/3730568410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5140 0x7f601c156c00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 -- 172.21.15.135:0/3730568410 >> 172.21.15.135:0/3730568410 conn(0x7f601c06e090 msgr2=0x7f601c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 -- 172.21.15.135:0/3730568410 shutdown_connections 2024-04-26T22:11:05.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.145+0000 7f6021e6b640 1 -- 172.21.15.135:0/3730568410 wait complete. 2024-04-26T22:11:05.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.146+0000 7f6021e6b640 1 Processor -- start 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 -- start start 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c06e8e0 0x7f601c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f601c0c5140 0x7f601c145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 0x7f601c145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f601c15ae40 con 0x7f601c0c5520 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f601c06ed90 con 0x7f601c0c5140 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f6021e6b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f601c15afc0 con 0x7f601c06e8e0 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f601bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 0x7f601c145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f601bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 0x7f601c145eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39316/0 (socket says 172.21.15.135:39316) 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f601bfff640 1 -- 172.21.15.135:0/256464440 learned_addr learned my addr 172.21.15.135:0/256464440 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.147+0000 7f601b7fe640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c06e8e0 0x7f601c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601affd640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f601c0c5140 0x7f601c145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601bfff640 1 -- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c06e8e0 msgr2=0x7f601c14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601bfff640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c06e8e0 0x7f601c14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601bfff640 1 -- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f601c0c5140 msgr2=0x7f601c145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601bfff640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f601c0c5140 0x7f601c145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601bfff640 1 -- 172.21.15.135:0/256464440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c1eda70 con 0x7f601c0c5520 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f601bfff640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 0x7f601c145eb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6014007d20 tx=0x7f601400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014014420 con 0x7f601c0c5520 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6014019070 con 0x7f601c0c5520 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c1463f0 con 0x7f601c0c5520 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014014840 con 0x7f601c0c5520 2024-04-26T22:11:05.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.148+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c1ecf90 con 0x7f601c0c5520 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.149+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c0694b0 con 0x7f601c0c5520 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.150+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60140149e0 con 0x7f601c0c5520 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.150+0000 7f6003fff640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f600806c8f0 0x7f600806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.150+0000 7f601b7fe640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f600806c8f0 0x7f600806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.150+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7f601408cb00 con 0x7f601c0c5520 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.151+0000 7f601b7fe640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f600806c8f0 0x7f600806edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f600c034000 tx=0x7f600c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.155+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f601c0694b0 con 0x7f601c0c5520 2024-04-26T22:11:05.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.331+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f601c0aafb0 con 0x7f601c0c5520 2024-04-26T22:11:05.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.332+0000 7f6003fff640 1 -- 172.21.15.135:0/256464440 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+22533 (secure 0 0 0) 0x7f6014059fe0 con 0x7f601c0c5520 2024-04-26T22:11:05.333 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:05.333 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":40,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:10:00.929237+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-04-26T21:59:55.002813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-04-26T21:59:55.939338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6837/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6836/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6834/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6835/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.135:6837/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.139:6832/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6835/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6834/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6835/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.139:6833/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6836/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6834/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.139:6834/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6834/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.139:6835/302589348":"2024-04-27T22:05:52.818178+0000","172.21.15.139:6832/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.139:6835/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6833/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.135:6836/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6836/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.135:6837/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6835/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.135:6837/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.135:6835/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6834/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6835/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.139:6834/302589348":"2024-04-27T22:05:52.818178+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:11:05.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f600806c8f0 msgr2=0x7f600806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f600806c8f0 0x7f600806edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f600c034000 tx=0x7f600c02e000 comp rx=0 tx=0).stop 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 msgr2=0x7f601c145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 0x7f601c145eb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6014007d20 tx=0x7f601400e4b0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 shutdown_connections 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f600806c8f0 0x7f600806edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f601c0c5520 0x7f601c145eb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f601c0c5140 0x7f601c145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 --2- 172.21.15.135:0/256464440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f601c06e8e0 0x7f601c14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 >> 172.21.15.135:0/256464440 conn(0x7f601c06e090 msgr2=0x7f601c153470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 shutdown_connections 2024-04-26T22:11:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.334+0000 7f6021e6b640 1 -- 172.21.15.135:0/256464440 wait complete. 2024-04-26T22:11:05.347 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail a 2024-04-26T22:11:05.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.444+0000 7fb50c346640 1 Processor -- start 2024-04-26T22:11:05.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.444+0000 7fb50c346640 1 -- start start 2024-04-26T22:11:05.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50c346640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb50413feb0 0x7fb5041576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50c346640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb504156c10 con 0x7fb5040c5050 2024-04-26T22:11:05.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50c346640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5040c5570 con 0x7fb5040c2b80 2024-04-26T22:11:05.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50c346640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb50415a8b0 con 0x7fb50413feb0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50a0e2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb50413feb0 0x7fb5041576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb5098e1640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb5040c5050 0x7fb50415a1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57646/0 (socket says 172.21.15.135:57646) 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50a0e2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb50413feb0 0x7fb5041576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39322/0 (socket says 172.21.15.135:39322) 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb5098e1640 1 -- 172.21.15.135:0/2368652350 learned_addr learned my addr 172.21.15.135:0/2368652350 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50a0e2640 1 -- 172.21.15.135:0/2368652350 >> v1:172.21.15.139:6789/0 conn(0x7fb5040c2b80 legacy=0x7fb50406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50a0e2640 1 -- 172.21.15.135:0/2368652350 >> v1:172.21.15.135:6789/0 conn(0x7fb5040c5050 legacy=0x7fb50415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb5090e0640 1 -- 172.21.15.135:0/2368652350 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969876959 0 0) 0x7fb504156c10 con 0x7fb5040c5050 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.445+0000 7fb50a0e2640 1 -- 172.21.15.135:0/2368652350 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040c5850 con 0x7fb50413feb0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.446+0000 7fb50a0e2640 1 --2- 172.21.15.135:0/2368652350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb5041576d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb500009d90 tx=0x7fb50002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dae103fa6a1d299 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.446+0000 7fb5090e0640 1 -- 172.21.15.135:0/2368652350 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500007740 con 0x7fb50413feb0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.446+0000 7fb5090e0640 1 -- 172.21.15.135:0/2368652350 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb5000078e0 con 0x7fb50413feb0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.446+0000 7fb5090e0640 1 -- 172.21.15.135:0/2368652350 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500007cd0 con 0x7fb50413feb0 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 -- 172.21.15.135:0/2368652350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 msgr2=0x7fb5041576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 --2- 172.21.15.135:0/2368652350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb5041576d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb500009d90 tx=0x7fb50002a580 comp rx=0 tx=0).stop 2024-04-26T22:11:05.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 -- 172.21.15.135:0/2368652350 shutdown_connections 2024-04-26T22:11:05.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 --2- 172.21.15.135:0/2368652350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb5041576d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 -- 172.21.15.135:0/2368652350 >> 172.21.15.135:0/2368652350 conn(0x7fb5040c5ce0 msgr2=0x7fb5040c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:05.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 -- 172.21.15.135:0/2368652350 shutdown_connections 2024-04-26T22:11:05.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.447+0000 7fb50c346640 1 -- 172.21.15.135:0/2368652350 wait complete. 2024-04-26T22:11:05.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.448+0000 7fb50c346640 1 Processor -- start 2024-04-26T22:11:05.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.448+0000 7fb50c346640 1 -- start start 2024-04-26T22:11:05.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5040c2b80 0x7fb5041ec480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5040c5050 0x7fb50413c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb50413cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb50406da30 con 0x7fb50413feb0 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb504157240 con 0x7fb5040c5050 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb50406dbb0 con 0x7fb5040c2b80 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb50413cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb50413cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39336/0 (socket says 172.21.15.135:39336) 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a0e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5040c2b80 0x7fb5041ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 -- 172.21.15.135:0/1670987251 learned_addr learned my addr 172.21.15.135:0/1670987251 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb5098e1640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5040c5050 0x7fb50413c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 -- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5040c2b80 msgr2=0x7fb5041ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5040c2b80 0x7fb5041ec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 -- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5040c5050 msgr2=0x7fb50413c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5040c5050 0x7fb50413c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 -- 172.21.15.135:0/1670987251 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041eccc0 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50a8e3640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb50413cf20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00b410 tx=0x7fb4fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc015030 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb4fc004020 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004470 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.449+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041ecfa0 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.450+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb504140570 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.450+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040507a0 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.451+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4fc018070 con 0x7fb50413feb0 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.451+0000 7fb4faffd640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb4e806c8f0 0x7fb4e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.452+0000 7fb50a0e2640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb4e806c8f0 0x7fb4e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.452+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 7870+0+0 (secure 0 0 0) 0x7fb4fc08bca0 con 0x7fb50413feb0 2024-04-26T22:11:05.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.452+0000 7fb50a0e2640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb4e806c8f0 0x7fb4e806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb500034000 tx=0x7fb50002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.456+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4fc059280 con 0x7fb50413feb0 2024-04-26T22:11:05.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.668+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "a"} v 0) -- 0x7fb50413de10 con 0x7fb50413feb0 2024-04-26T22:11:05.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.670+0000 7fb4faffd640 1 -- 172.21.15.135:0/1670987251 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "a"}]=0 a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 151+0+0 (secure 0 0 0) 0x7fb4fc05d580 con 0x7fb50413feb0 2024-04-26T22:11:05.670 INFO:teuthology.orchestra.run.smithi135.stderr:a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.672+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb4e806c8f0 msgr2=0x7fb4e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.672+0000 7fb50c346640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb4e806c8f0 0x7fb4e806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb500034000 tx=0x7fb50002f000 comp rx=0 tx=0).stop 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.672+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 msgr2=0x7fb50413cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.672+0000 7fb50c346640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb50413cf20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00b410 tx=0x7fb4fc00b8d0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 shutdown_connections 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb4e806c8f0 0x7fb4e806edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb50413feb0 0x7fb50413cf20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5040c5050 0x7fb50413c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 --2- 172.21.15.135:0/1670987251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5040c2b80 0x7fb5041ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 >> 172.21.15.135:0/1670987251 conn(0x7fb5040c5ce0 msgr2=0x7fb5040c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:05.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 shutdown_connections 2024-04-26T22:11:05.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.673+0000 7fb50c346640 1 -- 172.21.15.135:0/1670987251 wait complete. 2024-04-26T22:11:05.686 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm a --yes-i-really-mean-it 2024-04-26T22:11:05.687 INFO:tasks.ceph.mds.d.smithi139.stderr: -2674> 2024-04-26T22:10:06.702+0000 7fe9282db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:11:05.715 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-04-26T22:11:05.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae918c4640 1 Processor -- start 2024-04-26T22:11:05.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae918c4640 1 -- start start 2024-04-26T22:11:05.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae918c4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae8c156bb0 0x7fae8c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae918c4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8c0c8df0 con 0x7fae8c154160 2024-04-26T22:11:05.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae918c4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8c154540 con 0x7fae8c0ca5a0 2024-04-26T22:11:05.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae918c4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fae8c15a6e0 con 0x7fae8c156bb0 2024-04-26T22:11:05.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae8c156bb0 0x7fae8c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae8c156bb0 0x7fae8c15a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39348/0 (socket says 172.21.15.135:39348) 2024-04-26T22:11:05.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8affd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fae8c154160 0x7fae8c0c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57662/0 (socket says 172.21.15.135:57662) 2024-04-26T22:11:05.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8b7fe640 1 -- 172.21.15.135:0/2513425102 learned_addr learned my addr 172.21.15.135:0/2513425102 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:05.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8b7fe640 1 -- 172.21.15.135:0/2513425102 >> v1:172.21.15.139:6789/0 conn(0x7fae8c0ca5a0 legacy=0x7fae8c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8b7fe640 1 -- 172.21.15.135:0/2513425102 >> v1:172.21.15.135:6789/0 conn(0x7fae8c154160 legacy=0x7fae8c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae89ffb640 1 -- 172.21.15.135:0/2513425102 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867390635 0 0) 0x7fae7c001110 con 0x7fae8c154160 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.788+0000 7fae8b7fe640 1 -- 172.21.15.135:0/2513425102 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c15a910 con 0x7fae8c156bb0 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae8b7fe640 1 --2- 172.21.15.135:0/2513425102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c156bb0 0x7fae8c15a1a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0af040 tx=0x7fae8402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed64c30b2ef8adc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae89ffb640 1 -- 172.21.15.135:0/2513425102 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84036070 con 0x7fae8c156bb0 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae89ffb640 1 -- 172.21.15.135:0/2513425102 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fae84031600 con 0x7fae8c156bb0 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae89ffb640 1 -- 172.21.15.135:0/2513425102 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840319f0 con 0x7fae8c156bb0 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 -- 172.21.15.135:0/2513425102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c156bb0 msgr2=0x7fae8c15a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 --2- 172.21.15.135:0/2513425102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c156bb0 0x7fae8c15a1a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0af040 tx=0x7fae8402a730 comp rx=0 tx=0).stop 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 -- 172.21.15.135:0/2513425102 shutdown_connections 2024-04-26T22:11:05.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 --2- 172.21.15.135:0/2513425102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c156bb0 0x7fae8c15a1a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 -- 172.21.15.135:0/2513425102 >> 172.21.15.135:0/2513425102 conn(0x7fae8c0ae010 msgr2=0x7fae8c0ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:05.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 -- 172.21.15.135:0/2513425102 shutdown_connections 2024-04-26T22:11:05.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.789+0000 7fae918c4640 1 -- 172.21.15.135:0/2513425102 wait complete. 2024-04-26T22:11:05.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.790+0000 7fae918c4640 1 Processor -- start 2024-04-26T22:11:05.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 -- start start 2024-04-26T22:11:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c0ca5a0 0x7fae8c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 0x7fae8c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c156bb0 0x7fae8c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c15aed0 con 0x7fae8c154160 2024-04-26T22:11:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c0c9100 con 0x7fae8c0ca5a0 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae918c4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae8c0c7510 con 0x7fae8c156bb0 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 0x7fae8c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 0x7fae8c158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39352/0 (socket says 172.21.15.135:39352) 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8a7fc640 1 -- 172.21.15.135:0/3646718673 learned_addr learned my addr 172.21.15.135:0/3646718673 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8affd640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c0ca5a0 0x7fae8c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8a7fc640 1 -- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c156bb0 msgr2=0x7fae8c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8a7fc640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c156bb0 0x7fae8c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.791+0000 7fae8a7fc640 1 -- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c0ca5a0 msgr2=0x7fae8c152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae8a7fc640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c0ca5a0 0x7fae8c152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae8a7fc640 1 -- 172.21.15.135:0/3646718673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c158db0 con 0x7fae8c154160 2024-04-26T22:11:05.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae8a7fc640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 0x7fae8c158330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c460 tx=0x7fae8000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80016020 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fae80012870 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c155980 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80012c80 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.792+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c154ea0 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.793+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c076650 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.793+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae80019020 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.794+0000 7fae73fff640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae7806c8f0 0x7fae7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.794+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 7956+0+0 (secure 0 0 0) 0x7fae8008bed0 con 0x7fae8c154160 2024-04-26T22:11:05.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.794+0000 7fae8affd640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae7806c8f0 0x7fae7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.794+0000 7fae8affd640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae7806c8f0 0x7fae7806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae7c000f60 tx=0x7fae7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:05.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:05.799+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae80020070 con 0x7fae8c154160 2024-04-26T22:11:06.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.008+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true} v 0) -- 0x7fae8c0507a0 con 0x7fae8c154160 2024-04-26T22:11:06.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.674+0000 7fae73fff640 1 -- 172.21.15.135:0/3646718673 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true}]=0 v83) ==== 99+0+0 (secure 0 0 0) 0x7fae8c158db0 con 0x7fae8c154160 2024-04-26T22:11:06.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae7806c8f0 msgr2=0x7fae7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae7806c8f0 0x7fae7806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae7c000f60 tx=0x7fae7c009040 comp rx=0 tx=0).stop 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 msgr2=0x7fae8c158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 0x7fae8c158330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c460 tx=0x7fae8000c920 comp rx=0 tx=0).stop 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 shutdown_connections 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fae7806c8f0 0x7fae7806edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c156bb0 0x7fae8c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c154160 0x7fae8c158330 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 --2- 172.21.15.135:0/3646718673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c0ca5a0 0x7fae8c152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.675+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 >> 172.21.15.135:0/3646718673 conn(0x7fae8c0ae010 msgr2=0x7fae8c150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.676+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 shutdown_connections 2024-04-26T22:11:06.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.676+0000 7fae918c4640 1 -- 172.21.15.135:0/3646718673 wait complete. 2024-04-26T22:11:06.689 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_metadata a_metadata --yes-i-really-really-mean-it 2024-04-26T22:11:06.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.792+0000 7f9c57d9d640 1 Processor -- start 2024-04-26T22:11:06.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.792+0000 7f9c57d9d640 1 -- start start 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c57d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c500c3d70 0x7f9c500c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c57d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c501541f0 0x7f9c501559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c57d9d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c501511e0 con 0x7f9c5006e720 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c57d9d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5006f540 con 0x7f9c501541f0 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c57d9d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9c500ae0c0 con 0x7f9c500c3d70 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c501541f0 0x7f9c501559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c501541f0 0x7f9c501559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39354/0 (socket says 172.21.15.135:39354) 2024-04-26T22:11:06.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 -- 172.21.15.135:0/2477198845 learned_addr learned my addr 172.21.15.135:0/2477198845 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:06.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 -- 172.21.15.135:0/2477198845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c500c3d70 msgr2=0x7f9c500c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:06.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 --2- 172.21.15.135:0/2477198845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c500c3d70 0x7f9c500c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 -- 172.21.15.135:0/2477198845 >> v1:172.21.15.139:6789/0 conn(0x7f9c5006e720 legacy=0x7f9c501552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:06.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.793+0000 7f9c5633a640 1 -- 172.21.15.135:0/2477198845 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501546f0 con 0x7f9c501541f0 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c5633a640 1 --2- 172.21.15.135:0/2477198845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501559f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009e50 tx=0x7f9c4802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88fc7b1ecc299f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c54b37640 1 -- 172.21.15.135:0/2477198845 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48036070 con 0x7f9c501541f0 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c54b37640 1 -- 172.21.15.135:0/2477198845 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c48031600 con 0x7f9c501541f0 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c54b37640 1 -- 172.21.15.135:0/2477198845 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c480319f0 con 0x7f9c501541f0 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c57d9d640 1 -- 172.21.15.135:0/2477198845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 msgr2=0x7f9c501559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/2477198845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501559f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009e50 tx=0x7f9c4802a630 comp rx=0 tx=0).stop 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c57d9d640 1 -- 172.21.15.135:0/2477198845 shutdown_connections 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/2477198845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501559f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/2477198845 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c500c3d70 0x7f9c500c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.794+0000 7f9c57d9d640 1 -- 172.21.15.135:0/2477198845 >> 172.21.15.135:0/2477198845 conn(0x7f9c5006c310 msgr2=0x7f9c5006c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.795+0000 7f9c57d9d640 1 -- 172.21.15.135:0/2477198845 shutdown_connections 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.795+0000 7f9c57d9d640 1 -- 172.21.15.135:0/2477198845 wait complete. 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 Processor -- start 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 -- start start 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c5006e720 0x7f9c501f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c500c3d70 0x7f9c500c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50155f30 con 0x7f9c501541f0 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5006f260 con 0x7f9c500c3d70 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c57d9d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9c500c2700 con 0x7f9c5006e720 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c5633a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c5633a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501ecba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39364/0 (socket says 172.21.15.135:39364) 2024-04-26T22:11:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.796+0000 7f9c5633a640 1 -- 172.21.15.135:0/3997188525 learned_addr learned my addr 172.21.15.135:0/3997188525 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c55b39640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c5006e720 0x7f9c501f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c55338640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c500c3d70 0x7f9c500c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c5633a640 1 -- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c5006e720 msgr2=0x7f9c501f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c5633a640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c5006e720 0x7f9c501f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c5633a640 1 -- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c500c3d70 msgr2=0x7f9c500c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c5633a640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c500c3d70 0x7f9c500c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c5633a640 1 -- 172.21.15.135:0/3997188525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5014f110 con 0x7f9c501541f0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c55338640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c500c3d70 0x7f9c500c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c5633a640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501ecba0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009e50 tx=0x7f9c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48036050 con 0x7f9c501541f0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c48042070 con 0x7f9c501541f0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48009510 con 0x7f9c501541f0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5014f3f0 con 0x7f9c501541f0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.797+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c501f0490 con 0x7f9c501541f0 2024-04-26T22:11:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.798+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c500694b0 con 0x7f9c501541f0 2024-04-26T22:11:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.798+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c48007c10 con 0x7f9c501541f0 2024-04-26T22:11:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.799+0000 7f9c467fc640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c3406c8f0 0x7f9c3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:06.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.799+0000 7f9c55b39640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c3406c8f0 0x7f9c3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:06.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.799+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 7956+0+0 (secure 0 0 0) 0x7f9c480ad4c0 con 0x7f9c501541f0 2024-04-26T22:11:06.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.799+0000 7f9c55b39640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c3406c8f0 0x7f9c3406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00a810 tx=0x7f9c4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:06.803 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.803+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c4807b020 con 0x7f9c501541f0 2024-04-26T22:11:06.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:06.979+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c500c7f40 con 0x7f9c501541f0 2024-04-26T22:11:07.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.681+0000 7f9c467fc640 1 -- 172.21.15.135:0/3997188525 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'a_metadata' removed v42) ==== 166+0+0 (secure 0 0 0) 0x7f9c4807a920 con 0x7f9c501541f0 2024-04-26T22:11:07.681 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'a_metadata' removed 2024-04-26T22:11:07.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c3406c8f0 msgr2=0x7f9c3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c3406c8f0 0x7f9c3406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00a810 tx=0x7f9c4c008040 comp rx=0 tx=0).stop 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 msgr2=0x7f9c501ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501ecba0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009e50 tx=0x7f9c48004a40 comp rx=0 tx=0).stop 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 shutdown_connections 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f9c3406c8f0 0x7f9c3406edb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c501541f0 0x7f9c501ecba0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c500c3d70 0x7f9c500c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 --2- 172.21.15.135:0/3997188525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c5006e720 0x7f9c501f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.683+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 >> 172.21.15.135:0/3997188525 conn(0x7f9c5006c310 msgr2=0x7f9c500c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.684+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 shutdown_connections 2024-04-26T22:11:07.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.684+0000 7f9c57d9d640 1 -- 172.21.15.135:0/3997188525 wait complete. 2024-04-26T22:11:07.702 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_data a_data --yes-i-really-really-mean-it 2024-04-26T22:11:07.803 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.803+0000 7f724a3e6640 1 Processor -- start 2024-04-26T22:11:07.803 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.803+0000 7f724a3e6640 1 -- start start 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f724a3e6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f72441543b0 0x7f72440ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f724a3e6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f72440c8a10 0x7f72440c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f724a3e6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7244154790 con 0x7f72440cbd80 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f724a3e6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f72440cc210 con 0x7f72440c8a10 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f724a3e6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7244157ca0 con 0x7f72441543b0 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7248983640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f72440c8a10 0x7f72440c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f72437fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f72440cbd80 0x7f7244157590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57674/0 (socket says 172.21.15.135:57674) 2024-04-26T22:11:07.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7248983640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f72440c8a10 0x7f72440c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39376/0 (socket says 172.21.15.135:39376) 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f72437fe640 1 -- 172.21.15.135:0/745084678 learned_addr learned my addr 172.21.15.135:0/745084678 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7243fff640 1 --2- 172.21.15.135:0/745084678 >> v2:172.21.15.139:3300/0 conn(0x7f72441543b0 0x7f72440ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7248983640 1 -- 172.21.15.135:0/745084678 >> v2:172.21.15.139:3300/0 conn(0x7f72441543b0 msgr2=0x7f72440ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7248983640 1 --2- 172.21.15.135:0/745084678 >> v2:172.21.15.139:3300/0 conn(0x7f72441543b0 0x7f72440ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7248983640 1 -- 172.21.15.135:0/745084678 >> v1:172.21.15.135:6789/0 conn(0x7f72440cbd80 legacy=0x7f7244157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.804+0000 7f7248983640 1 -- 172.21.15.135:0/745084678 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724406c4b0 con 0x7f72440c8a10 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.805+0000 7f7248983640 1 --2- 172.21.15.135:0/745084678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440c6f50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7244073c20 tx=0x7f723c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf2871c2f3f020f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.805+0000 7f7242ffd640 1 -- 172.21.15.135:0/745084678 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c007740 con 0x7f72440c8a10 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.805+0000 7f7242ffd640 1 -- 172.21.15.135:0/745084678 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f723c0078e0 con 0x7f72440c8a10 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.805+0000 7f7242ffd640 1 -- 172.21.15.135:0/745084678 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c007cd0 con 0x7f72440c8a10 2024-04-26T22:11:07.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.805+0000 7f724a3e6640 1 -- 172.21.15.135:0/745084678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 msgr2=0x7f72440c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.805+0000 7f724a3e6640 1 --2- 172.21.15.135:0/745084678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440c6f50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7244073c20 tx=0x7f723c02a5f0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.806+0000 7f724a3e6640 1 -- 172.21.15.135:0/745084678 shutdown_connections 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.806+0000 7f724a3e6640 1 --2- 172.21.15.135:0/745084678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440c6f50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.806+0000 7f724a3e6640 1 --2- 172.21.15.135:0/745084678 >> v2:172.21.15.139:3300/0 conn(0x7f72441543b0 0x7f72440ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.806+0000 7f724a3e6640 1 -- 172.21.15.135:0/745084678 >> 172.21.15.135:0/745084678 conn(0x7f72440c8e30 msgr2=0x7f72440c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.806+0000 7f724a3e6640 1 -- 172.21.15.135:0/745084678 shutdown_connections 2024-04-26T22:11:07.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.806+0000 7f724a3e6640 1 -- 172.21.15.135:0/745084678 wait complete. 2024-04-26T22:11:07.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 Processor -- start 2024-04-26T22:11:07.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 -- start start 2024-04-26T22:11:07.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:07.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72441543b0 0x7f724414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:07.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72440af300 0x7f724414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7244156d20 con 0x7f72440c8a10 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7244157070 con 0x7f72441543b0 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.807+0000 7f724a3e6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f72440c7490 con 0x7f72440af300 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39384/0 (socket says 172.21.15.135:39384) 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 -- 172.21.15.135:0/1345869591 learned_addr learned my addr 172.21.15.135:0/1345869591 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f72437fe640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72441543b0 0x7f724414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 -- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72440af300 msgr2=0x7f724414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72440af300 0x7f724414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 -- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72441543b0 msgr2=0x7f724414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72441543b0 0x7f724414a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:07.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 -- 172.21.15.135:0/1345869591 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724414b1e0 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f72437fe640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72441543b0 0x7f724414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7243fff640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440ad120 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f723400d5f0 tx=0x7f723400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340125f0 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7234019070 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340129f0 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7244150700 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.808+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724414fc20 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.809+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72440694b0 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.809+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7234012b90 con 0x7f72440c8a10 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.810+0000 7f7240ff9640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f723006c8f0 0x7f723006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:07.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.810+0000 7f72437fe640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f723006c8f0 0x7f723006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:07.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.810+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 7528+0+0 (secure 0 0 0) 0x7f723408c5e0 con 0x7f72440c8a10 2024-04-26T22:11:07.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.810+0000 7f72437fe640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f723006c8f0 0x7f723006edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f72440ade90 tx=0x7f7238009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:07.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:07.814+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f723405a380 con 0x7f72440c8a10 2024-04-26T22:11:08.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.006+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f72440c6d30 con 0x7f72440c8a10 2024-04-26T22:11:08.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.682+0000 7f7240ff9640 1 -- 172.21.15.135:0/1345869591 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true}]=0 pool 'a_data' removed v43) ==== 154+0+0 (secure 0 0 0) 0x7f7234059c80 con 0x7f72440c8a10 2024-04-26T22:11:08.683 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'a_data' removed 2024-04-26T22:11:08.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.687+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f723006c8f0 msgr2=0x7f723006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:08.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f723006c8f0 0x7f723006edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f72440ade90 tx=0x7f7238009040 comp rx=0 tx=0).stop 2024-04-26T22:11:08.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 msgr2=0x7f72440ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:08.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440ad120 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f723400d5f0 tx=0x7f723400dab0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 shutdown_connections 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f723006c8f0 0x7f723006edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72440af300 0x7f724414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72441543b0 0x7f724414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 --2- 172.21.15.135:0/1345869591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72440c8a10 0x7f72440ad120 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 >> 172.21.15.135:0/1345869591 conn(0x7f72440c8e30 msgr2=0x7f7244153670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 shutdown_connections 2024-04-26T22:11:08.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.688+0000 7f724a3e6640 1 -- 172.21.15.135:0/1345869591 wait complete. 2024-04-26T22:11:08.712 INFO:tasks.cephfs.filesystem:Destroying file system b and related pools 2024-04-26T22:11:08.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:11:08.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.872+0000 7f723d938640 1 Processor -- start 2024-04-26T22:11:08.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.872+0000 7f723d938640 1 -- start start 2024-04-26T22:11:08.873 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.873+0000 7f723d938640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f72380c52a0 0x7f72380c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:08.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.873+0000 7f723d938640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f723815ba70 0x7f723815f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:08.874 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.873+0000 7f723d938640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72380c4330 con 0x7f72380ae460 2024-04-26T22:11:08.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.873+0000 7f723d938640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f72380c4f40 con 0x7f72380c52a0 2024-04-26T22:11:08.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.873+0000 7f723d938640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f723815f560 con 0x7f723815ba70 2024-04-26T22:11:08.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.874+0000 7f72377fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f723815ba70 0x7f723815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:08.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.874+0000 7f72377fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f723815ba70 0x7f723815f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34066/0 (socket says 172.21.15.135:34066) 2024-04-26T22:11:08.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.874+0000 7f72377fe640 1 -- 172.21.15.135:0/1124699638 learned_addr learned my addr 172.21.15.135:0/1124699638 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:08.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.874+0000 7f7236ffd640 1 --2- 172.21.15.135:0/1124699638 >> v2:172.21.15.139:3300/0 conn(0x7f72380c52a0 0x7f72380c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:08.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.875+0000 7f7236ffd640 1 -- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f723815ba70 msgr2=0x7f723815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:08.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.875+0000 7f7236ffd640 1 --2- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f723815ba70 0x7f723815f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.875+0000 7f7235ffb640 1 -- 172.21.15.135:0/1124699638 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587729605 0 0) 0x7f72380c4330 con 0x7f72380ae460 2024-04-26T22:11:08.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.875+0000 7f7236ffd640 1 -- 172.21.15.135:0/1124699638 >> v1:172.21.15.135:6789/0 conn(0x7f72380ae460 legacy=0x7f723815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:08.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.875+0000 7f7236ffd640 1 -- 172.21.15.135:0/1124699638 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72380c4610 con 0x7f72380c52a0 2024-04-26T22:11:08.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.877+0000 7f7236ffd640 1 --2- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72380c52a0 0x7f72380c5680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f72280097a0 tx=0x7f722802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618489a482c0a353 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:08.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.879+0000 7f7235ffb640 1 -- 172.21.15.135:0/1124699638 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228037070 con 0x7f72380c52a0 2024-04-26T22:11:08.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.879+0000 7f7235ffb640 1 -- 172.21.15.135:0/1124699638 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f722802adf0 con 0x7f72380c52a0 2024-04-26T22:11:08.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.879+0000 7f7235ffb640 1 -- 172.21.15.135:0/1124699638 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228032660 con 0x7f72380c52a0 2024-04-26T22:11:08.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.880+0000 7f723d938640 1 -- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72380c52a0 msgr2=0x7f72380c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:08.880 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.880+0000 7f723d938640 1 --2- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72380c52a0 0x7f72380c5680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f72280097a0 tx=0x7f722802a0a0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.883+0000 7f723d938640 1 -- 172.21.15.135:0/1124699638 shutdown_connections 2024-04-26T22:11:08.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.883+0000 7f723d938640 1 --2- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f723815ba70 0x7f723815f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.883+0000 7f723d938640 1 --2- 172.21.15.135:0/1124699638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72380c52a0 0x7f72380c5680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.883+0000 7f723d938640 1 -- 172.21.15.135:0/1124699638 >> 172.21.15.135:0/1124699638 conn(0x7f72380c89b0 msgr2=0x7f72380c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:08.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.888+0000 7f723d938640 1 -- 172.21.15.135:0/1124699638 shutdown_connections 2024-04-26T22:11:08.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.888+0000 7f723d938640 1 -- 172.21.15.135:0/1124699638 wait complete. 2024-04-26T22:11:08.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.889+0000 7f723d938640 1 Processor -- start 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 -- start start 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72380ae460 0x7f723820dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 0x7f7238153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f723815ba70 0x7f72381541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72380c4bd0 con 0x7f72380c52a0 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7238074e70 con 0x7f723815ba70 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.890+0000 7f723d938640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f723815bee0 con 0x7f72380ae460 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.891+0000 7f72367fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 0x7f7238153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:08.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f723815ba70 0x7f72381541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 0x7f7238153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34722/0 (socket says 172.21.15.135:34722) 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 -- 172.21.15.135:0/3265803053 learned_addr learned my addr 172.21.15.135:0/3265803053 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f7236ffd640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72380ae460 0x7f723820dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 -- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72380ae460 msgr2=0x7f723820dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72380ae460 0x7f723820dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 -- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f723815ba70 msgr2=0x7f72381541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f723815ba70 0x7f72381541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.892+0000 7f72367fc640 1 -- 172.21.15.135:0/3265803053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72380c1f60 con 0x7f72380c52a0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.893+0000 7f72367fc640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 0x7f7238153c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f722c00c460 tx=0x7f722c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.893+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c017070 con 0x7f72380c52a0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.893+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7238158540 con 0x7f72380c52a0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.893+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723820e7f0 con 0x7f72380c52a0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.893+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f722c0035f0 con 0x7f72380c52a0 2024-04-26T22:11:08.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.893+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c003a00 con 0x7f72380c52a0 2024-04-26T22:11:08.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.895+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f722c01a070 con 0x7f72380c52a0 2024-04-26T22:11:08.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.900+0000 7f721ffff640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f722006cb00 0x7f722006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:08.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.900+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6979+0+0 (secure 0 0 0) 0x7f722c08b3c0 con 0x7f72380c52a0 2024-04-26T22:11:08.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.901+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7218005290 con 0x7f72380c52a0 2024-04-26T22:11:08.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.903+0000 7f7236ffd640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f722006cb00 0x7f722006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:08.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.904+0000 7f7236ffd640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f722006cb00 0x7f722006efc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7228034000 tx=0x7f722802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:08.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:08.913+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72380c1f60 con 0x7f72380c52a0 2024-04-26T22:11:09.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.169+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7218005580 con 0x7f72380c52a0 2024-04-26T22:11:09.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.169+0000 7f721ffff640 1 -- 172.21.15.135:0/3265803053 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5192 (secure 0 0 0) 0x7f722c059360 con 0x7f72380c52a0 2024-04-26T22:11:09.170 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:09.170 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3131080838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3131080838},{"type":"v1","addr":"172.21.15.139:6833","nonce":3131080838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1333118834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1333118834},{"type":"v1","addr":"172.21.15.135:6837","nonce":1333118834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:11:09.170 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 83 2024-04-26T22:11:09.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f722006cb00 msgr2=0x7f722006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f722006cb00 0x7f722006efc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7228034000 tx=0x7f722802e000 comp rx=0 tx=0).stop 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 msgr2=0x7f7238153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 0x7f7238153c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f722c00c460 tx=0x7f722c00c920 comp rx=0 tx=0).stop 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 shutdown_connections 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f722006cb00 0x7f722006efc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f723815ba70 0x7f72381541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72380c52a0 0x7f7238153c90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 --2- 172.21.15.135:0/3265803053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72380ae460 0x7f723820dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.171+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 >> 172.21.15.135:0/3265803053 conn(0x7f72380c89b0 msgr2=0x7f72380c63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.172+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 shutdown_connections 2024-04-26T22:11:09.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.172+0000 7f723d938640 1 -- 172.21.15.135:0/3265803053 wait complete. 2024-04-26T22:11:09.182 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:11:09.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 Processor -- start 2024-04-26T22:11:09.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 -- start start 2024-04-26T22:11:09.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5680c40e0 0x7fa5680720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa5680c46e0 0x7fa5680c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa568154ba0 con 0x7fa5680c3060 2024-04-26T22:11:09.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa568154e80 con 0x7fa5680c46e0 2024-04-26T22:11:09.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56e459640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa568155160 con 0x7fa5680c40e0 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56c9f6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa5680c46e0 0x7fa5680c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa56c9f6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa5680c46e0 0x7fa5680c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34738/0 (socket says 172.21.15.135:34738) 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.280+0000 7fa567fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa5680c3060 0x7fa568151e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:40368/0 (socket says 172.21.15.135:40368) 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa56c9f6640 1 -- 172.21.15.135:0/4015455021 learned_addr learned my addr 172.21.15.135:0/4015455021 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa56c9f6640 1 -- 172.21.15.135:0/4015455021 >> v2:172.21.15.139:3300/0 conn(0x7fa5680c40e0 msgr2=0x7fa5680720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa56c9f6640 1 --2- 172.21.15.135:0/4015455021 >> v2:172.21.15.139:3300/0 conn(0x7fa5680c40e0 0x7fa5680720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa56c9f6640 1 -- 172.21.15.135:0/4015455021 >> v1:172.21.15.135:6789/0 conn(0x7fa5680c3060 legacy=0x7fa568151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa56c9f6640 1 -- 172.21.15.135:0/4015455021 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa568155440 con 0x7fa5680c46e0 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa566ffd640 1 -- 172.21.15.135:0/4015455021 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291110815 0 0) 0x7fa568154ba0 con 0x7fa5680c3060 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.281+0000 7fa56c9f6640 1 --2- 172.21.15.135:0/4015455021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c46e0 0x7fa5680c4b70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa56806d250 tx=0x7fa56002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774ff5c867e6f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa566ffd640 1 -- 172.21.15.135:0/4015455021 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560036070 con 0x7fa5680c46e0 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa566ffd640 1 -- 172.21.15.135:0/4015455021 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa560031600 con 0x7fa5680c46e0 2024-04-26T22:11:09.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa566ffd640 1 -- 172.21.15.135:0/4015455021 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600319f0 con 0x7fa5680c46e0 2024-04-26T22:11:09.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 -- 172.21.15.135:0/4015455021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c46e0 msgr2=0x7fa5680c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 --2- 172.21.15.135:0/4015455021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c46e0 0x7fa5680c4b70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa56806d250 tx=0x7fa56002a6a0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 -- 172.21.15.135:0/4015455021 shutdown_connections 2024-04-26T22:11:09.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 --2- 172.21.15.135:0/4015455021 >> v2:172.21.15.139:3300/0 conn(0x7fa5680c40e0 0x7fa5680720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 --2- 172.21.15.135:0/4015455021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c46e0 0x7fa5680c4b70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 -- 172.21.15.135:0/4015455021 >> 172.21.15.135:0/4015455021 conn(0x7fa56806c310 msgr2=0x7fa56806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 -- 172.21.15.135:0/4015455021 shutdown_connections 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.282+0000 7fa56e459640 1 -- 172.21.15.135:0/4015455021 wait complete. 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.283+0000 7fa56e459640 1 Processor -- start 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 -- start start 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5680c3060 0x7fa56814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 0x7fa5680c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5680c46e0 0x7fa5681e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5680c3440 con 0x7fa5680c40e0 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5680c3aa0 con 0x7fa5680c46e0 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56e459640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa5680c35c0 con 0x7fa5680c3060 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 0x7fa5680c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 0x7fa5680c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34744/0 (socket says 172.21.15.135:34744) 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa5677fe640 1 -- 172.21.15.135:0/1354105122 learned_addr learned my addr 172.21.15.135:0/1354105122 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:09.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa56c9f6640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5680c46e0 0x7fa5681e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa567fff640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5680c3060 0x7fa56814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.284+0000 7fa5677fe640 1 -- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5680c3060 msgr2=0x7fa56814d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa5677fe640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5680c3060 0x7fa56814d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa5677fe640 1 -- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5680c46e0 msgr2=0x7fa5681e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa5677fe640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5680c46e0 0x7fa5681e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa5677fe640 1 -- 172.21.15.135:0/1354105122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa568156e60 con 0x7fa5680c40e0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa56c9f6640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5680c46e0 0x7fa5681e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa5677fe640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 0x7fa5680c6e50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00b4d0 tx=0x7fa55c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c014070 con 0x7fa5680c40e0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa55c010870 con 0x7fa5680c40e0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56814def0 con 0x7fa5680c40e0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c010c80 con 0x7fa5680c40e0 2024-04-26T22:11:09.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.285+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56814ccf0 con 0x7fa5680c40e0 2024-04-26T22:11:09.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.286+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5680507a0 con 0x7fa5680c40e0 2024-04-26T22:11:09.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.286+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa55c017070 con 0x7fa5680c40e0 2024-04-26T22:11:09.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.287+0000 7fa564ff9640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa55406c8f0 0x7fa55406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.287+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6979+0+0 (secure 0 0 0) 0x7fa55c08be10 con 0x7fa5680c40e0 2024-04-26T22:11:09.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.287+0000 7fa567fff640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa55406c8f0 0x7fa55406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.287+0000 7fa567fff640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa55406c8f0 0x7fa55406edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa558000f60 tx=0x7fa558007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.291+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa55c059d40 con 0x7fa5680c40e0 2024-04-26T22:11:09.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.501+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56806f430 con 0x7fa5680c40e0 2024-04-26T22:11:09.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.501+0000 7fa564ff9640 1 -- 172.21.15.135:0/1354105122 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5192 (secure 0 0 0) 0x7fa55c010e20 con 0x7fa5680c40e0 2024-04-26T22:11:09.502 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:09.502 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3131080838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3131080838},{"type":"v1","addr":"172.21.15.139:6833","nonce":3131080838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/1220545584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1220545584},{"type":"v1","addr":"172.21.15.135:6835","nonce":1220545584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1333118834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1333118834},{"type":"v1","addr":"172.21.15.135:6837","nonce":1333118834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:00:06.148422+0000","modified":"2024-04-26T22:10:06.967034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5291":{"gid":5291,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":38,"addr":"172.21.15.139:6835/1717329133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1717329133},{"type":"v1","addr":"172.21.15.139:6835","nonce":1717329133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5291,"qdb_cluster":[5291]},"id":2}]} 2024-04-26T22:11:09.502 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 83 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa55406c8f0 msgr2=0x7fa55406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa55406c8f0 0x7fa55406edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa558000f60 tx=0x7fa558007b50 comp rx=0 tx=0).stop 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 msgr2=0x7fa5680c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 0x7fa5680c6e50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00b4d0 tx=0x7fa55c00b990 comp rx=0 tx=0).stop 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 shutdown_connections 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fa55406c8f0 0x7fa55406edb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5680c46e0 0x7fa5681e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5680c40e0 0x7fa5680c6e50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 --2- 172.21.15.135:0/1354105122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5680c3060 0x7fa56814d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 >> 172.21.15.135:0/1354105122 conn(0x7fa56806c310 msgr2=0x7fa5680744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:09.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 shutdown_connections 2024-04-26T22:11:09.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.504+0000 7fa56e459640 1 -- 172.21.15.135:0/1354105122 wait complete. 2024-04-26T22:11:09.515 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:11:09.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.615+0000 7f7c428d9640 1 Processor -- start 2024-04-26T22:11:09.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.615+0000 7f7c428d9640 1 -- start start 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c428d9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 0x7f7c3c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c428d9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c3c0c52c0 0x7f7c3c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c428d9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c3c0c3810 con 0x7f7c3c1570d0 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c428d9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c0c4330 con 0x7f7c3c0c52c0 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c428d9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c0c3ab0 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 0x7f7c3c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c40e76640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c3c0c52c0 0x7f7c3c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 0x7f7c3c0c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.135:34104/0 (socket says 172.21.15.135:34104) 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 -- 172.21.15.135:0/681963005 learned_addr learned my addr 172.21.15.135:0/681963005 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 -- 172.21.15.135:0/681963005 >> v2:172.21.15.139:3300/0 conn(0x7f7c3c0c52c0 msgr2=0x7f7c3c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 --2- 172.21.15.135:0/681963005 >> v2:172.21.15.139:3300/0 conn(0x7f7c3c0c52c0 0x7f7c3c1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 -- 172.21.15.135:0/681963005 >> v1:172.21.15.139:6789/0 conn(0x7f7c3c1570d0 legacy=0x7f7c3c150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c3b7fe640 1 -- 172.21.15.135:0/681963005 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c0c4580 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.616+0000 7f7c40e76640 1 --2- 172.21.15.135:0/681963005 >> v2:172.21.15.139:3300/0 conn(0x7f7c3c0c52c0 0x7f7c3c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.617+0000 7f7c3b7fe640 1 --2- 172.21.15.135:0/681963005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 0x7f7c3c0c4d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0c44b0 tx=0x7f7c3002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22bd4230639458ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.617+0000 7f7c3affd640 1 -- 172.21.15.135:0/681963005 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30037070 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.617+0000 7f7c3affd640 1 -- 172.21.15.135:0/681963005 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7c30032440 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.617+0000 7f7c3affd640 1 -- 172.21.15.135:0/681963005 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30032830 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 -- 172.21.15.135:0/681963005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 msgr2=0x7f7c3c0c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 --2- 172.21.15.135:0/681963005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 0x7f7c3c0c4d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0c44b0 tx=0x7f7c3002a130 comp rx=0 tx=0).stop 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 -- 172.21.15.135:0/681963005 shutdown_connections 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 --2- 172.21.15.135:0/681963005 >> v2:172.21.15.139:3300/0 conn(0x7f7c3c0c52c0 0x7f7c3c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 --2- 172.21.15.135:0/681963005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c0ab3f0 0x7f7c3c0c4d80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 -- 172.21.15.135:0/681963005 >> 172.21.15.135:0/681963005 conn(0x7f7c3c06c310 msgr2=0x7f7c3c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 -- 172.21.15.135:0/681963005 shutdown_connections 2024-04-26T22:11:09.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.618+0000 7f7c428d9640 1 -- 172.21.15.135:0/681963005 wait complete. 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.619+0000 7f7c428d9640 1 Processor -- start 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.619+0000 7f7c428d9640 1 -- start start 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 0x7f7c3c147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c3c0c52c0 0x7f7c3c1535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c1570d0 0x7f7c3c153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1512e0 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c151160 con 0x7f7c3c0c52c0 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c0aba60 con 0x7f7c3c1570d0 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 0x7f7c3c147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 0x7f7c3c147b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34748/0 (socket says 172.21.15.135:34748) 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 -- 172.21.15.135:0/1207025053 learned_addr learned my addr 172.21.15.135:0/1207025053 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:09.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3b7fe640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c3c0c52c0 0x7f7c3c1535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c40e76640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c1570d0 0x7f7c3c153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 -- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c1570d0 msgr2=0x7f7c3c153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c1570d0 0x7f7c3c153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 -- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c3c0c52c0 msgr2=0x7f7c3c1535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c3c0c52c0 0x7f7c3c1535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 -- 172.21.15.135:0/1207025053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c1edb20 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c3bfff640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 0x7f7c3c147b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00c3c0 tx=0x7f7c2c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0052d0 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7c2c005490 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c1ede00 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.620+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c007280 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.621+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c1ee3c0 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.621+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c150de0 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.622+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c2c007420 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.622+0000 7f7c38ff9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7c2806c8f0 0x7f7c2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.622+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6979+0+0 (secure 0 0 0) 0x7f7c2c08b390 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.622+0000 7f7c3b7fe640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7c2806c8f0 0x7f7c2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.623+0000 7f7c3b7fe640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7c2806c8f0 0x7f7c2806edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c30034000 tx=0x7f7c3002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.627+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c2c058c70 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.807+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c3c06f830 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.808+0000 7f7c38ff9640 1 -- 172.21.15.135:0/1207025053 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+18672 (secure 0 0 0) 0x7f7c2c05cf70 con 0x7f7c3c0ab3f0 2024-04-26T22:11:09.808 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-26T22:11:09.808 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":43,"fsid":"c1882db8-5327-4bd0-9178-0a7c5b1ec94f","created":"2024-04-26T21:59:32.963912+0000","modified":"2024-04-26T22:11:08.682081+0000","last_up_change":"2024-04-26T21:59:44.084424+0000","last_in_change":"2024-04-26T21:59:35.067593+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:59:45.083457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-04-26T22:00:03.970577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-04-26T22:00:05.113722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2a884f4-36b9-4e32-a4dd-0a9e19967ac8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6817","nonce":3857357919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6819","nonce":3857357919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6823","nonce":3857357919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":3857357919},{"type":"v1","addr":"172.21.15.135:6821","nonce":3857357919}]},"public_addr":"172.21.15.135:6817/3857357919","cluster_addr":"172.21.15.135:6819/3857357919","heartbeat_back_addr":"172.21.15.135:6823/3857357919","heartbeat_front_addr":"172.21.15.135:6821/3857357919","state":["exists","up"]},{"osd":1,"uuid":"366143a9-a0ba-4cde-85d6-c871f36fc1b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6809","nonce":3325535972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6811","nonce":3325535972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6815","nonce":3325535972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3325535972},{"type":"v1","addr":"172.21.15.135:6813","nonce":3325535972}]},"public_addr":"172.21.15.135:6809/3325535972","cluster_addr":"172.21.15.135:6811/3325535972","heartbeat_back_addr":"172.21.15.135:6815/3325535972","heartbeat_front_addr":"172.21.15.135:6813/3325535972","state":["exists","up"]},{"osd":2,"uuid":"628c0d3c-f2aa-4c02-a1b8-2e61dc6dc346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6825","nonce":626924944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6827","nonce":626924944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6831","nonce":626924944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":626924944},{"type":"v1","addr":"172.21.15.135:6829","nonce":626924944}]},"public_addr":"172.21.15.135:6825/626924944","cluster_addr":"172.21.15.135:6827/626924944","heartbeat_back_addr":"172.21.15.135:6831/626924944","heartbeat_front_addr":"172.21.15.135:6829/626924944","state":["exists","up"]},{"osd":3,"uuid":"5ef0dd21-2c67-4f9a-86e0-9febe0b401ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6801","nonce":3564981873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6803","nonce":3564981873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6807","nonce":3564981873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3564981873},{"type":"v1","addr":"172.21.15.135:6805","nonce":3564981873}]},"public_addr":"172.21.15.135:6801/3564981873","cluster_addr":"172.21.15.135:6803/3564981873","heartbeat_back_addr":"172.21.15.135:6807/3564981873","heartbeat_front_addr":"172.21.15.135:6805/3564981873","state":["exists","up"]},{"osd":4,"uuid":"16e214f8-07e4-425a-a636-9841b787e311","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6809","nonce":3666291794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6811","nonce":3666291794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6815","nonce":3666291794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":3666291794},{"type":"v1","addr":"172.21.15.139:6813","nonce":3666291794}]},"public_addr":"172.21.15.139:6809/3666291794","cluster_addr":"172.21.15.139:6811/3666291794","heartbeat_back_addr":"172.21.15.139:6815/3666291794","heartbeat_front_addr":"172.21.15.139:6813/3666291794","state":["exists","up"]},{"osd":5,"uuid":"d5a4f44c-f092-499b-8b8a-a122bee206e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6801","nonce":2023089101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6803","nonce":2023089101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6807","nonce":2023089101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2023089101},{"type":"v1","addr":"172.21.15.139:6805","nonce":2023089101}]},"public_addr":"172.21.15.139:6801/2023089101","cluster_addr":"172.21.15.139:6803/2023089101","heartbeat_back_addr":"172.21.15.139:6807/2023089101","heartbeat_front_addr":"172.21.15.139:6805/2023089101","state":["exists","up"]},{"osd":6,"uuid":"76127437-7143-4255-b484-d3e40f69bc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6817","nonce":3816888463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6819","nonce":3816888463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6823","nonce":3816888463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3816888463},{"type":"v1","addr":"172.21.15.139:6821","nonce":3816888463}]},"public_addr":"172.21.15.139:6817/3816888463","cluster_addr":"172.21.15.139:6819/3816888463","heartbeat_back_addr":"172.21.15.139:6823/3816888463","heartbeat_front_addr":"172.21.15.139:6821/3816888463","state":["exists","up"]},{"osd":7,"uuid":"1f27f035-a26f-481c-9677-46400f4d8871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6825","nonce":3012659507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6827","nonce":3012659507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6831","nonce":3012659507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3012659507},{"type":"v1","addr":"172.21.15.139:6829","nonce":3012659507}]},"public_addr":"172.21.15.139:6825/3012659507","cluster_addr":"172.21.15.139:6827/3012659507","heartbeat_back_addr":"172.21.15.139:6831/3012659507","heartbeat_front_addr":"172.21.15.139:6829/3012659507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.317583+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.270179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.228508+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.231555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.233914+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.188521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.264572+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:59:40.269612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6833/112034263":"2024-04-27T22:11:05.670159+0000","172.21.15.139:6834/302589348":"2024-04-27T22:05:52.818178+0000","172.21.15.135:6835/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.135:6834/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.135:6837/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.135:6836/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.135:6836/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.135:6835/2910412510":"2024-04-27T22:04:37.805449+0000","172.21.15.139:6833/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.139:6835/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.139:6832/3183513686":"2024-04-27T22:03:27.791314+0000","172.21.15.139:6835/302589348":"2024-04-27T22:05:52.818178+0000","172.21.15.135:6837/3652200818":"2024-04-27T22:04:12.801037+0000","172.21.15.139:6834/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.139:6834/2608264407":"2024-04-27T22:03:07.784944+0000","172.21.15.135:6834/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.135:6834/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.135:6836/3981978490":"2024-04-27T22:01:37.770149+0000","172.21.15.139:6833/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6834/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.135:6835/3214490783":"2024-04-27T22:06:12.821933+0000","172.21.15.139:6832/3664037452":"2024-04-27T22:08:57.853641+0000","172.21.15.135:6837/2165063360":"2024-04-27T22:07:07.830780+0000","172.21.15.139:6835/3733666635":"2024-04-27T22:07:32.835007+0000","172.21.15.135:6835/3800113814":"2024-04-27T22:10:00.929175+0000","172.21.15.135:6834/1037298395":"2024-04-27T22:07:37.836479+0000","172.21.15.135:6835/1410523687":"2024-04-27T22:01:27.768292+0000","172.21.15.135:6836/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.135:6837/3969385858":"2024-04-27T22:09:52.864568+0000","172.21.15.139:6832/112034263":"2024-04-27T22:11:05.670159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:11:09.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7c2806c8f0 msgr2=0x7f7c2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7c2806c8f0 0x7f7c2806edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c30034000 tx=0x7f7c3002e000 comp rx=0 tx=0).stop 2024-04-26T22:11:09.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 msgr2=0x7f7c3c147b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 0x7f7c3c147b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00c3c0 tx=0x7f7c2c00c7c0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 shutdown_connections 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f7c2806c8f0 0x7f7c2806edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c3c1570d0 0x7f7c3c153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c3c0c52c0 0x7f7c3c1535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 --2- 172.21.15.135:0/1207025053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3c0ab3f0 0x7f7c3c147b80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 >> 172.21.15.135:0/1207025053 conn(0x7f7c3c06c310 msgr2=0x7f7c3c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 shutdown_connections 2024-04-26T22:11:09.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.810+0000 7f7c428d9640 1 -- 172.21.15.135:0/1207025053 wait complete. 2024-04-26T22:11:09.822 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail b 2024-04-26T22:11:09.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.920+0000 7f092886d640 1 Processor -- start 2024-04-26T22:11:09.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.920+0000 7f092886d640 1 -- start start 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092886d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f09240c3060 0x7f09240720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092886d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f09240c46e0 0x7f09240c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092886d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0924154ba0 con 0x7f09240c40e0 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092886d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0924154e80 con 0x7f09240c3060 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092886d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0924155160 con 0x7f09240c46e0 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f09240c3060 0x7f09240720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f09240c3060 0x7f09240720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34760/0 (socket says 172.21.15.135:34760) 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 -- 172.21.15.135:0/864486667 learned_addr learned my addr 172.21.15.135:0/864486667 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f0922d9d640 1 --2- 172.21.15.135:0/864486667 >> v2:172.21.15.139:3300/0 conn(0x7f09240c46e0 0x7f09240c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 -- 172.21.15.135:0/864486667 >> v2:172.21.15.139:3300/0 conn(0x7f09240c46e0 msgr2=0x7f09240c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 --2- 172.21.15.135:0/864486667 >> v2:172.21.15.139:3300/0 conn(0x7f09240c46e0 0x7f09240c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 -- 172.21.15.135:0/864486667 >> v1:172.21.15.139:6789/0 conn(0x7f09240c40e0 legacy=0x7f0924151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:09.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.921+0000 7f092259c640 1 -- 172.21.15.135:0/864486667 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0924155440 con 0x7f09240c3060 2024-04-26T22:11:09.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092259c640 1 --2- 172.21.15.135:0/864486667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c3060 0x7f09240720a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f09240c4570 tx=0x7f091402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd30371a5c595c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f0921d9b640 1 -- 172.21.15.135:0/864486667 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914037070 con 0x7f09240c3060 2024-04-26T22:11:09.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f0921d9b640 1 -- 172.21.15.135:0/864486667 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0914032440 con 0x7f09240c3060 2024-04-26T22:11:09.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f0921d9b640 1 -- 172.21.15.135:0/864486667 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914032830 con 0x7f09240c3060 2024-04-26T22:11:09.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 -- 172.21.15.135:0/864486667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c3060 msgr2=0x7f09240720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 --2- 172.21.15.135:0/864486667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c3060 0x7f09240720a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f09240c4570 tx=0x7f091402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 -- 172.21.15.135:0/864486667 shutdown_connections 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 --2- 172.21.15.135:0/864486667 >> v2:172.21.15.139:3300/0 conn(0x7f09240c46e0 0x7f09240c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 --2- 172.21.15.135:0/864486667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c3060 0x7f09240720a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 -- 172.21.15.135:0/864486667 >> 172.21.15.135:0/864486667 conn(0x7f092406c310 msgr2=0x7f092406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 -- 172.21.15.135:0/864486667 shutdown_connections 2024-04-26T22:11:09.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.922+0000 7f092886d640 1 -- 172.21.15.135:0/864486667 wait complete. 2024-04-26T22:11:09.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 Processor -- start 2024-04-26T22:11:09.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 -- start start 2024-04-26T22:11:09.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09240c3060 0x7f0924151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240c40e0 0x7f09241474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 0x7f0924147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09240c3440 con 0x7f09240c46e0 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09240c3aa0 con 0x7f09240c40e0 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f092886d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f09240c35c0 con 0x7f09240c3060 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f0922d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 0x7f0924147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f0922d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 0x7f0924147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34768/0 (socket says 172.21.15.135:34768) 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.924+0000 7f0922d9d640 1 -- 172.21.15.135:0/612096054 learned_addr learned my addr 172.21.15.135:0/612096054 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f091bfff640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240c40e0 0x7f09241474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f092259c640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09240c3060 0x7f0924151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f0922d9d640 1 -- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09240c3060 msgr2=0x7f0924151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f0922d9d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09240c3060 0x7f0924151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f0922d9d640 1 -- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240c40e0 msgr2=0x7f09241474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f0922d9d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240c40e0 0x7f09241474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f0922d9d640 1 -- 172.21.15.135:0/612096054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0924157840 con 0x7f09240c46e0 2024-04-26T22:11:09.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f0922d9d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 0x7f0924147a20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f091c00cc30 tx=0x7f091c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c00ce40 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f091c003990 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0924157b20 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c003d60 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.925+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0924151780 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.926+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092414c030 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.927+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f091c003fc0 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.927+0000 7f091affd640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.927+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6979+0+0 (secure 0 0 0) 0x7f091c08c490 con 0x7f09240c46e0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.927+0000 7f092259c640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:09.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.927+0000 7f092259c640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f090406c8f0 0x7f090406edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f091402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:09.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:09.931+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f092414c030 con 0x7f09240c46e0 2024-04-26T22:11:10.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.145+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "b"} v 0) -- 0x7f09240507a0 con 0x7f09240c46e0 2024-04-26T22:11:10.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.147+0000 7f091affd640 1 -- 172.21.15.135:0/612096054 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "b"}]=0 b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 151+0+0 (secure 0 0 0) 0x7f091c014030 con 0x7f09240c46e0 2024-04-26T22:11:10.147 INFO:teuthology.orchestra.run.smithi135.stderr:b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T22:11:10.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f090406c8f0 msgr2=0x7f090406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:10.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f090406c8f0 0x7f090406edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f091402e000 comp rx=0 tx=0).stop 2024-04-26T22:11:10.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 msgr2=0x7f0924147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:10.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 0x7f0924147a20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f091c00cc30 tx=0x7f091c00a5c0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 shutdown_connections 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09240c46e0 0x7f0924147a20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240c40e0 0x7f09241474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 --2- 172.21.15.135:0/612096054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09240c3060 0x7f0924151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 >> 172.21.15.135:0/612096054 conn(0x7f092406c310 msgr2=0x7f0924151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 shutdown_connections 2024-04-26T22:11:10.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.150+0000 7f092886d640 1 -- 172.21.15.135:0/612096054 wait complete. 2024-04-26T22:11:10.158 INFO:tasks.ceph.mds.b.smithi139.stderr: -8160> 2024-04-26T22:10:10.967+0000 7f4cf4f7f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:11:10.163 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm b --yes-i-really-mean-it 2024-04-26T22:11:10.192 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-04-26T22:11:10.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.261+0000 7f97a3d12640 1 Processor -- start 2024-04-26T22:11:10.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.262+0000 7f97a3d12640 1 -- start start 2024-04-26T22:11:10.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.262+0000 7f97a3d12640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c46c0 0x7f979c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:10.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.262+0000 7f97a3d12640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f979c0c3720 0x7f979c0c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:10.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.262+0000 7f97a3d12640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f979c1577b0 con 0x7f979c13ff20 2024-04-26T22:11:10.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.262+0000 7f97a3d12640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f979c157560 con 0x7f979c0c3720 2024-04-26T22:11:10.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.262+0000 7f97a3d12640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f979c0ab960 con 0x7f979c0c46c0 2024-04-26T22:11:10.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a22af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f979c0c3720 0x7f979c0c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:10.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a12ad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c46c0 0x7f979c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:10.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a1aae640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f979c13ff20 0x7f979c15a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.135:58942/0 (socket says 172.21.15.135:58942) 2024-04-26T22:11:10.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a1aae640 1 -- 172.21.15.135:0/1763134800 learned_addr learned my addr 172.21.15.135:0/1763134800 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:10.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a22af640 1 -- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c46c0 msgr2=0x7f979c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a22af640 1 --2- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c46c0 0x7f979c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a22af640 1 -- 172.21.15.135:0/1763134800 >> v1:172.21.15.139:6789/0 conn(0x7f979c13ff20 legacy=0x7f979c15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a0aac640 1 -- 172.21.15.135:0/1763134800 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357148525 0 0) 0x7f979c1577b0 con 0x7f979c13ff20 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.263+0000 7f97a22af640 1 -- 172.21.15.135:0/1763134800 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c0ab5f0 con 0x7f979c0c3720 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a22af640 1 --2- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c0c3720 0x7f979c0c3bb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f97940097a0 tx=0x7f979402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ffcea738b173f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a0aac640 1 -- 172.21.15.135:0/1763134800 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794037070 con 0x7f979c0c3720 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a0aac640 1 -- 172.21.15.135:0/1763134800 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9794032440 con 0x7f979c0c3720 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a0aac640 1 -- 172.21.15.135:0/1763134800 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794032830 con 0x7f979c0c3720 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 -- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c0c3720 msgr2=0x7f979c0c3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 --2- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c0c3720 0x7f979c0c3bb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f97940097a0 tx=0x7f979402a180 comp rx=0 tx=0).stop 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 -- 172.21.15.135:0/1763134800 shutdown_connections 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 --2- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c0c3720 0x7f979c0c3bb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 --2- 172.21.15.135:0/1763134800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c46c0 0x7f979c15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 -- 172.21.15.135:0/1763134800 >> 172.21.15.135:0/1763134800 conn(0x7f979c0ca2f0 msgr2=0x7f979c0ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:10.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 -- 172.21.15.135:0/1763134800 shutdown_connections 2024-04-26T22:11:10.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.264+0000 7f97a3d12640 1 -- 172.21.15.135:0/1763134800 wait complete. 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.265+0000 7f97a3d12640 1 Processor -- start 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.265+0000 7f97a3d12640 1 -- start start 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a3d12640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c3720 0x7f979c13cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a3d12640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 0x7f979c0ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a3d12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c1efb60 0x7f979c0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a3d12640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f979c140300 con 0x7f979c13ff20 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a3d12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f979c0ac710 con 0x7f979c1efb60 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a3d12640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f979c140480 con 0x7f979c0c3720 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a12ad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 0x7f979c0ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:10.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a12ad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 0x7f979c0ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34782/0 (socket says 172.21.15.135:34782) 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a12ad640 1 -- 172.21.15.135:0/725931544 learned_addr learned my addr 172.21.15.135:0/725931544 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a22af640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c1efb60 0x7f979c0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a1aae640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c3720 0x7f979c13cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a12ad640 1 -- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c3720 msgr2=0x7f979c13cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.266+0000 7f97a12ad640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c3720 0x7f979c13cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97a12ad640 1 -- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c1efb60 msgr2=0x7f979c0c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97a12ad640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c1efb60 0x7f979c0c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97a12ad640 1 -- 172.21.15.135:0/725931544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c0ae6a0 con 0x7f979c13ff20 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97a12ad640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 0x7f979c0ad5b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f978c00e9b0 tx=0x7f978c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c017070 con 0x7f979c13ff20 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f978c00c9a0 con 0x7f979c13ff20 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c0ade00 con 0x7f979c13ff20 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c00cdb0 con 0x7f979c13ff20 2024-04-26T22:11:10.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.267+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c0ae3c0 con 0x7f979c13ff20 2024-04-26T22:11:10.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.268+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979c0c85a0 con 0x7f979c13ff20 2024-04-26T22:11:10.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.268+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f978c01a070 con 0x7f979c13ff20 2024-04-26T22:11:10.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.269+0000 7f97927fc640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f978006c8f0 0x7f978006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:10.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.269+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7065+0+0 (secure 0 0 0) 0x7f978c08c0b0 con 0x7f979c13ff20 2024-04-26T22:11:10.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.269+0000 7f97a1aae640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f978006c8f0 0x7f978006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:10.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.269+0000 7f97a1aae640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f978006c8f0 0x7f978006edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f97980012e0 tx=0x7f9798001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:10.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.273+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f979c0c85a0 con 0x7f979c13ff20 2024-04-26T22:11:10.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:10.486+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true} v 0) -- 0x7f979c0766c0 con 0x7f979c13ff20 2024-04-26T22:11:11.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.149+0000 7f97927fc640 1 -- 172.21.15.135:0/725931544 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true}]=0 v85) ==== 99+0+0 (secure 0 0 0) 0x7f978c0598b0 con 0x7f979c13ff20 2024-04-26T22:11:11.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f978006c8f0 msgr2=0x7f978006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:11.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f978006c8f0 0x7f978006edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f97980012e0 tx=0x7f9798001630 comp rx=0 tx=0).stop 2024-04-26T22:11:11.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 msgr2=0x7f979c0ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:11.151 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 0x7f979c0ad5b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f978c00e9b0 tx=0x7f978c00ee70 comp rx=0 tx=0).stop 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 shutdown_connections 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f978006c8f0 0x7f978006edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f979c1efb60 0x7f979c0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f979c13ff20 0x7f979c0ad5b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 --2- 172.21.15.135:0/725931544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f979c0c3720 0x7f979c13cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 >> 172.21.15.135:0/725931544 conn(0x7f979c0ca2f0 msgr2=0x7f979c1543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 shutdown_connections 2024-04-26T22:11:11.152 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.151+0000 7f97a3d12640 1 -- 172.21.15.135:0/725931544 wait complete. 2024-04-26T22:11:11.164 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_metadata b_metadata --yes-i-really-really-mean-it 2024-04-26T22:11:11.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.268+0000 7fb77970a640 1 Processor -- start 2024-04-26T22:11:11.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.268+0000 7fb77970a640 1 -- start start 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb77970a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb774151ef0 0x7fb7740c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb77970a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb774074a20 0x7fb7740c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb77970a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7741522d0 con 0x7fb774073ac0 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb77970a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb7741557e0 con 0x7fb774074a20 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb77970a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb7740c7650 con 0x7fb774151ef0 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb774074a20 0x7fb7740c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb774074a20 0x7fb7740c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34786/0 (socket says 172.21.15.135:34786) 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 -- 172.21.15.135:0/1875117950 learned_addr learned my addr 172.21.15.135:0/1875117950 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 -- 172.21.15.135:0/1875117950 >> v2:172.21.15.139:3300/0 conn(0x7fb774151ef0 msgr2=0x7fb7740c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb772ffd640 1 --2- 172.21.15.135:0/1875117950 >> v2:172.21.15.139:3300/0 conn(0x7fb774151ef0 0x7fb7740c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:11.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 --2- 172.21.15.135:0/1875117950 >> v2:172.21.15.139:3300/0 conn(0x7fb774151ef0 0x7fb7740c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 -- 172.21.15.135:0/1875117950 >> v1:172.21.15.139:6789/0 conn(0x7fb774073ac0 legacy=0x7fb774070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb76bfff640 1 -- 172.21.15.135:0/1875117950 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb774154bd0 con 0x7fb774074a20 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.269+0000 7fb772ffd640 1 --2- 172.21.15.135:0/1875117950 >> v2:172.21.15.139:3300/0 conn(0x7fb774151ef0 0x7fb7740c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.270+0000 7fb76bfff640 1 --2- 172.21.15.135:0/1875117950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774074a20 0x7fb7740c7110 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb7740c7ee0 tx=0x7fb75c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7ef6e6d30e6564 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.270+0000 7fb7727fc640 1 -- 172.21.15.135:0/1875117950 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c037070 con 0x7fb774074a20 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.270+0000 7fb7727fc640 1 -- 172.21.15.135:0/1875117950 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb75c032440 con 0x7fb774074a20 2024-04-26T22:11:11.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.270+0000 7fb7727fc640 1 -- 172.21.15.135:0/1875117950 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c032830 con 0x7fb774074a20 2024-04-26T22:11:11.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.270+0000 7fb77970a640 1 -- 172.21.15.135:0/1875117950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774074a20 msgr2=0x7fb7740c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.270+0000 7fb77970a640 1 --2- 172.21.15.135:0/1875117950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774074a20 0x7fb7740c7110 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb7740c7ee0 tx=0x7fb75c02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.271+0000 7fb77970a640 1 -- 172.21.15.135:0/1875117950 shutdown_connections 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.271+0000 7fb77970a640 1 --2- 172.21.15.135:0/1875117950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774074a20 0x7fb7740c7110 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.271+0000 7fb77970a640 1 --2- 172.21.15.135:0/1875117950 >> v2:172.21.15.139:3300/0 conn(0x7fb774151ef0 0x7fb7740c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.271+0000 7fb77970a640 1 -- 172.21.15.135:0/1875117950 >> 172.21.15.135:0/1875117950 conn(0x7fb774075d50 msgr2=0x7fb77406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.271+0000 7fb77970a640 1 -- 172.21.15.135:0/1875117950 shutdown_connections 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.271+0000 7fb77970a640 1 -- 172.21.15.135:0/1875117950 wait complete. 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 Processor -- start 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 -- start start 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 0x7fb7741401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb774074a20 0x7fb7741f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb774151ef0 0x7fb7740c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb774155470 con 0x7fb774073ac0 2024-04-26T22:11:11.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb774155190 con 0x7fb774151ef0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb77970a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb774154880 con 0x7fb774074a20 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 0x7fb7741401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb772ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 0x7fb7741401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34796/0 (socket says 172.21.15.135:34796) 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.272+0000 7fb772ffd640 1 -- 172.21.15.135:0/493509517 learned_addr learned my addr 172.21.15.135:0/493509517 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb772ffd640 1 -- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb774074a20 msgr2=0x7fb7741f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb772ffd640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb774074a20 0x7fb7741f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb772ffd640 1 -- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb774151ef0 msgr2=0x7fb7740c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb7737fe640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb774151ef0 0x7fb7740c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb772ffd640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb774151ef0 0x7fb7740c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb772ffd640 1 -- 172.21.15.135:0/493509517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77414ff70 con 0x7fb774073ac0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb772ffd640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 0x7fb7741401f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb76400c820 tx=0x7fb76400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764015070 con 0x7fb774073ac0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb764007a30 con 0x7fb774073ac0 2024-04-26T22:11:11.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77406df10 con 0x7fb774073ac0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764007e00 con 0x7fb774073ac0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.273+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77413f9f0 con 0x7fb774073ac0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.274+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7740c4a70 con 0x7fb774073ac0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.274+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb764018070 con 0x7fb774073ac0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.275+0000 7fb76b7fe640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb75406c920 0x7fb75406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.275+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 7065+0+0 (secure 0 0 0) 0x7fb76408bf90 con 0x7fb774073ac0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.275+0000 7fb76bfff640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb75406c920 0x7fb75406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:11.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.275+0000 7fb76bfff640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb75406c920 0x7fb75406ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb75c034000 tx=0x7fb75c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:11.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.279+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb764059e70 con 0x7fb774073ac0 2024-04-26T22:11:11.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:11.456+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb774069bc0 con 0x7fb774073ac0 2024-04-26T22:11:12.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.153+0000 7fb76b7fe640 1 -- 172.21.15.135:0/493509517 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'b_metadata' removed v45) ==== 166+0+0 (secure 0 0 0) 0x7fb764059770 con 0x7fb774073ac0 2024-04-26T22:11:12.153 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'b_metadata' removed 2024-04-26T22:11:12.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb75406c920 msgr2=0x7fb75406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb75406c920 0x7fb75406ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb75c034000 tx=0x7fb75c02c000 comp rx=0 tx=0).stop 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 msgr2=0x7fb7741401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 0x7fb7741401f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb76400c820 tx=0x7fb76400cce0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 shutdown_connections 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7fb75406c920 0x7fb75406ede0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb774151ef0 0x7fb7740c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb774074a20 0x7fb7741f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 --2- 172.21.15.135:0/493509517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774073ac0 0x7fb7741401f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.156+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 >> 172.21.15.135:0/493509517 conn(0x7fb774075d50 msgr2=0x7fb77406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.157+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 shutdown_connections 2024-04-26T22:11:12.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.157+0000 7fb77970a640 1 -- 172.21.15.135:0/493509517 wait complete. 2024-04-26T22:11:12.171 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_data b_data --yes-i-really-really-mean-it 2024-04-26T22:11:12.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.275+0000 7f4ec2d5f640 1 Processor -- start 2024-04-26T22:11:12.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.275+0000 7f4ec2d5f640 1 -- start start 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec2d5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4ebc06c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec2d5f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ebc06d920 con 0x7f4ebc0756f0 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec2d5f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ebc15ace0 con 0x7f4ebc070490 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec2d5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc15b260 con 0x7f4ebc0c3f90 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec0afb640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4ebc0756f0 0x7f4ebc156d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:40374/0 (socket says 172.21.15.135:40374) 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec0afb640 1 -- 172.21.15.135:0/1663424936 learned_addr learned my addr 172.21.15.135:0/1663424936 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323305094 0 0) 0x7f4ebc06d920 con 0x7f4ebc0756f0 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec12fc640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4ebc06c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eac003520 con 0x7f4ebc0756f0 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 420406521 0 0) 0x7f4eac003520 con 0x7f4ebc0756f0 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 msgr2=0x7f4ebc06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:12.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4ebc06c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 >> v1:172.21.15.139:6789/0 conn(0x7f4ebc070490 legacy=0x7f4ebc15f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc06db90 con 0x7f4ebc0756f0 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3254650183 0 0) 0x7f4eb0001670 con 0x7f4ebc0756f0 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 >> v1:172.21.15.135:6789/0 conn(0x7f4ebc0756f0 legacy=0x7f4ebc156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4eac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac003f60 0x7f4eac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4eac004d40 0x7f4eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc15b260 con 0x7f4eac004d40 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac00baa0 con 0x7f4eac003f60 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac00bca0 con 0x7f4ebc0c3f90 2024-04-26T22:11:12.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.276+0000 7f4ec12fc640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4eac004d40 0x7f4eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec0afb640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4eac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ebbfff640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac003f60 0x7f4eac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec12fc640 1 -- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 msgr2=0x7f4eac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec12fc640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4eac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec12fc640 1 -- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac003f60 msgr2=0x7f4eac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec12fc640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac003f60 0x7f4eac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec12fc640 1 -- 172.21.15.135:0/1663424936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac003520 con 0x7f4eac004d40 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec0afb640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ebc0c3f90 0x7f4eac0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ebbfff640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4eac003f60 0x7f4eac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.277+0000 7f4ec12fc640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4eac004d40 0x7f4eac0051c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fc0 tx=0x7f4eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65da437221d6bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4009c10 con 0x7f4eac004d40 2024-04-26T22:11:12.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4eb4036070 con 0x7f4eac004d40 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ebb7fe640 1 -- 172.21.15.135:0/1663424936 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40318a0 con 0x7f4eac004d40 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1663424936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4eac004d40 msgr2=0x7f4eac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1663424936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4eac004d40 0x7f4eac0051c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fc0 tx=0x7f4eac005830 comp rx=0 tx=0).stop 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ec12fc640 1 -- 172.21.15.135:0/1663424936 reap_dead start 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1663424936 shutdown_connections 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1663424936 >> 172.21.15.135:0/1663424936 conn(0x7f4ebc0c5ce0 msgr2=0x7f4ebc0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.278+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1663424936 shutdown_connections 2024-04-26T22:11:12.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.279+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1663424936 wait complete. 2024-04-26T22:11:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 Processor -- start 2024-04-26T22:11:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 -- start start 2024-04-26T22:11:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4eac004d40 0x7f4ebc0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 0x7f4ebc14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ebc154b50 0x7f4ebc1ed450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc154640 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0c4680 con 0x7f4ebc154b50 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.280+0000 7f4ec2d5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc1547c0 con 0x7f4eac004d40 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 0x7f4ebc14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 0x7f4ebc14d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:34820/0 (socket says 172.21.15.135:34820) 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 -- 172.21.15.135:0/1004081789 learned_addr learned my addr 172.21.15.135:0/1004081789 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ec0afb640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4eac004d40 0x7f4ebc0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ec12fc640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ebc154b50 0x7f4ebc1ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 -- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4eac004d40 msgr2=0x7f4ebc0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4eac004d40 0x7f4ebc0c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 -- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ebc154b50 msgr2=0x7f4ebc1ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ebc154b50 0x7f4ebc1ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 -- 172.21.15.135:0/1004081789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc159f90 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ec0afb640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4eac004d40 0x7f4ebc0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ebbfff640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 0x7f4ebc14d960 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e4e0 tx=0x7f4ea800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80052e0 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4ea8018070 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc1f1120 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8003850 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.281+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc1f1400 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.282+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ebc0694b0 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.283+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ea80039f0 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.283+0000 7f4eb97fa640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4ea006c8f0 0x7f4ea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.283+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6637+0+0 (secure 0 0 0) 0x7f4ea808bbb0 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.283+0000 7f4ec0afb640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4ea006c8f0 0x7f4ea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.283+0000 7f4ec0afb640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4ea006c8f0 0x7f4ea006edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001cd0 tx=0x7f4eb0001e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:11:12.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.287+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ea8059d00 con 0x7f4ebc14f4a0 2024-04-26T22:11:12.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:12.461+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ebc0c6950 con 0x7f4ebc14f4a0 2024-04-26T22:11:13.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.154+0000 7f4eb97fa640 1 -- 172.21.15.135:0/1004081789 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true}]=0 pool 'b_data' removed v46) ==== 154+0+0 (secure 0 0 0) 0x7f4ea8059600 con 0x7f4ebc14f4a0 2024-04-26T22:11:13.155 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'b_data' removed 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4ea006c8f0 msgr2=0x7f4ea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4ea006c8f0 0x7f4ea006edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001cd0 tx=0x7f4eb0001e00 comp rx=0 tx=0).stop 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 msgr2=0x7f4ebc14d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 0x7f4ebc14d960 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e4e0 tx=0x7f4ea800e9a0 comp rx=0 tx=0).stop 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 shutdown_connections 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:6832/3236798717,v1:172.21.15.135:6833/3236798717] conn(0x7f4ea006c8f0 0x7f4ea006edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ebc154b50 0x7f4ebc1ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:13.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ebc14f4a0 0x7f4ebc14d960 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:13.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 --2- 172.21.15.135:0/1004081789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4eac004d40 0x7f4ebc0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:11:13.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 >> 172.21.15.135:0/1004081789 conn(0x7f4ebc0c5ce0 msgr2=0x7f4ebc06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:11:13.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 shutdown_connections 2024-04-26T22:11:13.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-26T22:11:13.157+0000 7f4ec2d5f640 1 -- 172.21.15.135:0/1004081789 wait complete. 2024-04-26T22:11:13.179 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T22:11:13.179 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:11:13.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.196 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:11:13.196 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:11:13.196 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.262 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:11:13.262 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T22:11:13.263 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.275 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T22:11:13.275 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T22:11:13.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.330 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T22:11:13.331 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T22:11:13.331 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T22:11:13.331 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.412 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T22:11:13.412 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T22:11:13.412 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.488 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T22:11:13.488 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T22:11:13.488 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.573 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T22:11:13.573 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T22:11:13.573 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.651 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T22:11:13.651 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T22:11:13.651 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.719 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T22:11:13.719 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T22:11:13.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.782 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T22:11:13.783 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T22:11:13.783 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.851 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T22:11:13.851 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T22:11:13.852 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.928 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T22:11:13.928 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T22:11:13.928 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T22:11:13.928 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.955 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T22:11:13.955 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T22:11:13.955 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:13.982 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T22:11:13.982 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T22:11:13.983 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T22:11:13.983 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:14.007 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T22:11:14.007 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T22:11:14.007 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:14.033 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T22:11:14.033 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T22:11:14.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:11:14.084 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T22:11:14.084 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T22:11:14.084 DEBUG:teuthology.orchestra.run.smithi135:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'Replacing daemon mds' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T22:11:14.151 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:11:14.151 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T22:11:14.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:11:14.528 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T22:11:14.619 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:11:14.619 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T22:11:14.700 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:11:14.700 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T22:11:14.790 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:11:14.791 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T22:11:15.093 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:11:15.094 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T22:11:15.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:11:15.223 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T22:11:15.290 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:11:15.290 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T22:11:15.392 INFO:tasks.ceph:Archiving mon data... 2024-04-26T22:11:15.393 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/data/mon.a.tgz 2024-04-26T22:11:15.393 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-04-26T22:11:15.415 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.BOc94gEnjc 2024-04-26T22:11:15.416 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BOc94gEnjc 2024-04-26T22:11:15.611 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.BOc94gEnjc 2024-04-26T22:11:15.703 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.BOc94gEnjc is 432KB 2024-04-26T22:11:15.799 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.BOc94gEnjc 2024-04-26T22:11:15.814 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/data/mon.b.tgz 2024-04-26T22:11:15.814 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-04-26T22:11:15.847 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.VgML0k68xH 2024-04-26T22:11:15.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.VgML0k68xH 2024-04-26T22:11:16.016 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.VgML0k68xH 2024-04-26T22:11:16.118 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.VgML0k68xH is 456KB 2024-04-26T22:11:16.221 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.VgML0k68xH 2024-04-26T22:11:16.235 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/data/mon.c.tgz 2024-04-26T22:11:16.236 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-04-26T22:11:16.289 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.px6ic0bBj1 2024-04-26T22:11:16.289 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.px6ic0bBj1 2024-04-26T22:11:16.439 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.px6ic0bBj1 2024-04-26T22:11:16.521 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.px6ic0bBj1 is 456KB 2024-04-26T22:11:16.618 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.px6ic0bBj1 2024-04-26T22:11:16.632 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T22:11:16.632 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T22:11:16.635 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T22:11:16.724 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T22:11:16.760 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T22:11:16.760 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T22:11:16.762 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/remote/smithi135/crash 2024-04-26T22:11:16.762 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T22:11:16.791 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/remote/smithi139/crash 2024-04-26T22:11:16.792 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T22:11:16.823 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T22:11:16.824 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:11:16.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:11:16.925 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T22:11:16.960 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T22:11:16.998 INFO:tasks.ceph:Compressing logs... 2024-04-26T22:11:16.998 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:11:17.003 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34642.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34431.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34642.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34431.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34471.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34471.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34602.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40869.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T22:11:17.090 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.40869.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40869.log.gz 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34391.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34391.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34515.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34682.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34682.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T22:11:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-04-26T22:11:17.096 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.34562.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34431.log.gz 2024-04-26T22:11:17.107 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34471.log.gz 2024-04-26T22:11:17.139 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-04-26T22:11:17.140 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34642.log.gz 2024-04-26T22:11:17.140 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34515.log.gz 2024-04-26T22:11:17.140 INFO:teuthology.orchestra.run.smithi139.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-04-26T22:11:17.140 INFO:teuthology.orchestra.run.smithi139.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34682.log.gz 2024-04-26T22:11:17.167 INFO:teuthology.orchestra.run.smithi139.stderr: 94.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57255.log 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59980.log 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58640.log 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57255.log: /var/log/ceph/ceph-client.admin.59980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54776.log 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57255.log.gz 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59980.log.gz 2024-04-26T22:11:17.205 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54692.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58640.log.gz 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45815.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59125.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54776.log.gz 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54692.log.gz 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40830.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58917.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40830.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59125.log.gz 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45437.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53571.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45437.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58917.log.gz 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53571.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40830.log.gz 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43422.log 2024-04-26T22:11:17.206 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53571.log.gz 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43842.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42761.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55888.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60400.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42087.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55888.log.gz 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.44202.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46319.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60400.log.gz 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56460.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57086.log 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43842.log.gz 2024-04-26T22:11:17.207 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57044.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45300.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54734.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54197.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57636.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59938.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42758.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54566.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43926.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51715.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45689.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46997.log 2024-04-26T22:11:17.208 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55620.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42427.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56544.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55130.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58798.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40631.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45899.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52789.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56417.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58739.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2024-04-26T22:11:17.209 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57350.log 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57086.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56460.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43043.log: /var/log/ceph/ceph-client.admin.46151.log: /var/log/ceph/ceph-client.admin.43800.log: /var/log/ceph/ceph-client.admin.57044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55046.log 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45300.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43422.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54734.log: /var/log/ceph/ceph-client.admin.43117.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57086.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54197.log: 89.5%/var/log/ceph/ceph-client.admin.57636.log: /var/log/ceph/ceph-client.admin.57350.log: -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57044.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59938.log: 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53786.log 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54734.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54197.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57350.log.gz -- replaced with /var/log/ceph/ceph-client.admin.57636.log.gz 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59770.log 2024-04-26T22:11:17.210 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59938.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42758.log: /var/log/ceph/ceph-client.admin.55046.log: /var/log/ceph/ceph-client.admin.41393.log: /var/log/ceph/ceph-client.admin.54566.log: /var/log/ceph/ceph-client.admin.43926.log: /var/log/ceph/ceph-client.admin.51715.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45689.log: /var/log/ceph/ceph-client.admin.53786.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55046.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46997.log: -5 --verbose 87.2% -- /var/log/ceph/ceph-client.admin.51894.log -- replaced with /var/log/ceph/ceph-client.admin.54566.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45437.log.gz/var/log/ceph/ceph-client.admin.53084.log: 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59770.log: 87.9% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53786.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51715.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5%gzip -5 --verbose/var/log/ceph/ceph.log: -- /var/log/ceph/ceph-client.admin.47124.log 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42427.log: -- replaced with /var/log/ceph/ceph-client.admin.45815.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 87.9% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59770.log.gz -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55130.log: -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56544.log: /var/log/ceph/ceph-client.admin.51894.log: /var/log/ceph/ceph-client.admin.55620.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55130.log.gz 2024-04-26T22:11:17.211 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46871.log 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51894.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55620.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56544.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58798.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46319.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40631.log: /var/log/ceph/ceph-client.admin.45899.log: /var/log/ceph/ceph-client.admin.52789.log: /var/log/ceph/ceph-client.admin.47124.log: gzip/var/log/ceph/ceph-client.admin.56417.log: /var/log/ceph/ceph-client.admin.45131.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58798.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58739.log: -5 --verbose/var/log/ceph/ceph-osd.2.log: --/var/log/ceph/ceph-client.admin.53945.log: /var/log/ceph/ceph-client.admin.43296.log 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9%/var/log/ceph/ceph-client.admin.46871.log: -- replaced with /var/log/ceph/ceph-client.admin.52789.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56417.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58739.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55191.log 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60514.log 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55337.log 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60514.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55191.log.gz 2024-04-26T22:11:17.212 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46619.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60514.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40459.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43926.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40700.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55337.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45300.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45689.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57888.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57678.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42088.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57888.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.57678.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56756.log 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57888.log.gz 2024-04-26T22:11:17.213 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42088.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.59247.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57678.log.gz 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56756.log.gz 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60274.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56480.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59247.log.gz 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60274.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42764.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60274.log.gz 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56480.log.gz 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58308.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45605.log 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44370.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58308.log.gz 2024-04-26T22:11:17.214 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59812.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47689.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47478.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59812.log.gz 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34679.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40957.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.34679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41910.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47436.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52874.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57002.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.34588.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52874.log.gz 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60358.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58351.log 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4%/var/log/ceph/ceph-client.admin.60358.log: -- replaced with /var/log/ceph/ceph-client.admin.57002.log.gz 2024-04-26T22:11:17.215 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58351.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60358.log.gz 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54818.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58351.log.gz 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34315.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46445.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.34315.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54818.log.gz 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59043.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46445.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28364.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56903.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph.tmp-client.admin.28364.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59043.log.gz 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54650.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28364.log.gz 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44496.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41528.log 2024-04-26T22:11:17.216 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56903.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54650.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44496.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gzgzip 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55834.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55379.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55834.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55834.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42087.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55379.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45647.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.3.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40631.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.52370.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56584.log 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz 2024-04-26T22:11:17.217 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45647.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46871.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60740.log 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56584.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46619.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57846.log 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56584.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40459.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60740.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59644.log 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40700.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57846.log.gz 2024-04-26T22:11:17.218 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59644.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54376.log 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51809.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59644.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59896.log 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45605.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56263.log 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47689.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54376.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47478.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40957.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41910.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59896.log.gz 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47585.log 2024-04-26T22:11:17.219 INFO:teuthology.orchestra.run.smithi135.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47436.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41528.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42088.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56263.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42762.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44496.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45647.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz -5 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53354.log 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47585.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42427.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41349.log 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42758.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42761.log.gz 2024-04-26T22:11:17.220 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44286.log 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42764.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53354.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.41349.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58182.log 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59373.log 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43296.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52916.log 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58182.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51529.log 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59373.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42419.log 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52916.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47585.log.gz 2024-04-26T22:11:17.221 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47524.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52916.log.gz 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42419.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44286.log.gz 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51529.log.gz 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57534.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60866.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57534.log.gz 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5/var/log/ceph/ceph-client.admin.60866.log: --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58140.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60866.log.gz 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47310.log: /var/log/ceph/ceph-client.admin.46745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43380.log 2024-04-26T22:11:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58140.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.42419.log.gz -5 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr: --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43380.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58140.log.gz 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46807.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47812.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56179.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.47812.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59415.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60698.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54608.log 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56179.log.gz 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59415.log.gz 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr: -5/var/log/ceph/ceph-client.admin.60698.log: --verbose 88.9% -- /var/log/ceph/ceph-osd.1.log -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-04-26T22:11:17.223 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47524.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54608.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60698.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54608.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.1.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44412.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53252.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.44412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57930.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42752.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53252.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46235.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57446.log 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57930.log.gz 2024-04-26T22:11:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54464.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57446.log.gz 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52150.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51571.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54113.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52150.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr: 87.3%/var/log/ceph/ceph-client.admin.54113.log: -- replaced with /var/log/ceph/ceph-client.admin.52150.log.gz 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41129.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54113.log.gz 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41349.log.gz 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47039.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43506.log 2024-04-26T22:11:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53987.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47039.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47750.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54334.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56798.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40587.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54334.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55949.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56798.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44412.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46067.log 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46235.log.gz 2024-04-26T22:11:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55949.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gzgzip 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55704.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41129.log.gz 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55949.log.gz 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43338.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56628.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55704.log.gz 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59728.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56628.log.gz 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54420.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58453.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54420.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59728.log.gz 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52318.log 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54420.log.gz 2024-04-26T22:11:17.227 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56712.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58453.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47039.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53000.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58453.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56712.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52318.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59001.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56712.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52747.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53000.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42960.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59001.log: /var/log/ceph/ceph-client.admin.52747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43590.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59001.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43506.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52747.log.gz 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42960.log: /var/log/ceph/ceph-client.admin.56365.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41178.log 2024-04-26T22:11:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42624.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56365.log.gz 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56666.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45563.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45899.log.gz 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43716.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52473.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56666.log.gz 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60190.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60106.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60190.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.46445.log.gz --verbose 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: -- /var/log/ceph/ceph-client.admin.42086.log 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52473.log.gz 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gz 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5%/var/log/ceph/ceph-client.admin.60106.log: -- replaced with /var/log/ceph/ceph-client.admin.47750.log.gz 2024-04-26T22:11:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57488.log 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40587.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46067.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42086.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60106.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43338.log.gz -- replaced with /var/log/ceph/ceph-client.admin.60190.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42960.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40501.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57488.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41178.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43590.log.gz 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53210.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42624.log.gz 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45563.log.gz 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40872.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.53210.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57576.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53210.log.gz 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40915.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42423.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43716.log.gz 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40915.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57576.log.gz 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43548.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44996.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.0.44996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53847.log 2024-04-26T22:11:17.231 INFO:teuthology.orchestra.run.smithi135.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44996.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41041.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60574.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40113.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41041.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53847.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.60574.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52066.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60574.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52190.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52066.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55523.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52190.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52066.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52190.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40332.log 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55523.log.gz 2024-04-26T22:11:17.232 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42425.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53415.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40417.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53415.log: /var/log/ceph/ceph-client.admin.40417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55088.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53415.log.gz 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44052.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55088.log.gz 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40290.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52108.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53294.log 2024-04-26T22:11:17.233 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57972.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53294.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52108.log.gz 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54292.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53294.log.gz 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45941.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47394.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57972.log.gz 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45941.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.54292.log.gz 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60782.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60148.log 2024-04-26T22:11:17.234 INFO:teuthology.orchestra.run.smithi135.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60782.log.gz 2024-04-26T22:11:17.237 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55421.log 2024-04-26T22:11:17.237 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47646.log 2024-04-26T22:11:17.237 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55421.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43380.log.gz 2024-04-26T22:11:17.237 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56030.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60148.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47646.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55421.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52274.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42426.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52274.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56030.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52705.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52274.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42426.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42086.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44328.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52603.log 2024-04-26T22:11:17.238 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52705.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40872.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40915.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41444.log 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52603.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42423.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52645.log 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43548.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52603.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42090.log 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5/var/log/ceph/ceph-client.admin.52645.log: --verbose -- /var/log/ceph/ceph-client.admin.41742.log 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41041.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42090.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52645.log.gz 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55662.log 2024-04-26T22:11:17.239 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40113.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60232.log 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55662.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56221.log 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40332.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60232.log: /var/log/ceph/ceph-client.admin.56221.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47394.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47646.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60232.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56221.log.gz 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53613.log 2024-04-26T22:11:17.240 INFO:teuthology.orchestra.run.smithi135.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40290.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.44538.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40417.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42425.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- -- replaced with /var/log/ceph/ceph-client.admin.42426.log.gz /var/log/ceph/ceph-client.admin.40375.log 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53613.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.44052.log.gz -5 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61160.log 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53613.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40375.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45941.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41444.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41742.log.gz 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-04-26T22:11:17.241 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.61160.log: /var/log/ceph/ceph-client.admin.61034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56137.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61160.log.gz 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40209.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42090.log.gz 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46913.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59169.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56137.log.gz 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45173.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41784.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45173.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59169.log.gz 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46403.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58098.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53104.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47082.log 2024-04-26T22:11:17.242 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53104.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58098.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53042.log 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59457.log 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53104.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41084.log 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40375.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53042.log.gz/var/log/ceph/ceph-client.admin.59457.log: 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42424.log 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59457.log.gz 2024-04-26T22:11:17.243 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40209.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46913.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45173.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42424.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41784.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51673.log 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46403.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47082.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41084.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42424.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61202.log 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54986.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.51673.log.gz -5 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46361.log 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.61202.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56842.log 2024-04-26T22:11:17.244 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46361.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61202.log.gz 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41486.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52958.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41486.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56842.log.gz 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46955.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46361.log.gz 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60316.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46955.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.52958.log.gz --verbose 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr: -- /var/log/ceph/ceph-client.admin.58579.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59083.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58224.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60316.log.gz 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58579.log.gz 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53477.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59083.log: /var/log/ceph/ceph-client.admin.58224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59331.log 2024-04-26T22:11:17.245 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59083.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58224.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60064.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53477.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44454.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41486.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57212.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59331.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60064.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56031.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41826.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52234.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57212.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56031.log.gz 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51487.log 2024-04-26T22:11:17.246 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59499.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46955.log.gz 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51487.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.52234.log.gz --verbose 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr: -- /var/log/ceph/ceph-client.admin.46487.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59499.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51487.log.gz 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45983.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59499.log.gz 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5/var/log/ceph/ceph-client.admin.46487.log: --verbose -- /var/log/ceph/ceph-client.admin.58014.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43884.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58537.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43884.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58014.log.gz -5 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41616.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58537.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44454.log.gz 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-04-26T22:11:17.247 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58537.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41826.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41616.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46487.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59602.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45983.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.47164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54155.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43884.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44640.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42082.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54155.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59602.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41616.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.44640.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43632.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54155.log.gz 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52022.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42082.log: /var/log/ceph/ceph-client.admin.43632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2024-04-26T22:11:17.248 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40788.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52022.log.gz 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43116.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46277.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56323.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55295.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.56323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51757.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42763.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56323.log.gz 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44160.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55295.log.gz 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42091.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51757.log.gz 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55750.log 2024-04-26T22:11:17.249 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51613.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55750.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.51613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46025.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55750.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58495.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.46025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57762.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57720.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58495.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57720.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.57762.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57308.log 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41305.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57720.log.gz 2024-04-26T22:11:17.250 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45857.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53519.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45857.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.57308.log.gz 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr: /var/log/ceph/ceph-client.admin.41700.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53519.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53519.log.gz 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55784.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54071.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44640.log.gz 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40999.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54071.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.55784.log.gz --verbose 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr: -- /var/log/ceph/ceph-client.admin.59686.log 2024-04-26T22:11:17.251 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40999.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42082.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54071.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55566.log 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43632.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54239.log 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59686.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57804.log 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56095.log 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55566.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.40788.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53740.log 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54239.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.57804.log: /var/log/ceph/ceph-client.admin.56095.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57804.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.43116.log.gz 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60442.log 2024-04-26T22:11:17.252 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42763.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56095.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40021.log 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53740.log: /var/log/ceph/ceph-client.admin.60442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44160.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53740.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40021.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42091.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59289.log 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60442.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.52515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59854.log 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46025.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59289.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60824.log 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.59854.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59289.log.gz 2024-04-26T22:11:17.253 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60022.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53168.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59854.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60022.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45857.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60824.log.gzgzip 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41700.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60022.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45394.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.53168.log: /var/log/ceph/ceph-client.admin.52832.log: gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40999.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% /var/log/ceph/ceph-client.admin.55253.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53168.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41658.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52430.log 2024-04-26T22:11:17.254 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55253.log.gz 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60614.log/var/log/ceph/ceph-client.admin.52430.log: 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40021.log.gz 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45258.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.52430.log.gz --verbose 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr: -- /var/log/ceph/ceph-client.admin.61076.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60992.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.61076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60614.log.gz 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60992.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61076.log.gz 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61118.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60992.log.gz 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.41574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.61118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45773.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.61118.log.gz 2024-04-26T22:11:17.255 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58266.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60656.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.45350.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46807.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55481.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58266.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58959.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.55481.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60656.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47812.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55481.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/ceph-client.admin.58959.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.40543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42428.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58959.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45394.log.gz 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60950.log 2024-04-26T22:11:17.256 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42413.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58056.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60950.log.gz 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54029.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58056.log: -- /var/log/ceph/ceph-client.admin.43049.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.54029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58875.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58056.log.gz 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54029.log.gz 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.58875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.44094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60908.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58875.log.gz 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.60908.log: /var/log/ceph/ceph-client.admin.47226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2024-04-26T22:11:17.257 INFO:teuthology.orchestra.run.smithi135.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60908.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.42760.log: /var/log/ceph/ceph-client.admin.54504.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41658.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42428.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42413.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43049.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-04-26T22:11:17.258 INFO:teuthology.orchestra.run.smithi135.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-04-26T22:11:17.259 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34391.log.gz 2024-04-26T22:11:17.259 INFO:teuthology.orchestra.run.smithi139.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T22:11:17.348 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34679.log.gz 2024-04-26T22:11:17.378 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T22:11:17.396 INFO:teuthology.orchestra.run.smithi135.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-04-26T22:11:17.414 INFO:teuthology.orchestra.run.smithi135.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34315.log.gz 2024-04-26T22:11:17.467 INFO:teuthology.orchestra.run.smithi135.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T22:11:18.289 INFO:teuthology.orchestra.run.smithi139.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T22:11:18.440 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T22:11:18.682 INFO:teuthology.orchestra.run.smithi135.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T22:11:26.975 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:11:27.139 INFO:teuthology.orchestra.run.smithi135.stderr:Error initializing cluster client: ObjectNotFound('RADOS object not found (error calling conf_read_file)') 2024-04-26T22:11:27.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:11:27.143 ERROR:tasks.mds_thrash.fs.[a]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/mds_thrash.py", line 124, in _run self.do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/mds_thrash.py", line 247, in do_thrash status = self.fs.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/filesystem.py", line 407, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi135 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-26T22:11:32.028 INFO:tasks.daemonwatchdog.daemon_watchdog:thrasher.fs.[a] failed 2024-04-26T22:11:32.028 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-26T22:11:32.029 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-26T22:11:32.029 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:11:32.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17570 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-26T22:11:32.048 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:11:32.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:11:32.049 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T22:11:32.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:11:32.108 INFO:teuthology.orchestra.run.smithi135.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-26T22:11:32.108 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-26T22:11:32.108 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:11:32.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17808 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-26T22:11:32.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:11:32.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:11:32.124 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T22:11:32.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:11:32.180 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-26T22:11:33.497 INFO:teuthology.orchestra.run.smithi135.stderr: 96.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T22:11:35.140 INFO:teuthology.orchestra.run.smithi135.stderr: 96.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T22:11:48.083 INFO:teuthology.orchestra.run.smithi135.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T22:11:49.010 INFO:teuthology.orchestra.run.smithi135.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T22:11:53.871 INFO:teuthology.orchestra.run.smithi135.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T22:12:05.393 INFO:teuthology.orchestra.run.smithi135.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T22:12:05.394 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-04-26T22:12:05.394 INFO:teuthology.orchestra.run.smithi135.stderr:real 0m48.378s 2024-04-26T22:12:05.395 INFO:teuthology.orchestra.run.smithi135.stderr:user 2m52.850s 2024-04-26T22:12:05.395 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m6.094s 2024-04-26T22:12:20.839 INFO:teuthology.orchestra.run.smithi139.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T22:12:21.894 INFO:teuthology.orchestra.run.smithi139.stderr: 95.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T22:12:25.367 INFO:teuthology.orchestra.run.smithi139.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T22:12:25.867 INFO:teuthology.orchestra.run.smithi139.stderr: 95.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T22:12:45.743 INFO:teuthology.orchestra.run.smithi139.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T22:12:51.082 INFO:teuthology.orchestra.run.smithi139.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T22:12:51.084 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-26T22:12:51.084 INFO:teuthology.orchestra.run.smithi139.stderr:real 1m34.032s 2024-04-26T22:12:51.084 INFO:teuthology.orchestra.run.smithi139.stderr:user 2m47.713s 2024-04-26T22:12:51.084 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m6.320s 2024-04-26T22:12:51.084 INFO:tasks.ceph:Archiving logs... 2024-04-26T22:12:51.085 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/remote/smithi135/log 2024-04-26T22:12:51.086 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:13:14.739 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/remote/smithi139/log 2024-04-26T22:13:14.740 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:13:38.769 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T22:13:38.779 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T22:13:38.779 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:13:38.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:13:38.880 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T22:13:38.889 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T22:13:38.889 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:13:38.892 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:13:38.907 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:13:38.909 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:13:38.914 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:13:38.914 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-04-26T22:13:38.914 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +1497us[+1489us] +/- 44ms 2024-04-26T22:13:38.914 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 +86us[ +86us] +/- 60ms 2024-04-26T22:13:38.915 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 26 -2410us[-2410us] +/- 71ms 2024-04-26T22:13:38.915 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:13:38.928 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:13:38.928 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-04-26T22:13:38.928 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1487us[+1468us] +/- 44ms 2024-04-26T22:13:38.928 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +68us[ +68us] +/- 60ms 2024-04-26T22:13:38.928 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -2450us[-2469us] +/- 71ms 2024-04-26T22:13:38.928 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:13:38.929 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T22:13:38.939 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T22:13:38.939 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T22:13:38.948 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:13:39.073 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:13:39.160 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:13:39.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:13:39.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:13:39.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:13:39.219 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T22:13:39.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T22:13:39.239 INFO:teuthology.task.internal:Duration was 1212.580778 seconds 2024-04-26T22:13:39.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T22:13:39.249 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T22:13:39.249 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:13:39.252 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:13:39.304 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:13:39.311 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:13:39.718 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T22:13:39.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-04-26T22:13:39.718 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:13:39.739 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-04-26T22:13:39.740 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:13:39.771 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T22:13:39.772 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:13:39.782 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:13:39.839 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T22:13:39.839 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:13:39.848 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:13:39.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T22:13:40.001 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T22:13:40.001 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:13:40.037 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:13:40.076 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T22:13:40.087 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:13:40.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:13:40.131 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-04-26T22:13:40.152 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-04-26T22:13:40.165 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:13:40.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:13:40.199 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:13:40.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:13:40.219 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T22:13:40.230 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T22:13:40.231 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/remote/smithi135 2024-04-26T22:13:40.231 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:13:40.273 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675239/remote/smithi139 2024-04-26T22:13:40.273 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:13:40.306 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T22:13:40.306 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:13:40.309 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:13:40.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T22:13:40.367 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T22:13:40.367 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T22:13:40.376 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T22:13:40.376 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:13:40.379 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:13:40.391 INFO:teuthology.orchestra.run.smithi135.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:13 /home/ubuntu/cephtest 2024-04-26T22:13:40.409 INFO:teuthology.orchestra.run.smithi139.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:13 /home/ubuntu/cephtest 2024-04-26T22:13:40.410 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T22:13:40.421 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T22:13:40.444 INFO:teuthology.run:Summary data: description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} duration: 1212.5807783603668 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T22:13:40.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T22:13:40.530 INFO:teuthology.run:pass